org.apache.spark.sql.execution
SparkSqlAstBuilder
Related Doc:
package execution
Type to keep track of a table header: (identifier, isTemporary, ifNotExists, isExternal).
Create top level table schema.
Create top level table schema.
Create a StructType from a sequence of StructFields.
Create a StructType from a sequence of StructFields.
Create an expression from the given context.
Create an expression from the given context. This method just passes the context on to the visitor and only takes care of typing (We assume that the visitor returns an Expression here).
Determine if a plan should be explained at all.
Determine if a plan should be explained at all.
Create an AlterTableAddPartitionCommand command.
Create an AlterTableAddPartitionCommand command.
For example:
ALTER TABLE table ADD [IF NOT EXISTS] PARTITION spec [LOCATION 'loc1']
ALTER VIEW view ADD [IF NOT EXISTS] PARTITION specALTER VIEW ... ADD PARTITION ... is not supported because the concept of partitioning is associated with physical tables
Create an alias (SubqueryAlias) for a sub-query.
Create an alias (SubqueryAlias) for a sub-query. This is practically the same as visitAliasedRelation and visitNamedExpression, ANTLR4 however requires us to use 3 different hooks.
Create an alias (SubqueryAlias) for a join relation.
Create an alias (SubqueryAlias) for a join relation. This is practically the same as visitAliasedQuery and visitNamedExpression, ANTLR4 however requires us to use 3 different hooks.
Alter the query of a view.
Alter the query of a view. This creates a AlterViewAsCommand command.
For example:
ALTER VIEW [db_name.]view_name AS SELECT ...;
Create an AnalyzeTableCommand command or an AnalyzeColumnCommand command.
Create an AnalyzeTableCommand command or an AnalyzeColumnCommand command. Example SQL for analyzing table :
ANALYZE TABLE table COMPUTE STATISTICS [NOSCAN];
Example SQL for analyzing columns :
ANALYZE TABLE table COMPUTE STATISTICS FOR COLUMNS column1, column2;
Create a binary arithmetic expression.
Create a binary arithmetic expression. The following arithmetic operators are supported: - Multiplication: '*' - Division: '/' - Hive Long Division: 'DIV' - Modulo: '%' - Addition: '+' - Subtraction: '-' - Binary AND: '&' - Binary XOR - Binary OR: '|'
Create a unary arithmetic expression.
Create a unary arithmetic expression. The following arithmetic operators are supported: - Plus: '+' - Minus: '-' - Bitwise Not: '~'
Create a BigDecimal Literal expression.
Create a BigDecimal Literal expression.
Create a Long Literal expression.
Create a Long Literal expression.
Create a Boolean literal expression.
Create a Boolean literal expression.
Create a BucketSpec.
Create a BucketSpec.
Create a CacheTableCommand logical plan.
Create a CacheTableCommand logical plan.
Create a Cast expression.
Create a Cast expression.
Override the default behavior for all visit methods.
Override the default behavior for all visit methods. This will only return a non-null result when the context has only one child. This is done because there is no generic method to combine the results of the context children. In all other cases null is returned.
Create a ClearCacheCommand logical plan.
Create a ClearCacheCommand logical plan.
Create a top level StructField from a column definition.
Create a top level StructField from a column definition.
Create a StructType from a number of column definitions.
Create a StructType from a number of column definitions.
Create an UnresolvedAttribute expression.
Create an UnresolvedAttribute expression.
Create a comparison expression.
Create a comparison expression. This compares two expressions. The following comparison operators are supported: - Equal: '=' or '==' - Null-safe Equal: '<=>' - Not Equal: '<>' or '!=' - Less than: '<' - Less then or Equal: '<=' - Greater than: '>' - Greater then or Equal: '>='
Create a StructField from a column definition.
Create a StructField from a column definition.
Create a StructType from a number of column definitions.
Create a StructType from a number of column definitions.
Create a complex DataType.
Create a complex DataType. Arrays, Maps and Structures are supported.
Convert a constants list into a String sequence.
Convert a constants list into a String sequence.
Create a CreateDatabaseCommand command.
Create a CreateDatabaseCommand command.
For example:
CREATE DATABASE [IF NOT EXISTS] database_name [COMMENT database_comment] [LOCATION path] [WITH DBPROPERTIES (key1=val1, key2=val2, ...)]
Create a CatalogStorageFormat for creating tables.
Create a CatalogStorageFormat for creating tables.
Format: STORED AS ...
Create a CreateFunctionCommand command.
Create a CreateFunctionCommand command.
For example:
CREATE [TEMPORARY] FUNCTION [db_name.]function_name AS class_name
[USING JAR|FILE|ARCHIVE 'file_uri' [, JAR|FILE|ARCHIVE 'file_uri']];
Create a table, returning a CreateTable logical plan.
Create a table, returning a CreateTable logical plan.
This is not used to create datasource tables, which is handled through "CREATE TABLE ... USING ...".
Note: several features are currently not supported - temporary tables, bucketing, skewed columns and storage handlers (STORED BY).
Expected format:
CREATE [EXTERNAL] TABLE [IF NOT EXISTS] [db_name.]table_name [(col1[:] data_type [COMMENT col_comment], ...)] [COMMENT table_comment] [PARTITIONED BY (col2[:] data_type [COMMENT col_comment], ...)] [ROW FORMAT row_format] [STORED AS file_format] [LOCATION path] [TBLPROPERTIES (property_name=property_value, ...)] [AS select_statement];
Validate a create table statement and return the TableIdentifier.
Validate a create table statement and return the TableIdentifier.
Create a CreateTableLikeCommand command.
Create a CreateTableLikeCommand command.
For example:
CREATE TABLE [IF NOT EXISTS] [db_name.]table_name LIKE [other_db_name.]existing_table_name
Create a data source table, returning a CreateTable logical plan.
Create a data source table, returning a CreateTable logical plan.
Expected format:
CREATE [EXTERNAL] TABLE [IF NOT EXISTS] [db_name.]table_name USING table_provider [OPTIONS table_property_list] [PARTITIONED BY (col_name, col_name, ...)] [CLUSTERED BY (col_name, col_name, ...) [SORTED BY (col_name [ASC|DESC], ...)] INTO num_buckets BUCKETS ] [AS select_statement];
Creates a CreateTempViewUsing logical plan.
Creates a CreateTempViewUsing logical plan.
Create or replace a view.
Create or replace a view. This creates a CreateViewCommand command.
For example:
CREATE [OR REPLACE] [[GLOBAL] TEMPORARY] VIEW [IF NOT EXISTS] [db_name.]view_name [(column_name [COMMENT column_comment], ...) ] [COMMENT view_comment] [TBLPROPERTIES (property_name = property_value, ...)] AS SELECT ...;
Create a decimal literal for a regular decimal number.
Create a decimal literal for a regular decimal number.
Create a dereference expression.
Create a dereference expression. The return type depends on the type of the parent, this can either be a UnresolvedAttribute (if the parent is an UnresolvedAttribute), or an UnresolvedExtractValue if the parent is some expression.
Create a DescribeDatabaseCommand command.
Create a DescribeDatabaseCommand command.
For example:
DESCRIBE DATABASE [EXTENDED] database;
Create a plan for a DESCRIBE FUNCTION command.
Create a plan for a DESCRIBE FUNCTION command.
Create a DescribeTableCommand logical plan.
Create a DescribeTableCommand logical plan.
Create a Double Literal expression.
Create a Double Literal expression.
Create a DropDatabaseCommand command.
Create a DropDatabaseCommand command.
For example:
DROP (DATABASE|SCHEMA) [IF EXISTS] database [RESTRICT|CASCADE];
Create a DropFunctionCommand command.
Create a DropFunctionCommand command.
For example:
DROP [TEMPORARY] FUNCTION [IF EXISTS] function;
Create a DropTableCommand command.
Create a DropTableCommand command.
Create an AlterTableDropPartitionCommand command
Create an AlterTableDropPartitionCommand command
For example:
ALTER TABLE table DROP [IF EXISTS] PARTITION spec1[, PARTITION spec2, ...] [PURGE]; ALTER VIEW view DROP [IF EXISTS] PARTITION spec1[, PARTITION spec2, ...];
ALTER VIEW ... DROP PARTITION ... is not supported because the concept of partitioning is associated with physical tables
Create a filtering correlated sub-query (EXISTS).
Create a filtering correlated sub-query (EXISTS).
Create an ExplainCommand logical plan.
Create an ExplainCommand logical plan. The syntax of using this command in SQL is:
EXPLAIN (EXTENDED | CODEGEN) SELECT * FROM ...
Fail an unsupported Hive native command.
Fail an unsupported Hive native command.
Create or resolve a FrameBoundary.
Create or resolve a FrameBoundary. Simple math expressions are allowed for Value Preceding/Following boundaries. These expressions must be constant (foldable) and return an integer value.
Create a logical plan for a given 'FROM' clause.
Create a logical plan for a given 'FROM' clause. Note that we support multiple (comma separated) relations here, these get converted into a single plan by condition-less inner join.
Create a (windowed) Function expression.
Create a (windowed) Function expression.
Create a function database (optional) and name pair.
Create a function database (optional) and name pair.
Resolve a HiveSerDe based on the name given and return it as a CatalogStorageFormat.
Resolve a HiveSerDe based on the name given and return it as a CatalogStorageFormat.
Create a Sequence of Strings for a parenthesis enclosed alias list.
Create a Sequence of Strings for a parenthesis enclosed alias list.
Create a Sequence of Strings for an identifier list.
Create a Sequence of Strings for an identifier list.
Create an inline table (a virtual table in Hive parlance).
Create an inline table (a virtual table in Hive parlance).
Create an integral literal expression.
Create an integral literal expression. The code selects the most narrow integral type possible, either a BigDecimal, a Long or an Integer is returned.
Create a CalendarInterval literal expression.
Create a CalendarInterval literal expression. An interval expression can contain multiple unit value pairs, for instance: interval 2 months 2 days.
Create a CalendarInterval for a unit value pair.
Create a CalendarInterval for a unit value pair. Two unit configuration types are supported: - Single unit. - From-To unit (only 'YEAR TO MONTH' and 'DAY TO SECOND' are supported).
Create a LoadDataCommand command.
Create a LoadDataCommand command.
For example:
LOAD DATA [LOCAL] INPATH 'filepath' [OVERWRITE] INTO TABLE tablename [PARTITION (partcol1=val1, partcol2=val2 ...)]
Create location string.
Create location string.
Combine a number of boolean expressions into a balanced expression tree.
Combine a number of boolean expressions into a balanced expression tree. These expressions are either combined by a logical And or a logical Or.
A balanced binary tree is created because regular left recursive trees cause considerable performance degradations and can cause stack overflows.
Invert a boolean expression.
Invert a boolean expression.
Create a AddFileCommand, AddJarCommand, ListFilesCommand or ListJarsCommand command depending on the requested operation on resources.
Create a AddFileCommand, AddJarCommand, ListFilesCommand or ListJarsCommand command depending on the requested operation on resources. Expected format:
ADD (FILE[S] <filepath ...> | JAR[S] <jarpath ...>) LIST (FILE[S] [filepath ...] | JAR[S] [jarpath ...])
Create a logical plan which allows for multiple inserts using one 'from' statement.
Create a logical plan which allows for multiple inserts using one 'from' statement. These queries have the following SQL form:
[WITH cte...]? FROM src [INSERT INTO tbl1 SELECT *]+
For example:
FROM db.tbl1 A INSERT INTO dbo.tbl1 SELECT * WHERE A.value = 10 LIMIT 5 INSERT INTO dbo.tbl2 SELECT * WHERE A.value = 12
This (Hive) feature cannot be combined with set-operators.
Create an aliased expression if an alias is specified.
Create an aliased expression if an alias is specified. Both single and multi-aliases are supported.
Create a named logical plan.
Create a named logical plan.
This is only used for Common Table Expressions.
Convert a nested constants list into a sequence of string sequences.
Convert a nested constants list into a sequence of string sequences.
Create a partition specification map without optional values.
Create a partition specification map without optional values.
Create a NULL literal expression.
Create a NULL literal expression.
Create an expression for an expression between parentheses.
Create an expression for an expression between parentheses. This is need because the ANTLR visitor cannot automatically convert the nested context into an expression.
Create a partition specification map.
Create a partition specification map.
Create a predicated expression.
Create a predicated expression. A predicated expression is a normal expression with a predicate attached to it, for example:
a + 1 IS NULL
Resolve/create a primitive type.
Resolve/create a primitive type.
Create a top-level plan with Common Table Expressions.
Create a top-level plan with Common Table Expressions.
Create a logical plan using a query specification.
Create a logical plan using a query specification.
Create an AlterTableRecoverPartitionsCommand command
Create an AlterTableRecoverPartitionsCommand command
For example:
ALTER TABLE table RECOVER PARTITIONS;
Create a RefreshTable logical plan.
Create a RefreshTable logical plan.
Create a RefreshTable logical plan.
Create a RefreshTable logical plan.
Create a single relation referenced in a FROM claused.
Create a single relation referenced in a FROM claused. This method is used when a part of the join condition is nested, for example:
select * from t1 join (t2 cross join t3) on col1 = col2
Create a AlterTableRenameCommand command.
Create a AlterTableRenameCommand command.
For example:
ALTER TABLE table1 RENAME TO table2; ALTER VIEW view1 RENAME TO view2;
Create an AlterTableRenamePartitionCommand command
Create an AlterTableRenamePartitionCommand command
For example:
ALTER TABLE table PARTITION spec1 RENAME TO PARTITION spec2;
Create a AlterTableRecoverPartitionsCommand command.
Create a AlterTableRecoverPartitionsCommand command.
For example:
MSCK REPAIR TABLE tablename
Create a ResetCommand logical plan.
Create a ResetCommand logical plan. Example SQL :
RESET;
Create a CreateStruct expression.
Create a CreateStruct expression.
Create a delimited row format properties object.
Create a delimited row format properties object.
Create SERDE row format name and properties pair.
Create SERDE row format name and properties pair.
Create a condition based CaseWhen expression.
Create a condition based CaseWhen expression. This has the following SQL syntax:
CASE WHEN [predicate] THEN [expression] ... ELSE [expression] END
the parse tree
Create a SetCommand logical plan.
Create a SetCommand logical plan.
Note that we assume that everything after the SET keyword is assumed to be a part of the
key-value pair. The split between key and value is made by searching for the first =
character in the raw string.
Create an AlterDatabasePropertiesCommand command.
Create an AlterDatabasePropertiesCommand command.
For example:
ALTER (DATABASE|SCHEMA) database SET DBPROPERTIES (property_name=property_value, ...);
Connect two queries by a Set operator.
Connect two queries by a Set operator.
Supported Set operators are: - UNION [DISTINCT] - UNION ALL - EXCEPT [DISTINCT] - MINUS [DISTINCT] - INTERSECT [DISTINCT]
Create an AlterTableSetLocationCommand command
Create an AlterTableSetLocationCommand command
For example:
ALTER TABLE table [PARTITION spec] SET LOCATION "loc";
Create an AlterTableSetPropertiesCommand command.
Create an AlterTableSetPropertiesCommand command.
For example:
ALTER TABLE table SET TBLPROPERTIES ('comment' = new_comment);
ALTER VIEW view SET TBLPROPERTIES ('comment' = new_comment);
Create an AlterTableSerDePropertiesCommand command.
Create an AlterTableSerDePropertiesCommand command.
For example:
ALTER TABLE table [PARTITION spec] SET SERDE serde_name [WITH SERDEPROPERTIES props]; ALTER TABLE table [PARTITION spec] SET SERDEPROPERTIES serde_properties;
A command for users to list the column names for a table.
A command for users to list the column names for a table. This function creates a ShowColumnsCommand logical plan.
The syntax of using this command in SQL is:
SHOW COLUMNS (FROM | IN) table_identifier [(FROM | IN) database];
Creates a ShowCreateTableCommand
Creates a ShowCreateTableCommand
Create a ShowDatabasesCommand logical plan.
Create a ShowDatabasesCommand logical plan. Example SQL:
SHOW (DATABASES|SCHEMAS) [LIKE 'identifier_with_wildcards'];
Create a plan for a SHOW FUNCTIONS command.
Create a plan for a SHOW FUNCTIONS command.
A command for users to list the partition names of a table.
A command for users to list the partition names of a table. If partition spec is specified, partitions that match the spec are returned. Otherwise an empty result set is returned.
This function creates a ShowPartitionsCommand logical plan
The syntax of using this command in SQL is:
SHOW PARTITIONS table_identifier [partition_spec];
Create a ShowTablesCommand logical plan.
Create a ShowTablesCommand logical plan. Example SQL :
SHOW TABLES [(IN|FROM) database_name] [[LIKE] 'identifier_with_wildcards'];
A command for users to list the properties for a table.
A command for users to list the properties for a table. If propertyKey is specified, the value for the propertyKey is returned. If propertyKey is not specified, all the keys and their corresponding values are returned. The syntax of using this command in SQL is:
SHOW TBLPROPERTIES table_name[('propertyKey')];
Create a value based CaseWhen expression.
Create a value based CaseWhen expression. This has the following SQL form:
CASE [expression] WHEN [value] THEN [expression] ... ELSE [expression] END
Create a logical plan for a regular (single-insert) query.
Create a logical plan for a regular (single-insert) query.
Create a Short Literal expression.
Create a Short Literal expression.
Create a SortOrder expression.
Create a SortOrder expression.
Create a star (i.e.
Create a star (i.e. all) expression; this selects all elements (in the specified object). Both un-targeted (global) and targeted aliases are supported.
Convert a constant of any type into a string.
Convert a constant of any type into a string. This is typically used in DDL commands, and its main purpose is to prevent slight differences due to back to back conversions i.e.: String -> Literal -> String.
Create a String literal expression.
Create a String literal expression.
Create a logical plan for a sub-query.
Create a logical plan for a sub-query.
Create a ScalarSubquery expression.
Create a ScalarSubquery expression.
Create an UnresolvedExtractValue expression, this is used for subscript access to an array.
Create an UnresolvedExtractValue expression, this is used for subscript access to an array.
Create an un-aliased table reference.
Create an un-aliased table reference. This is typically used for top-level table references, for example:
INSERT INTO db.tbl2 TABLE db.tbl1
Create a CatalogStorageFormat.
Create a CatalogStorageFormat.
Create a TableIdentifier from a 'tableName' or 'databaseName'.'tableName' pattern.
Create a TableIdentifier from a 'tableName' or 'databaseName'.'tableName' pattern.
Create an aliased table reference.
Create an aliased table reference. This is typically used in FROM clauses.
A table property key can either be String or a collection of dot separated elements.
A table property key can either be String or a collection of dot separated elements. This function extracts the property key based on whether its a string literal or a table property identifier.
Convert a table property list into a key-value map.
Convert a table property list into a key-value map. This should be called through visitPropertyKeyValues or visitPropertyKeys.
A table property value can be String, Integer, Boolean or Decimal.
A table property value can be String, Integer, Boolean or Decimal. This function extracts the property value based on whether its a string, integer, boolean or decimal literal.
Create a table-valued function call with arguments, e.g.
Create a table-valued function call with arguments, e.g. range(1000)
Create a current timestamp/date expression.
Create a current timestamp/date expression. These are different from regular function because they do not require the user to specify braces when calling them.
Create a Byte Literal expression.
Create a Byte Literal expression.
Create a TruncateTableCommand command.
Create a TruncateTableCommand command.
For example:
TRUNCATE TABLE tablename [PARTITION (partcol1=val1, partcol2=val2 ...)]
Create a typed Literal expression.
Create a typed Literal expression. A typed literal has the following SQL syntax:
[TYPE] '[VALUE]'Currently Date, Timestamp and Binary typed literals are supported.
Create an UncacheTableCommand logical plan.
Create an UncacheTableCommand logical plan.
Create an AlterTableUnsetPropertiesCommand command.
Create an AlterTableUnsetPropertiesCommand command.
For example:
ALTER TABLE table UNSET TBLPROPERTIES [IF EXISTS] ('comment', 'key');
ALTER VIEW view UNSET TBLPROPERTIES [IF EXISTS] ('comment', 'key');
Create a SetDatabaseCommand logical plan.
Create a SetDatabaseCommand logical plan.
Create a window definition, i.e.
Create a window definition, i.e. WindowSpecDefinition.
Create a reference to a window frame, i.e.
Create a reference to a window frame, i.e. WindowSpecReference.
Create a ScriptInputOutputSchema.
Create a ScriptInputOutputSchema.
Builder that converts an ANTLR ParseTree into a LogicalPlan/Expression/TableIdentifier.