sources

Type Members

1. trait AlterableRelation extends AnyRef

   

   Annotations

   @DeveloperApi()

2. case class And(left: Filter, right: Filter) extends Filter with Product with Serializable

   

   A filter that evaluates to true iff both left or right evaluate to true.

   A filter that evaluates to true iff both left or right evaluate to true.

   Annotations

   @Stable()

   Since

   1.3.0

3. abstract class BaseRelation extends AnyRef

   

   Represents a collection of tuples with a known schema.

   Represents a collection of tuples with a known schema. Classes that extend BaseRelation must be able to produce the schema of their data in the form of a StructType. Concrete implementation should inherit from one of the descendant Scan classes, which define various abstract methods for execution.

   BaseRelations must also define an equality function that only returns true when the two instances will return the same data. This equality function is used when determining when it is safe to substitute cached results for a given relation.

   Annotations

   @Stable()

   Since

   1.3.0

4. trait BulkPutRelation extends BaseRelation with NativeTableRelation

   

5. trait CastDouble extends AnyRef

   

   Optimized cast for a column in a row to double.

6. trait CastLongTime extends AnyRef

   

   Cast a given column in a schema to epoch time in long milliseconds.

7. trait CatalystScan extends AnyRef

   

   ::Experimental:: An interface for experimenting with a more direct connection to the query planner.

   ::Experimental:: An interface for experimenting with a more direct connection to the query planner. Compared to PrunedFilteredScan, this operator receives the raw expressions from the org.apache.spark.sql.catalyst.plans.logical.LogicalPlan. Unlike the other APIs this interface is NOT designed to be binary compatible across releases and thus should only be used for experimentation.

   Annotations

   @Experimental() @Unstable()

   Since

   1.3.0

8. case class CompletePlan(plan: LogicalPlan, replaced: Seq[Replacement]) extends SubPlan with Product with Serializable

   

9. case class ConnectionProperties(url: String, driver: String, dialect: JdbcDialect, poolProps: Map[String, String], connProps: Properties, executorConnProps: Properties, hikariCP: Boolean) extends Product with Serializable

   

10. trait CreatableRelationProvider extends AnyRef

    
    Annotations

    @Stable()

    Since

    1.3.0

11. trait DataSourceRegister extends AnyRef

    

    Data sources should implement this trait so that they can register an alias to their data source.

    Data sources should implement this trait so that they can register an alias to their data source. This allows users to give the data source alias as the format type over the fully qualified class name.

    A new instance of this class will be instantiated each time a DDL call is made.

    Annotations

    @Stable()

    Since

    1.5.0

12. trait DeletableRelation extends BaseRelation with MutableRelation

    

    Annotations

    @DeveloperApi()

13. case class Delete(table: LogicalPlan, child: LogicalPlan, keyColumns: Seq[Attribute]) extends LogicalPlan with TableMutationPlan with Product with Serializable

    

    Plan for delete from a column or row table.

14. trait DestroyRelation extends BaseRelation

    

    Annotations

    @DeveloperApi()

15. case class EqualNullSafe(attribute: String, value: Any) extends Filter with Product with Serializable

    

    Performs equality comparison, similar to EqualTo.

    Performs equality comparison, similar to EqualTo. However, this differs from EqualTo in that it returns true (rather than NULL) if both inputs are NULL, and false (rather than NULL) if one of the input is NULL and the other is not NULL.

    Annotations

    @Stable()

    Since

    1.5.0

16. case class EqualTo(attribute: String, value: Any) extends Filter with Product with Serializable

    

    A filter that evaluates to true iff the attribute evaluates to a value equal to value.

    A filter that evaluates to true iff the attribute evaluates to a value equal to value.

    Annotations

    @Stable()

    Since

    1.3.0

17. trait ExternalSchemaRelationProvider extends RelationProvider

    

    ::DeveloperApi:: Marker interface for data sources that allow for extended schema specification in CREATE TABLE (like constraints in RDBMS databases).

    ::DeveloperApi:: Marker interface for data sources that allow for extended schema specification in CREATE TABLE (like constraints in RDBMS databases). The schema string is passed as SnappyExternalCatalog.SCHEMADDL_PROPERTY in the relation provider parameters.

    Annotations

    @DeveloperApi()

18. case class ExternalTableDMLCmd(storeRelation: LogicalRelation, command: String, childOutput: Seq[Attribute]) extends LeafNode with RunnableCommand with TableMutationPlan with Product with Serializable

    

19. abstract class Filter extends AnyRef

    

    A filter predicate for data sources.

    A filter predicate for data sources.

    Annotations

    @Stable()

    Since

    1.3.0

20. case class GreaterThan(attribute: String, value: Any) extends Filter with Product with Serializable

    

    A filter that evaluates to true iff the attribute evaluates to a value greater than value.

    A filter that evaluates to true iff the attribute evaluates to a value greater than value.

    Annotations

    @Stable()

    Since

    1.3.0

21. case class GreaterThanOrEqual(attribute: String, value: Any) extends Filter with Product with Serializable

    

    A filter that evaluates to true iff the attribute evaluates to a value greater than or equal to value.

    A filter that evaluates to true iff the attribute evaluates to a value greater than or equal to value.

    Annotations

    @Stable()

    Since

    1.3.0

22. case class In(attribute: String, values: Array[Any]) extends Filter with Product with Serializable

    

    A filter that evaluates to true iff the attribute evaluates to one of the values in the array.

    A filter that evaluates to true iff the attribute evaluates to one of the values in the array.

    Annotations

    @Stable()

    Since

    1.3.0

23. trait IndexableRelation extends AnyRef

    

    Annotations

    @DeveloperApi()

24. final class Insert extends InsertIntoTable

    

    Unlike Spark's InsertIntoTable this plan provides the count of rows inserted as the output.

25. trait InsertableRelation extends AnyRef

    

    A BaseRelation that can be used to insert data into it through the insert method.

    A BaseRelation that can be used to insert data into it through the insert method. If overwrite in insert method is true, the old data in the relation should be overwritten with the new data. If overwrite in insert method is false, the new data should be appended.

    InsertableRelation has the following three assumptions. 1. It assumes that the data (Rows in the DataFrame) provided to the insert method exactly matches the ordinal of fields in the schema of the BaseRelation. 2. It assumes that the schema of this relation will not be changed. Even if the insert method updates the schema (e.g. a relation of JSON or Parquet data may have a schema update after an insert operation), the new schema will not be used. 3. It assumes that fields of the data provided in the insert method are nullable. If a data source needs to check the actual nullability of a field, it needs to do it in the insert method.

    Annotations

    @Stable()

    Since

    1.3.0

26. case class IsNotNull(attribute: String) extends Filter with Product with Serializable

    

    A filter that evaluates to true iff the attribute evaluates to a non-null value.

    A filter that evaluates to true iff the attribute evaluates to a non-null value.

    Annotations

    @Stable()

    Since

    1.3.0

27. case class IsNull(attribute: String) extends Filter with Product with Serializable

    

    A filter that evaluates to true iff the attribute evaluates to null.

    A filter that evaluates to true iff the attribute evaluates to null.

    Annotations

    @Stable()

    Since

    1.3.0

28. abstract class JdbcExtendedDialect extends JdbcDialect

    

    Some extensions to JdbcDialect used by Snappy implementation.

29. abstract class JoinOrderStrategy extends PredicateHelper

    

    Trait to apply different join order policies like Replicates with filters first, then largest colocated group, and finally non-colocated with filters, if any.

    Trait to apply different join order policies like Replicates with filters first, then largest colocated group, and finally non-colocated with filters, if any.

    One can change the ordering policies as part of query hints and later can be admin provided externally against a regex based query pattern.

    e.g. select * from /*+ joinOrder(replicates+filters, non-colocated+filters) */ table1, table2 where ....

    note: I think this should be at the query level instead of per select scope i.e. something like /*+ joinOrder(replicates+filters, non-colocated+filters) */ select * from tab1, (select xx from tab2, tab3 where ... ), tab4 where ...

30. case class LessThan(attribute: String, value: Any) extends Filter with Product with Serializable

    

    A filter that evaluates to true iff the attribute evaluates to a value less than value.

    A filter that evaluates to true iff the attribute evaluates to a value less than value.

    Annotations

    @Stable()

    Since

    1.3.0

31. case class LessThanOrEqual(attribute: String, value: Any) extends Filter with Product with Serializable

    

    A filter that evaluates to true iff the attribute evaluates to a value less than or equal to value.

    A filter that evaluates to true iff the attribute evaluates to a value less than or equal to value.

    Annotations

    @Stable()

    Since

    1.3.0

32. class Masquerader extends Serializable

    

33. trait MutableRelation extends BaseRelation with NativeTableRelation

    

    ::DeveloperApi

    ::DeveloperApi

    API for updates and deletes to a relation.

    Annotations

    @DeveloperApi()

34. trait NativeTableRelation extends BaseRelation with DestroyRelation

    

    Annotations

    @DeveloperApi()

35. trait NativeTableRowLevelSecurityRelation extends BaseRelation with NativeTableRelation with RowLevelSecurityRelation

    

    Annotations

    @DeveloperApi()

36. case class Not(child: Filter) extends Filter with Product with Serializable

    

    A filter that evaluates to true iff child is evaluated to false.

    A filter that evaluates to true iff child is evaluated to false.

    Annotations

    @Stable()

    Since

    1.3.0

37. case class Or(left: Filter, right: Filter) extends Filter with Product with Serializable

    

    A filter that evaluates to true iff at least one of left or right evaluates to true.

    A filter that evaluates to true iff at least one of left or right evaluates to true.

    Annotations

    @Stable()

    Since

    1.3.0

38. case class PartialPlan(curPlan: LogicalPlan, replaced: Seq[Replacement], outputSet: AttributeSet, input: Seq[LogicalPlan], conditions: Seq[Expression], colocatedGroups: Seq[ReplacementSet], partitioned: Seq[LogicalPlan], replicates: Seq[LogicalPlan], others: Seq[LogicalPlan]) extends SubPlan with Product with Serializable

    

39. trait PlanInsertableRelation extends BaseRelation with NativeTableRelation with InsertableRelation

    

40. trait PrunedFilteredScan extends AnyRef

    

    A BaseRelation that can eliminate unneeded columns and filter using selected predicates before producing an RDD containing all matching tuples as Row objects.

    A BaseRelation that can eliminate unneeded columns and filter using selected predicates before producing an RDD containing all matching tuples as Row objects.

    The actual filter should be the conjunction of all filters, i.e. they should be "and" together.

    The pushed down filters are currently purely an optimization as they will all be evaluated again. This means it is safe to use them with methods that produce false positives such as filtering partitions based on a bloom filter.

    Annotations

    @Stable()

    Since

    1.3.0

41. trait PrunedScan extends AnyRef

    

    A BaseRelation that can eliminate unneeded columns before producing an RDD containing all of its tuples as Row objects.

    A BaseRelation that can eliminate unneeded columns before producing an RDD containing all of its tuples as Row objects.

    Annotations

    @Stable()

    Since

    1.3.0

42. trait PrunedUnsafeFilteredScan extends AnyRef

    

    ::DeveloperApi:: A BaseRelation that can eliminate unneeded columns and filter using selected predicates before producing an RDD containing all matching tuples as Unsafe Row objects.

    ::DeveloperApi:: A BaseRelation that can eliminate unneeded columns and filter using selected predicates before producing an RDD containing all matching tuples as Unsafe Row objects.

    The actual filter should be the conjunction of all filters, i.e. they should be "and" together.

    The pushed down filters are currently purely an optimization as they will all be evaluated again. This means it is safe to use them with methods that produce false positives such as filtering partitions based on a bloom filter.

    Annotations

    @DeveloperApi()

    Since

    1.3.0

43. case class PutIntoTable(table: LogicalPlan, child: LogicalPlan) extends LogicalPlan with TableMutationPlan with Product with Serializable

    

44. trait RelationProvider extends AnyRef

    

    Implemented by objects that produce relations for a specific kind of data source.

    Implemented by objects that produce relations for a specific kind of data source. When Spark SQL is given a DDL operation with a USING clause specified (to specify the implemented RelationProvider), this interface is used to pass in the parameters specified by a user.

    Users may specify the fully qualified class name of a given data source. When that class is not found Spark SQL will append the class name DefaultSource to the path, allowing for less verbose invocation. For example, 'org.apache.spark.sql.json' would resolve to the data source 'org.apache.spark.sql.json.DefaultSource'

    A new instance of this class will be instantiated each time a DDL call is made.

    Annotations

    @Stable()

    Since

    1.3.0

45. case class Replacement(table: TABLE, index: INDEX, isPartitioned: Boolean = true) extends PredicateHelper with Product with Serializable

    

    Table to table or Table to index replacement.

46. case class ReplacementSet(chain: ArrayBuffer[Replacement], conditions: Seq[Expression]) extends Ordered[ReplacementSet] with PredicateHelper with Product with Serializable

    

    A set of possible replacements of table to indexes.

    A set of possible replacements of table to indexes.
    Note: The chain if consists of multiple partitioned tables, they must satisfy colocation criteria.

    chain

    Multiple replacements.

    conditions

    user provided join + filter conditions.

47. case class ResolveIndex()(implicit snappySession: SnappySession) extends Rule[LogicalPlan] with PredicateHelper with Product with Serializable

    

    Replace table with index if colocation criteria is satisfied.

48. case class ResolveQueryHints(snappySession: SnappySession) extends Rule[LogicalPlan] with Product with Serializable

    

    Replace table with index hint

49. trait RowInsertableRelation extends BaseRelation with NativeTableRelation

    

    Annotations

    @DeveloperApi()

50. trait RowLevelSecurityRelation extends AnyRef

    

51. trait RowPutRelation extends BaseRelation with NativeTableRelation

    

52. case class SampleTableQuery(schemaFromExecutedPlan: Option[QueryExecution], child: LogicalPlan, error: Double, confidence: Double, sampleRatioAttribute: Masquerader, bootstrapMultiplicities: Option[Attribute], isAQPDebug: Boolean, isFixedSeed: Boolean, analysisType: Option[Type], bsMultiplictyAliasID: ExprId, numBSTrials: Int, behavior: Type, sampleTable: String, ruleNumProcessing: Int = 1, makeVisible: Boolean, isDistributed: Boolean, logicalPlanForQueryRouting: Option[LogicalPlan]) extends SnappyUnaryNode with Product with Serializable

    

53. trait SamplingRelation extends BaseRelation with SchemaInsertableRelation

    

    Annotations

    @DeveloperApi()

54. trait SchemaInsertableRelation extends InsertableRelation

    

    ::DeveloperApi

    ::DeveloperApi

    An extension to InsertableRelation that allows for data to be inserted (possibily having different schema) into the target relation after comparing against the result of insertSchema.

    Annotations

    @DeveloperApi()

55. trait SchemaRelationProvider extends AnyRef

    

    Implemented by objects that produce relations for a specific kind of data source with a given schema.

    Implemented by objects that produce relations for a specific kind of data source with a given schema. When Spark SQL is given a DDL operation with a USING clause specified ( to specify the implemented SchemaRelationProvider) and a user defined schema, this interface is used to pass in the parameters specified by a user.

    Users may specify the fully qualified class name of a given data source. When that class is not found Spark SQL will append the class name DefaultSource to the path, allowing for less verbose invocation. For example, 'org.apache.spark.sql.json' would resolve to the data source 'org.apache.spark.sql.json.DefaultSource'

    A new instance of this class will be instantiated each time a DDL call is made.

    The difference between a RelationProvider and a SchemaRelationProvider is that users need to provide a schema when using a SchemaRelationProvider. A relation provider can inherits both RelationProvider and SchemaRelationProvider if it can support both schema inference and user-specified schemas.

    Annotations

    @Stable()

    Since

    1.3.0

56. final class StatCounter extends StatVarianceCounter with Serializable

    

57. trait StatVarianceCounter extends Serializable

    

    A class for tracking the statistics of a set of numbers (count, mean and variance) in a numerically robust way.

    A class for tracking the statistics of a set of numbers (count, mean and variance) in a numerically robust way. Includes support for merging two StatVarianceCounters.

    Taken from Spark's StatCounter implementation removing max and min.

58. trait StreamSinkProvider extends AnyRef

    

    ::Experimental:: Implemented by objects that can produce a streaming Sink for a specific format or system.

    ::Experimental:: Implemented by objects that can produce a streaming Sink for a specific format or system.

    Annotations

    @Experimental() @Unstable()

    Since

    2.0.0

59. trait StreamSourceProvider extends AnyRef

    

    ::Experimental:: Implemented by objects that can produce a streaming Source for a specific format or system.

    ::Experimental:: Implemented by objects that can produce a streaming Source for a specific format or system.

    Annotations

    @Experimental() @Unstable()

    Since

    2.0.0

60. case class StringContains(attribute: String, value: String) extends Filter with Product with Serializable

    

    A filter that evaluates to true iff the attribute evaluates to a string that contains the string value.

    A filter that evaluates to true iff the attribute evaluates to a string that contains the string value.

    Annotations

    @Stable()

    Since

    1.3.1

61. case class StringEndsWith(attribute: String, value: String) extends Filter with Product with Serializable

    

    A filter that evaluates to true iff the attribute evaluates to a string that starts with value.

    A filter that evaluates to true iff the attribute evaluates to a string that starts with value.

    Annotations

    @Stable()

    Since

    1.3.1

62. case class StringStartsWith(attribute: String, value: String) extends Filter with Product with Serializable

    

    A filter that evaluates to true iff the attribute evaluates to a string that starts with value.

    A filter that evaluates to true iff the attribute evaluates to a string that starts with value.

    Annotations

    @Stable()

    Since

    1.3.1

63. trait SubPlan extends AnyRef

    

64. trait TableMutationPlan extends AnyRef

    

65. trait TableScan extends AnyRef

    

    A BaseRelation that can produce all of its tuples as an RDD of Row objects.

    A BaseRelation that can produce all of its tuples as an RDD of Row objects.

    Annotations

    @Stable()

    Since

    1.3.0

66. trait UpdatableRelation extends BaseRelation with MutableRelation with NativeTableRelation

    

    Annotations

    @DeveloperApi()

67. case class Update(table: LogicalPlan, child: LogicalPlan, keyColumns: Seq[Attribute], updateColumns: Seq[Attribute], updateExpressions: Seq[Expression]) extends LogicalPlan with TableMutationPlan with Product with Serializable

    

    Plan for update of a column or row table.

    Plan for update of a column or row table. The "table" passed should be a resolved one (by parser and other callers) else there is ambiguity in column resolution of updateColumns/expressions between table and child.