Package

org.apache.spark.sql

cassandra

Permalink

package cassandra

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. cassandra
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Type Members

  1. case class AnalyzedPredicates(handledByCassandra: Set[Filter], handledBySpark: Set[Filter]) extends Product with Serializable

    Permalink
  2. case class Auto(ratio: Double) extends DseSearchOptimizationSetting with Product with Serializable

    Permalink
  3. class BasicCassandraPredicatePushDown[Predicate] extends AnyRef

    Permalink

    Determines which filter predicates can be pushed down to Cassandra.

    Determines which filter predicates can be pushed down to Cassandra.

    1. Only push down no-partition key column predicates with =, >, <, >=, <= predicate 2. Only push down primary key column predicates with = or IN predicate. 3. If there are regular columns in the pushdown predicates, they should have at least one EQ expression on an indexed column and no IN predicates. 4. All partition column predicates must be included in the predicates to be pushed down, only the last part of the partition key can be an IN predicate. For each partition column, only one predicate is allowed. 5. For cluster column predicates, only last predicate can be non-EQ predicate including IN predicate, and preceding column predicates must be EQ predicates. If there is only one cluster column predicate, the predicates could be any non-IN predicate. 6. There is no pushdown predicates if there is any OR condition or NOT IN condition. 7. We're not allowed to push down multiple predicates for the same column if any of them is equality or IN predicate.

    The list of predicates to be pushed down is available in predicatesToPushDown property. The list of predicates that cannot be pushed down is available in predicatesToPreserve property.

  4. trait CassandraMetadataFunction extends UnaryExpression with Unevaluable

    Permalink
  5. trait CassandraPredicateRules extends AnyRef

    Permalink
  6. implicit final class CassandraSQLContextFunctions extends AnyVal

    Permalink
  7. final class CassandraSQLRow extends GettableData with Row with Serializable

    Permalink
  8. case class CassandraSourceOptions(pushdown: Boolean = true, confirmTruncate: Boolean = false, cassandraConfs: Map[String, String] = Map.empty) extends Product with Serializable

    Permalink

    Store data source options

  9. case class CassandraSourceRelation(tableRef: TableRef, userSpecifiedSchema: Option[StructType], filterPushdown: Boolean, confirmTruncate: Boolean, tableSizeInBytes: Option[Long], connector: CassandraConnector, readConf: ReadConf, writeConf: WriteConf, sparkConf: SparkConf, sqlContext: SQLContext, directJoinSetting: DirectJoinSetting = Automatic) extends BaseRelation with InsertableRelation with PrunedFilteredScan with CassandraTableDefProvider with Logging with Product with Serializable

    Permalink

    Implements BaseRelation]], InsertableRelation]] and PrunedFilteredScan]] It inserts data to and scans Cassandra table.

    Implements BaseRelation]], InsertableRelation]] and PrunedFilteredScan]] It inserts data to and scans Cassandra table. If filterPushdown is true, it pushs down some filters to CQL

  10. implicit final class CassandraSparkSessionFunctions extends AnyVal

    Permalink
  11. class CassandraStreamingSinkRelation extends Sink

    Permalink
  12. case class CassandraTTL(child: Expression) extends UnaryExpression with CassandraMetadataFunction with Product with Serializable

    Permalink
  13. trait CassandraTableDefProvider extends AnyRef

    Permalink
  14. case class CassandraWriteTime(child: Expression) extends UnaryExpression with CassandraMetadataFunction with Product with Serializable

    Permalink
  15. implicit final class DataFrameReaderWrapper extends AnyVal

    Permalink
  16. implicit final class DataFrameWriterWrapper[T] extends AnyVal

    Permalink
  17. implicit final class DataStreamWriterWrapper[T] extends AnyVal

    Permalink
  18. class DefaultSource extends RelationProvider with SchemaRelationProvider with CreatableRelationProvider with StreamSinkProvider with Logging

    Permalink

    Cassandra data source extends RelationProvider, SchemaRelationProvider and CreatableRelationProvider.

    Cassandra data source extends RelationProvider, SchemaRelationProvider and CreatableRelationProvider. It's used internally by Spark SQL to create Relation for a table which specifies the Cassandra data source e.g.

    CREATE TEMPORARY TABLE tmpTable USING org.apache.spark.sql.cassandra OPTIONS ( table "table", keyspace "keyspace", cluster "test_cluster", pushdown "true", spark.cassandra.input.fetch.sizeInRows "10", spark.cassandra.output.consistency.level "ONE", spark.cassandra.connection.timeoutMS "1000" )

  19. sealed trait DirectJoinSetting extends AnyRef

    Permalink
  20. sealed trait DseSearchOptimizationSetting extends AnyRef

    Permalink
  21. class NullableUnresolvedAttribute extends UnresolvedAttribute

    Permalink
  22. trait PredicateOps[Predicate] extends AnyRef

    Permalink

    A unified API for predicates, used by BasicCassandraPredicatePushDown.

    A unified API for predicates, used by BasicCassandraPredicatePushDown.

    Keeps all the Spark-specific stuff out of BasicCassandraPredicatePushDown It is also easy to plug-in custom predicate implementations for unit-testing.

  23. class SolrPredicateRules extends CassandraPredicateRules with Logging

    Permalink

Value Members

  1. object AlwaysOff extends DirectJoinSetting with Product with Serializable

    Permalink
  2. object AlwaysOn extends DirectJoinSetting with Product with Serializable

    Permalink
  3. object Automatic extends DirectJoinSetting with Product with Serializable

    Permalink
  4. val CassandraFormat: String

    Permalink

    A data frame format used to access Cassandra through Connector

  5. object CassandraMetaDataRule extends Rule[LogicalPlan]

    Permalink
  6. object CassandraMetadataFunction

    Permalink
  7. object CassandraSQLContextParams

    Permalink
  8. object CassandraSQLRow extends Serializable

    Permalink
  9. object CassandraSourceRelation extends Logging with Serializable

    Permalink
  10. object CassandraStreamingSinkRelation

    Permalink
  11. object DataTypeConverter extends Logging

    Permalink

    Convert Cassandra data type to Catalyst data type

  12. object DefaultSource

    Permalink
  13. object DsePredicateRules extends CassandraPredicateRules with Logging

    Permalink

    A series of pushdown rules that only apply when connecting to Datastax Enterprise

  14. object InClausePredicateRules extends CassandraPredicateRules with Logging

    Permalink
  15. object Off extends DseSearchOptimizationSetting with Product with Serializable

    Permalink
  16. object On extends DseSearchOptimizationSetting with Product with Serializable

    Permalink
  17. object PredicateOps

    Permalink

    Provides PredicateOps adapters for Expression and Filter classes

  18. object SolrConstants

    Permalink
  19. object TimeUUIDPredicateRules extends CassandraPredicateRules with Logging

    Permalink

    All non-equal predicates on a TimeUUID column are going to fail and fail in silent way.

    All non-equal predicates on a TimeUUID column are going to fail and fail in silent way. The basic issue here is that when you use a comparison on a time UUID column in C* it compares based on the Time portion of the UUID. When Spark executes this filter (unhandled behavior) it will compare lexically, this will lead to results being incorrectly filtered out of the set. As long as the range predicate is handled completely by the connector the correct result will be obtained.

  20. def cassandraOptions(table: String, keyspace: String, cluster: String = ..., pushdownEnable: Boolean = true): Map[String, String]

    Permalink

    Returns a map of options which configure the path to Cassandra table as well as whether pushdown is enabled or not

  21. package execution

    Permalink
  22. def ttl(column: String): Column

    Permalink
  23. def ttl(column: Column): Column

    Permalink
  24. def writeTime(column: String): Column

    Permalink
  25. def writeTime(column: Column): Column

    Permalink

Inherited from AnyRef

Inherited from Any

Ungrouped