Package

org.apache.spark.sql

cassandra

Permalink

package cassandra

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. cassandra
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Type Members

  1. case class AnalyzedPredicates(handledByCassandra: Set[Filter], handledBySpark: Set[Filter]) extends Product with Serializable

    Permalink
  2. class BasicCassandraPredicatePushDown[Predicate] extends AnyRef

    Permalink

    Determines which filter predicates can be pushed down to Cassandra.

    Determines which filter predicates can be pushed down to Cassandra.

    1. Only push down no-partition key column predicates with =, >, <, >=, <= predicate 2. Only push down primary key column predicates with = or IN predicate. 3. If there are regular columns in the pushdown predicates, they should have at least one EQ expression on an indexed column and no IN predicates. 4. All partition column predicates must be included in the predicates to be pushed down, only the last part of the partition key can be an IN predicate. For each partition column, only one predicate is allowed. 5. For cluster column predicates, only last predicate can be non-EQ predicate including IN predicate, and preceding column predicates must be EQ predicates. If there is only one cluster column predicate, the predicates could be any non-IN predicate. 6. There is no pushdown predicates if there is any OR condition or NOT IN condition. 7. We're not allowed to push down multiple predicates for the same column if any of them is equality or IN predicate.

    The list of predicates to be pushed down is available in predicatesToPushDown property. The list of predicates that cannot be pushed down is available in predicatesToPreserve property.

  3. trait CassandraPredicateRules extends AnyRef

    Permalink
  4. implicit final class CassandraSQLContextFunctions extends AnyVal

    Permalink
  5. final class CassandraSQLRow extends GettableData with Row with Serializable

    Permalink
  6. case class CassandraSourceOptions(pushdown: Boolean = true, confirmTruncate: Boolean = false, cassandraConfs: Map[String, String] = Map.empty) extends Product with Serializable

    Permalink

    Store data source options

  7. implicit final class CassandraSparkSessionFunctions extends AnyVal

    Permalink
  8. implicit final class DataFrameReaderWrapper extends AnyVal

    Permalink
  9. implicit final class DataFrameWriterWrapper[T] extends AnyVal

    Permalink
  10. class DefaultSource extends RelationProvider with SchemaRelationProvider with CreatableRelationProvider with Logging

    Permalink

    Cassandra data source extends RelationProvider, SchemaRelationProvider and CreatableRelationProvider.

    Cassandra data source extends RelationProvider, SchemaRelationProvider and CreatableRelationProvider. It's used internally by Spark SQL to create Relation for a table which specifies the Cassandra data source e.g.

    CREATE TEMPORARY TABLE tmpTable USING org.apache.spark.sql.cassandra OPTIONS ( table "table", keyspace "keyspace", cluster "test_cluster", pushdown "true", spark.cassandra.input.fetch.size_in_rows "10", spark.cassandra.output.consistency.level "ONE", spark.cassandra.connection.timeout_ms "1000" )

  11. trait PredicateOps[Predicate] extends AnyRef

    Permalink

    A unified API for predicates, used by BasicCassandraPredicatePushDown.

    A unified API for predicates, used by BasicCassandraPredicatePushDown.

    Keeps all the Spark-specific stuff out of BasicCassandraPredicatePushDown It is also easy to plug-in custom predicate implementations for unit-testing.

  12. case class TableRef(table: String, keyspace: String, cluster: Option[String] = None) extends Product with Serializable

    Permalink

    Store table name, keyspace name and option cluster name, keyspace is equivalent to database

Value Members

  1. val CassandraFormat: String

    Permalink

    A data frame format used to access Cassandra through Connector

  2. object CassandraSQLContextParams

    Permalink
  3. object CassandraSQLRow extends Serializable

    Permalink
  4. object CassandraSourceRelation

    Permalink
  5. object DataTypeConverter extends Logging

    Permalink

    Convert Cassandra data type to Catalyst data type

  6. object DefaultSource

    Permalink
  7. object PredicateOps

    Permalink

    Provides PredicateOps adapters for Expression and Filter classes

  8. def cassandraOptions(table: String, keyspace: String, cluster: String = ..., pushdownEnable: Boolean = true): Map[String, String]

    Permalink

    Returns a map of options which configure the path to Cassandra table as well as whether pushdown is enabled or not

Inherited from AnyRef

Inherited from Any

Ungrouped