p

org.apache.spark.sql

cassandra

package cassandra

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. cassandra
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Type Members

  1. case class AnalyzedPredicates(handledByCassandra: Set[Filter], handledBySpark: Set[Filter]) extends Product with Serializable
  2. case class Auto(ratio: Double) extends DseSearchOptimizationSetting with Product with Serializable
  3. class BasicCassandraPredicatePushDown[Predicate] extends AnyRef

    Determines which filter predicates can be pushed down to Cassandra.

    Determines which filter predicates can be pushed down to Cassandra.

    1. Only push down no-partition key column predicates with =, >, <, >=, <= predicate 2. Only push down primary key column predicates with = or IN predicate. 3. If there are regular columns in the pushdown predicates, they should have at least one EQ expression on an indexed column and no IN predicates. 4. All partition column predicates must be included in the predicates to be pushed down, only the last part of the partition key can be an IN predicate. For each partition column, only one predicate is allowed. 5. For cluster column predicates, only last predicate can be non-EQ predicate including IN predicate, and preceding column predicates must be EQ predicates. If there is only one cluster column predicate, the predicates could be any non-IN predicate. 6. There is no pushdown predicates if there is any OR condition or NOT IN condition. 7. We're not allowed to push down multiple predicates for the same column if any of them is equality or IN predicate.

    The list of predicates to be pushed down is available in predicatesToPushDown property. The list of predicates that cannot be pushed down is available in predicatesToPreserve property.

  4. trait CassandraMetadataFunction extends UnaryExpression with Unevaluable
  5. trait CassandraPredicateRules extends AnyRef
  6. implicit final class CassandraSQLContextFunctions extends AnyVal
  7. final class CassandraSQLRow extends GettableData with Row with Serializable
  8. case class CassandraSourceOptions(pushdown: Boolean = true, confirmTruncate: Boolean = false, cassandraConfs: Map[String, String] = Map.empty) extends Product with Serializable

    Store data source options

  9. implicit final class CassandraSparkSessionFunctions extends AnyVal
  10. case class CassandraTTL(child: Expression) extends UnaryExpression with CassandraMetadataFunction with Product with Serializable
  11. trait CassandraTableDefProvider extends AnyRef
  12. case class CassandraWriteTime(child: Expression) extends UnaryExpression with CassandraMetadataFunction with Product with Serializable
  13. implicit final class DataFrameReaderWrapper extends AnyVal
  14. implicit final class DataFrameWriterWrapper[T] extends AnyVal
  15. implicit final class DataStreamWriterWrapper[T] extends AnyVal
  16. class DefaultSource extends TableProvider with DataSourceRegister

    A Pointer to the DatasourceV2 Implementation of The Cassandra Source

    A Pointer to the DatasourceV2 Implementation of The Cassandra Source

    CREATE TEMPORARY TABLE tmpTable USING org.apache.spark.sql.cassandra OPTIONS ( table "table", keyspace "keyspace", cluster "test_cluster", pushdown "true", spark.cassandra.input.fetch.sizeInRows "10", spark.cassandra.output.consistency.level "ONE", spark.cassandra.connection.timeoutMS "1000" )

  17. sealed trait DirectJoinSetting extends AnyRef
  18. sealed trait DseSearchOptimizationSetting extends AnyRef
  19. class NullableUnresolvedAttribute extends UnresolvedAttribute
  20. trait PredicateOps[Predicate] extends AnyRef

    A unified API for predicates, used by BasicCassandraPredicatePushDown.

    A unified API for predicates, used by BasicCassandraPredicatePushDown.

    Keeps all the Spark-specific stuff out of BasicCassandraPredicatePushDown It is also easy to plug-in custom predicate implementations for unit-testing.

  21. class SolrPredicateRules extends CassandraPredicateRules with Logging

Value Members

  1. val CassandraFormat: String

    A data frame format used to access Cassandra through Connector

  2. def cassandraOptions(table: String, keyspace: String, cluster: String = ..., pushdownEnable: Boolean = true): Map[String, String]

    Returns a map of options which configure the path to Cassandra table as well as whether pushdown is enabled or not

  3. def ttl(column: String): Column
  4. def ttl(column: Column): Column
  5. def writeTime(column: String): Column
  6. def writeTime(column: Column): Column
  7. object AlwaysOff extends DirectJoinSetting with Product with Serializable
  8. object AlwaysOn extends DirectJoinSetting with Product with Serializable
  9. object Automatic extends DirectJoinSetting with Product with Serializable
  10. object CassandraMetaDataRule extends Rule[LogicalPlan]
  11. object CassandraMetadataFunction
  12. object CassandraSQLContextParams
  13. object CassandraSQLRow extends Serializable
  14. object CassandraSourceRelation extends Logging
  15. object DataTypeConverter extends Logging

    Convert Cassandra data type to Catalyst data type

  16. object DefaultSource
  17. object DsePredicateRules extends CassandraPredicateRules with Logging

    A series of pushdown rules that only apply when connecting to Datastax Enterprise

  18. object InClausePredicateRules extends CassandraPredicateRules with Logging
  19. object Off extends DseSearchOptimizationSetting with Product with Serializable
  20. object On extends DseSearchOptimizationSetting with Product with Serializable
  21. object PredicateOps

    Provides PredicateOps adapters for Expression and Filter classes

  22. object SolrConstants
  23. object TimeUUIDPredicateRules extends CassandraPredicateRules with Logging

    All non-equal predicates on a TimeUUID column are going to fail and fail in silent way.

    All non-equal predicates on a TimeUUID column are going to fail and fail in silent way. The basic issue here is that when you use a comparison on a time UUID column in C* it compares based on the Time portion of the UUID. When Spark executes this filter (unhandled behavior) it will compare lexically, this will lead to results being incorrectly filtered out of the set. As long as the range predicate is handled completely by the connector the correct result will be obtained.

Inherited from AnyRef

Inherited from Any

Ungrouped