org.apache.spark.sql.cassandra

PredicatePushDown

class PredicatePushDown[Predicate] extends AnyRef

Determines which filter predicates can be pushed down to Cassandra.

  1. Only push down no-partition key column predicates with =, >, <, >=, <= predicate 2. Only push down primary key column predicates with = or IN predicate. 3. If there are regular columns in the pushdown predicates, they should have at least one EQ expression on an indexed column and no IN predicates. 4. All partition column predicates must be included in the predicates to be pushed down, only the last part of the partition key can be an IN predicate. For each partition column, only one predicate is allowed. 5. For cluster column predicates, only last predicate can be non-EQ predicate including IN predicate, and preceding column predicates must be EQ predicates. If there is only one cluster column predicate, the predicates could be any non-IN predicate. 6. There is no pushdown predicates if there is any OR condition or NOT IN condition. 7. We're not allowed to push down multiple predicates for the same column if any of them is equality or IN predicate.

The list of predicates to be pushed down is available in predicatesToPushDown property. The list of predicates that cannot be pushed down is available in predicatesToPreserve property.

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. PredicatePushDown
  2. AnyRef
  3. Any
Implicitly
  1. by any2stringadd
  2. by any2stringfmt
  3. by any2ArrowAssoc
  4. by any2Ensuring
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Instance Constructors

  1. new PredicatePushDown(predicates: Set[Predicate], table: TableDef)(implicit arg0: PredicateOps[Predicate])

    predicates

    list of filter predicates available in the user query

    table

    Cassandra table definition

Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any
  3. final def ##(): Int

    Definition Classes
    AnyRef → Any
  4. def +(other: String): String

    Implicit information
    This member is added by an implicit conversion from PredicatePushDown[Predicate] to StringAdd performed by method any2stringadd in scala.Predef.
    Definition Classes
    StringAdd
  5. def ->[B](y: B): (PredicatePushDown[Predicate], B)

    Implicit information
    This member is added by an implicit conversion from PredicatePushDown[Predicate] to ArrowAssoc[PredicatePushDown[Predicate]] performed by method any2ArrowAssoc in scala.Predef.
    Definition Classes
    ArrowAssoc
    Annotations
    @inline()
  6. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  7. final def ==(arg0: Any): Boolean

    Definition Classes
    Any
  8. final def asInstanceOf[T0]: T0

    Definition Classes
    Any
  9. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  10. def ensuring(cond: (PredicatePushDown[Predicate]) ⇒ Boolean, msg: ⇒ Any): PredicatePushDown[Predicate]

    Implicit information
    This member is added by an implicit conversion from PredicatePushDown[Predicate] to Ensuring[PredicatePushDown[Predicate]] performed by method any2Ensuring in scala.Predef.
    Definition Classes
    Ensuring
  11. def ensuring(cond: (PredicatePushDown[Predicate]) ⇒ Boolean): PredicatePushDown[Predicate]

    Implicit information
    This member is added by an implicit conversion from PredicatePushDown[Predicate] to Ensuring[PredicatePushDown[Predicate]] performed by method any2Ensuring in scala.Predef.
    Definition Classes
    Ensuring
  12. def ensuring(cond: Boolean, msg: ⇒ Any): PredicatePushDown[Predicate]

    Implicit information
    This member is added by an implicit conversion from PredicatePushDown[Predicate] to Ensuring[PredicatePushDown[Predicate]] performed by method any2Ensuring in scala.Predef.
    Definition Classes
    Ensuring
  13. def ensuring(cond: Boolean): PredicatePushDown[Predicate]

    Implicit information
    This member is added by an implicit conversion from PredicatePushDown[Predicate] to Ensuring[PredicatePushDown[Predicate]] performed by method any2Ensuring in scala.Predef.
    Definition Classes
    Ensuring
  14. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  15. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any
  16. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  17. def formatted(fmtstr: String): String

    Implicit information
    This member is added by an implicit conversion from PredicatePushDown[Predicate] to StringFormat performed by method any2stringfmt in scala.Predef.
    Definition Classes
    StringFormat
    Annotations
    @inline()
  18. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any
  19. def hashCode(): Int

    Definition Classes
    AnyRef → Any
  20. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any
  21. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  22. final def notify(): Unit

    Definition Classes
    AnyRef
  23. final def notifyAll(): Unit

    Definition Classes
    AnyRef
  24. val predicatesToPreserve: Set[Predicate]

    Returns the set of predicates that cannot be pushed down to Cassandra, so they must be applied by Spark

  25. val predicatesToPushDown: Set[Predicate]

    Returns the set of predicates that can be safely pushed down to Cassandra

  26. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef
  27. val timeUUIDNonEqual: IndexedSeq[Set[Predicate]]

    All non-equal predicates on a TimeUUID column are going to fail and fail in silent way.

    All non-equal predicates on a TimeUUID column are going to fail and fail in silent way. The basic issue here is that when you use a comparison on a time UUID column in C* it compares based on the Time portion of the UUID. When Spark repeats this filter (default behavior) it will compare lexically, this will lead to results being incorrectly filtered out of the set. Because there is no way to correctly do this computation in Spark < 1.6 we will simply throw an error to let the user know they cannot use the DataFrames API for this call.

  28. def toString(): String

    Definition Classes
    AnyRef → Any
  29. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  30. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  31. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  32. def [B](y: B): (PredicatePushDown[Predicate], B)

    Implicit information
    This member is added by an implicit conversion from PredicatePushDown[Predicate] to ArrowAssoc[PredicatePushDown[Predicate]] performed by method any2ArrowAssoc in scala.Predef.
    Definition Classes
    ArrowAssoc

Shadowed Implicit Value Members

  1. val self: Any

    Implicit information
    This member is added by an implicit conversion from PredicatePushDown[Predicate] to StringAdd performed by method any2stringadd in scala.Predef.
    Shadowing
    This implicitly inherited member is ambiguous. One or more implicitly inherited members have similar signatures, so calling this member may produce an ambiguous implicit conversion compiler error.
    To access this member you can use a type ascription:
    (predicatePushDown: StringAdd).self
    Definition Classes
    StringAdd
  2. val self: Any

    Implicit information
    This member is added by an implicit conversion from PredicatePushDown[Predicate] to StringFormat performed by method any2stringfmt in scala.Predef.
    Shadowing
    This implicitly inherited member is ambiguous. One or more implicitly inherited members have similar signatures, so calling this member may produce an ambiguous implicit conversion compiler error.
    To access this member you can use a type ascription:
    (predicatePushDown: StringFormat).self
    Definition Classes
    StringFormat

Deprecated Value Members

  1. def x: PredicatePushDown[Predicate]

    Implicit information
    This member is added by an implicit conversion from PredicatePushDown[Predicate] to ArrowAssoc[PredicatePushDown[Predicate]] performed by method any2ArrowAssoc in scala.Predef.
    Shadowing
    This implicitly inherited member is ambiguous. One or more implicitly inherited members have similar signatures, so calling this member may produce an ambiguous implicit conversion compiler error.
    To access this member you can use a type ascription:
    (predicatePushDown: ArrowAssoc[PredicatePushDown[Predicate]]).x
    Definition Classes
    ArrowAssoc
    Annotations
    @deprecated
    Deprecated

    (Since version 2.10.0) Use leftOfArrow instead

  2. def x: PredicatePushDown[Predicate]

    Implicit information
    This member is added by an implicit conversion from PredicatePushDown[Predicate] to Ensuring[PredicatePushDown[Predicate]] performed by method any2Ensuring in scala.Predef.
    Shadowing
    This implicitly inherited member is ambiguous. One or more implicitly inherited members have similar signatures, so calling this member may produce an ambiguous implicit conversion compiler error.
    To access this member you can use a type ascription:
    (predicatePushDown: Ensuring[PredicatePushDown[Predicate]]).x
    Definition Classes
    Ensuring
    Annotations
    @deprecated
    Deprecated

    (Since version 2.10.0) Use resultOfEnsuring instead

Inherited from AnyRef

Inherited from Any

Inherited by implicit conversion any2stringadd from PredicatePushDown[Predicate] to StringAdd

Inherited by implicit conversion any2stringfmt from PredicatePushDown[Predicate] to StringFormat

Inherited by implicit conversion any2ArrowAssoc from PredicatePushDown[Predicate] to ArrowAssoc[PredicatePushDown[Predicate]]

Inherited by implicit conversion any2Ensuring from PredicatePushDown[Predicate] to Ensuring[PredicatePushDown[Predicate]]

Ungrouped