com.datastax.spark.connector.rdd

CassandraJoinRDD

class CassandraJoinRDD[L, R] extends CassandraRDD[(L, R)] with CassandraTableRowReaderProvider[R]

An RDD that will do a selecting join between left RDD and the specified Cassandra Table This will perform individual selects to retrieve the rows from Cassandra and will take advantage of RDDs that have been partitioned with the com.datastax.spark.connector.rdd.partitioner.ReplicaPartitioner

L

item type on the left side of the join (any RDD)

R

item type on the right side of the join (fetched from Cassandra)

Linear Supertypes
CassandraTableRowReaderProvider[R], CassandraRDD[(L, R)], RDD[(L, R)], Logging, Serializable, Serializable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. CassandraJoinRDD
  2. CassandraTableRowReaderProvider
  3. CassandraRDD
  4. RDD
  5. Logging
  6. Serializable
  7. Serializable
  8. AnyRef
  9. Any
Implicitly
  1. by toPairRDDFunctions
  2. by toRDDFunctions
  3. by any2stringadd
  4. by any2stringfmt
  5. by any2ArrowAssoc
  6. by any2Ensuring
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Type Members

  1. type Self = CassandraJoinRDD[L, R]

    This is slightly different than Scala this.

    This is slightly different than Scala this.type. this.type is the unique singleton type of an object which is not compatible with other instances of the same type, so returning anything other than this is not really possible without lying to the compiler by explicit casts. Here SelfType is used to return a copy of the object - a different instance of the same type

    Definition Classes
    CassandraJoinRDDCassandraRDD

Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any
  3. final def ##(): Int

    Definition Classes
    AnyRef → Any
  4. def +(other: String): String

    Implicit information
    This member is added by an implicit conversion from CassandraJoinRDD[L, R] to StringAdd performed by method any2stringadd in scala.Predef.
    Definition Classes
    StringAdd
  5. def ++(other: RDD[(L, R)]): RDD[(L, R)]

    Definition Classes
    RDD
  6. def ->[B](y: B): (CassandraJoinRDD[L, R], B)

    Implicit information
    This member is added by an implicit conversion from CassandraJoinRDD[L, R] to ArrowAssoc[CassandraJoinRDD[L, R]] performed by method any2ArrowAssoc in scala.Predef.
    Definition Classes
    ArrowAssoc
    Annotations
    @inline()
  7. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  8. final def ==(arg0: Any): Boolean

    Definition Classes
    Any
  9. def aggregate[U](zeroValue: U)(seqOp: (U, (L, R)) ⇒ U, combOp: (U, U) ⇒ U)(implicit arg0: ClassTag[U]): U

    Definition Classes
    RDD
  10. def as[B, A0, A1, A2, A3, A4, A5, A6, A7, A8, A9, A10, A11](f: (A0, A1, A2, A3, A4, A5, A6, A7, A8, A9, A10, A11) ⇒ B)(implicit arg0: ClassTag[B], arg1: TypeConverter[A0], arg2: TypeConverter[A1], arg3: TypeConverter[A2], arg4: TypeConverter[A3], arg5: TypeConverter[A4], arg6: TypeConverter[A5], arg7: TypeConverter[A6], arg8: TypeConverter[A7], arg9: TypeConverter[A8], arg10: TypeConverter[A9], arg11: TypeConverter[A10], arg12: TypeConverter[A11]): CassandraRDD[B]

    Definition Classes
    CassandraRDD
  11. def as[B, A0, A1, A2, A3, A4, A5, A6, A7, A8, A9, A10](f: (A0, A1, A2, A3, A4, A5, A6, A7, A8, A9, A10) ⇒ B)(implicit arg0: ClassTag[B], arg1: TypeConverter[A0], arg2: TypeConverter[A1], arg3: TypeConverter[A2], arg4: TypeConverter[A3], arg5: TypeConverter[A4], arg6: TypeConverter[A5], arg7: TypeConverter[A6], arg8: TypeConverter[A7], arg9: TypeConverter[A8], arg10: TypeConverter[A9], arg11: TypeConverter[A10]): CassandraRDD[B]

    Definition Classes
    CassandraRDD
  12. def as[B, A0, A1, A2, A3, A4, A5, A6, A7, A8, A9](f: (A0, A1, A2, A3, A4, A5, A6, A7, A8, A9) ⇒ B)(implicit arg0: ClassTag[B], arg1: TypeConverter[A0], arg2: TypeConverter[A1], arg3: TypeConverter[A2], arg4: TypeConverter[A3], arg5: TypeConverter[A4], arg6: TypeConverter[A5], arg7: TypeConverter[A6], arg8: TypeConverter[A7], arg9: TypeConverter[A8], arg10: TypeConverter[A9]): CassandraRDD[B]

    Definition Classes
    CassandraRDD
  13. def as[B, A0, A1, A2, A3, A4, A5, A6, A7, A8](f: (A0, A1, A2, A3, A4, A5, A6, A7, A8) ⇒ B)(implicit arg0: ClassTag[B], arg1: TypeConverter[A0], arg2: TypeConverter[A1], arg3: TypeConverter[A2], arg4: TypeConverter[A3], arg5: TypeConverter[A4], arg6: TypeConverter[A5], arg7: TypeConverter[A6], arg8: TypeConverter[A7], arg9: TypeConverter[A8]): CassandraRDD[B]

    Definition Classes
    CassandraRDD
  14. def as[B, A0, A1, A2, A3, A4, A5, A6, A7](f: (A0, A1, A2, A3, A4, A5, A6, A7) ⇒ B)(implicit arg0: ClassTag[B], arg1: TypeConverter[A0], arg2: TypeConverter[A1], arg3: TypeConverter[A2], arg4: TypeConverter[A3], arg5: TypeConverter[A4], arg6: TypeConverter[A5], arg7: TypeConverter[A6], arg8: TypeConverter[A7]): CassandraRDD[B]

    Definition Classes
    CassandraRDD
  15. def as[B, A0, A1, A2, A3, A4, A5, A6](f: (A0, A1, A2, A3, A4, A5, A6) ⇒ B)(implicit arg0: ClassTag[B], arg1: TypeConverter[A0], arg2: TypeConverter[A1], arg3: TypeConverter[A2], arg4: TypeConverter[A3], arg5: TypeConverter[A4], arg6: TypeConverter[A5], arg7: TypeConverter[A6]): CassandraRDD[B]

    Definition Classes
    CassandraRDD
  16. def as[B, A0, A1, A2, A3, A4, A5](f: (A0, A1, A2, A3, A4, A5) ⇒ B)(implicit arg0: ClassTag[B], arg1: TypeConverter[A0], arg2: TypeConverter[A1], arg3: TypeConverter[A2], arg4: TypeConverter[A3], arg5: TypeConverter[A4], arg6: TypeConverter[A5]): CassandraRDD[B]

    Definition Classes
    CassandraRDD
  17. def as[B, A0, A1, A2, A3, A4](f: (A0, A1, A2, A3, A4) ⇒ B)(implicit arg0: ClassTag[B], arg1: TypeConverter[A0], arg2: TypeConverter[A1], arg3: TypeConverter[A2], arg4: TypeConverter[A3], arg5: TypeConverter[A4]): CassandraRDD[B]

    Definition Classes
    CassandraRDD
  18. def as[B, A0, A1, A2, A3](f: (A0, A1, A2, A3) ⇒ B)(implicit arg0: ClassTag[B], arg1: TypeConverter[A0], arg2: TypeConverter[A1], arg3: TypeConverter[A2], arg4: TypeConverter[A3]): CassandraRDD[B]

    Definition Classes
    CassandraRDD
  19. def as[B, A0, A1, A2](f: (A0, A1, A2) ⇒ B)(implicit arg0: ClassTag[B], arg1: TypeConverter[A0], arg2: TypeConverter[A1], arg3: TypeConverter[A2]): CassandraRDD[B]

    Definition Classes
    CassandraRDD
  20. def as[B, A0, A1](f: (A0, A1) ⇒ B)(implicit arg0: ClassTag[B], arg1: TypeConverter[A0], arg2: TypeConverter[A1]): CassandraRDD[B]

    Definition Classes
    CassandraRDD
  21. def as[B, A0](f: (A0) ⇒ B)(implicit arg0: ClassTag[B], arg1: TypeConverter[A0]): CassandraRDD[B]

    Maps each row into object of a different type using provided function taking column value(s) as argument(s).

    Maps each row into object of a different type using provided function taking column value(s) as argument(s). Can be used to convert each row to a tuple or a case class object:

    sc.cassandraTable("ks", "table")
      .select("column1")
      .as((s: String) => s)                 // yields CassandraRDD[String]
    
    sc.cassandraTable("ks", "table")
      .select("column1", "column2")
      .as((_: String, _: Long))             // yields CassandraRDD[(String, Long)]
    
    case class MyRow(key: String, value: Long)
    sc.cassandraTable("ks", "table")
      .select("column1", "column2")
      .as(MyRow)                            // yields CassandraRDD[MyRow]
    Definition Classes
    CassandraRDD
  22. final def asInstanceOf[T0]: T0

    Definition Classes
    Any
  23. def cache(): CassandraJoinRDD.this.type

    Definition Classes
    RDD
  24. def cartesian[U](other: RDD[U])(implicit arg0: ClassTag[U]): RDD[((L, R), U)]

    Definition Classes
    RDD
  25. lazy val cassandraPartitionerClassName: String

    Attributes
    protected
    Definition Classes
    CassandraTableRowReaderProvider
  26. def checkColumnsExistence(columns: Seq[SelectableColumnRef]): Seq[SelectableColumnRef]

    Attributes
    protected
    Definition Classes
    CassandraTableRowReaderProvider
  27. def checkValidJoin(): Seq[NamedColumnRef]

    This method will create the RowWriter required before the RDD is serialized.

    This method will create the RowWriter required before the RDD is serialized. This is called during getPartitions

    Attributes
    protected
  28. def checkpoint(): Unit

    Definition Classes
    RDD
  29. val classTag: ClassTag[R]

    Attributes
    protected
    Definition Classes
    CassandraJoinRDDCassandraTableRowReaderProvider
  30. def clearDependencies(): Unit

    Attributes
    protected
    Definition Classes
    RDD
  31. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  32. def clusteringOrder(order: ClusteringOrder): Self

    Adds a CQL ORDER BY clause to the query.

    Adds a CQL ORDER BY clause to the query. It can be applied only in case there are clustering columns and primary key predicate is pushed down in where. It is useful when the default direction of ordering rows within a single Cassandra partition needs to be changed.

    Definition Classes
    CassandraRDD
  33. val clusteringOrder: Option[ClusteringOrder]

    Definition Classes
    CassandraJoinRDDCassandraRDD
  34. def coalesce(numPartitions: Int, shuffle: Boolean)(implicit ord: Ordering[(L, R)]): RDD[(L, R)]

    Definition Classes
    RDD
  35. def collect[U](f: PartialFunction[(L, R), U])(implicit arg0: ClassTag[U]): RDD[U]

    Definition Classes
    RDD
  36. def collect(): Array[(L, R)]

    Definition Classes
    RDD
  37. val columnNames: ColumnSelector

  38. def compute(split: Partition, context: TaskContext): Iterator[(L, R)]

    When computing a CassandraPartitionKeyRDD the data is selected via single CQL statements from the specified C* Keyspace and Table.

    When computing a CassandraPartitionKeyRDD the data is selected via single CQL statements from the specified C* Keyspace and Table. This will be preformed on whatever data is available in the previous RDD in the chain.

    Definition Classes
    CassandraJoinRDD → RDD
  39. val connector: CassandraConnector

  40. def consistencyLevel: ConsistencyLevel

    Attributes
    protected
    Definition Classes
    CassandraTableRowReaderProvider
  41. def context: SparkContext

    Definition Classes
    RDD
  42. def convertTo[B](implicit arg0: ClassTag[B], arg1: RowReaderFactory[B]): CassandraRDD[B]

    Attributes
    protected
    Definition Classes
    CassandraRDD
  43. def copy(columnNames: ColumnSelector = columnNames, where: CqlWhereClause = where, limit: Option[Long] = limit, clusteringOrder: Option[ClusteringOrder] = None, readConf: ReadConf = readConf, connector: CassandraConnector = connector): Self

    Allows to copy this RDD with changing some of the properties

    Allows to copy this RDD with changing some of the properties

    Attributes
    protected
    Definition Classes
    CassandraJoinRDDCassandraRDD
  44. def count(): Long

    Definition Classes
    CassandraJoinRDD → RDD
  45. def countApprox(timeout: Long, confidence: Double): PartialResult[BoundedDouble]

    Definition Classes
    RDD
    Annotations
    @Experimental()
  46. def countApproxDistinct(relativeSD: Double): Long

    Definition Classes
    RDD
  47. def countApproxDistinct(p: Int, sp: Int): Long

    Definition Classes
    RDD
    Annotations
    @Experimental()
  48. def countByValue()(implicit ord: Ordering[(L, R)]): Map[(L, R), Long]

    Definition Classes
    RDD
  49. def countByValueApprox(timeout: Long, confidence: Double)(implicit ord: Ordering[(L, R)]): PartialResult[Map[(L, R), BoundedDouble]]

    Definition Classes
    RDD
    Annotations
    @Experimental()
  50. final def dependencies: Seq[Dependency[_]]

    Definition Classes
    RDD
  51. def distinct(): RDD[(L, R)]

    Definition Classes
    RDD
  52. def distinct(numPartitions: Int)(implicit ord: Ordering[(L, R)]): RDD[(L, R)]

    Definition Classes
    RDD
  53. def ensuring(cond: (CassandraJoinRDD[L, R]) ⇒ Boolean, msg: ⇒ Any): CassandraJoinRDD[L, R]

    Implicit information
    This member is added by an implicit conversion from CassandraJoinRDD[L, R] to Ensuring[CassandraJoinRDD[L, R]] performed by method any2Ensuring in scala.Predef.
    Definition Classes
    Ensuring
  54. def ensuring(cond: (CassandraJoinRDD[L, R]) ⇒ Boolean): CassandraJoinRDD[L, R]

    Implicit information
    This member is added by an implicit conversion from CassandraJoinRDD[L, R] to Ensuring[CassandraJoinRDD[L, R]] performed by method any2Ensuring in scala.Predef.
    Definition Classes
    Ensuring
  55. def ensuring(cond: Boolean, msg: ⇒ Any): CassandraJoinRDD[L, R]

    Implicit information
    This member is added by an implicit conversion from CassandraJoinRDD[L, R] to Ensuring[CassandraJoinRDD[L, R]] performed by method any2Ensuring in scala.Predef.
    Definition Classes
    Ensuring
  56. def ensuring(cond: Boolean): CassandraJoinRDD[L, R]

    Implicit information
    This member is added by an implicit conversion from CassandraJoinRDD[L, R] to Ensuring[CassandraJoinRDD[L, R]] performed by method any2Ensuring in scala.Predef.
    Definition Classes
    Ensuring
  57. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  58. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any
  59. def fetchSize: Int

    Attributes
    protected
    Definition Classes
    CassandraTableRowReaderProvider
  60. def filter(f: ((L, R)) ⇒ Boolean): RDD[(L, R)]

    Definition Classes
    RDD
  61. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  62. def first(): (L, R)

    Definition Classes
    RDD
  63. def firstParent[U](implicit arg0: ClassTag[U]): RDD[U]

    Attributes
    protected[org.apache.spark]
    Definition Classes
    RDD
  64. def flatMap[U](f: ((L, R)) ⇒ TraversableOnce[U])(implicit arg0: ClassTag[U]): RDD[U]

    Definition Classes
    RDD
  65. def fold(zeroValue: (L, R))(op: ((L, R), (L, R)) ⇒ (L, R)): (L, R)

    Definition Classes
    RDD
  66. def foreach(f: ((L, R)) ⇒ Unit): Unit

    Definition Classes
    RDD
  67. def foreachPartition(f: (Iterator[(L, R)]) ⇒ Unit): Unit

    Definition Classes
    RDD
  68. def formatted(fmtstr: String): String

    Implicit information
    This member is added by an implicit conversion from CassandraJoinRDD[L, R] to StringFormat performed by method any2stringfmt in scala.Predef.
    Definition Classes
    StringFormat
    Annotations
    @inline()
  69. def getCheckpointFile: Option[String]

    Definition Classes
    RDD
  70. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any
  71. def getDependencies: Seq[Dependency[_]]

    Attributes
    protected
    Definition Classes
    RDD
  72. def getPartitions: Array[Partition]

    Attributes
    protected
    Definition Classes
    CassandraJoinRDD → RDD
  73. def getPreferredLocations(split: Partition): Seq[String]

    Definition Classes
    CassandraJoinRDD → RDD
  74. def getStorageLevel: StorageLevel

    Definition Classes
    RDD
  75. def glom(): RDD[Array[(L, R)]]

    Definition Classes
    RDD
  76. def groupBy[K](f: ((L, R)) ⇒ K, p: Partitioner)(implicit kt: ClassTag[K], ord: Ordering[K]): RDD[(K, Iterable[(L, R)])]

    Definition Classes
    RDD
  77. def groupBy[K](f: ((L, R)) ⇒ K, numPartitions: Int)(implicit kt: ClassTag[K]): RDD[(K, Iterable[(L, R)])]

    Definition Classes
    RDD
  78. def groupBy[K](f: ((L, R)) ⇒ K)(implicit kt: ClassTag[K]): RDD[(K, Iterable[(L, R)])]

    Definition Classes
    RDD
  79. def hashCode(): Int

    Definition Classes
    AnyRef → Any
  80. val id: Int

    Definition Classes
    RDD
  81. def intersection(other: RDD[(L, R)], numPartitions: Int): RDD[(L, R)]

    Definition Classes
    RDD
  82. def intersection(other: RDD[(L, R)], partitioner: Partitioner)(implicit ord: Ordering[(L, R)]): RDD[(L, R)]

    Definition Classes
    RDD
  83. def intersection(other: RDD[(L, R)]): RDD[(L, R)]

    Definition Classes
    RDD
  84. def isCheckpointed: Boolean

    Definition Classes
    RDD
  85. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any
  86. def isTraceEnabled(): Boolean

    Attributes
    protected
    Definition Classes
    Logging
  87. final def iterator(split: Partition, context: TaskContext): Iterator[(L, R)]

    Definition Classes
    RDD
  88. lazy val joinColumnNames: Seq[NamedColumnRef]

  89. val joinColumns: ColumnSelector

  90. def joinWithCassandraTable[R](keyspaceName: String, tableName: String, selectedColumns: ColumnSelector = AllColumns, joinColumns: ColumnSelector = PartitionKeyColumns)(implicit connector: CassandraConnector = ..., newType: ClassTag[R], rrf: RowReaderFactory[R], ev: ValidRDDType[R], currentType: ClassTag[(L, R)], rwf: RowWriterFactory[(L, R)]): CassandraJoinRDD[(L, R), R]

    Uses the data from RDD to join with a Cassandra table without retrieving the entire table.

    Uses the data from RDD to join with a Cassandra table without retrieving the entire table. Any RDD which can be used to saveToCassandra can be used to joinWithCassandra as well as any RDD which only specifies the partition Key of a Cassandra Table. This method executes single partition requests against the Cassandra Table and accepts the functional modifiers that a normal com.datastax.spark.connector.rdd.CassandraTableScanRDD takes.

    By default this method only uses the Partition Key for joining but any combination of columns which are acceptable to C* can be used in the join. Specify columns using joinColumns as a parameter or the on() method.

    Example With Prior Repartitioning:

    val source = sc.parallelize(keys).map(x => new KVRow(x))
    val repart = source.repartitionByCassandraReplica(keyspace, tableName, 10)
    val someCass = repart.joinWithCassandraTable(keyspace, tableName)

    Example Joining on Clustering Columns:

    val source = sc.parallelize(keys).map(x => (x, x * 100))
    val someCass = source.joinWithCassandraTable(keyspace, wideTable).on(SomeColumns("key", "group"))
    Implicit information
    This member is added by an implicit conversion from CassandraJoinRDD[L, R] to RDDFunctions[(L, R)] performed by method toRDDFunctions in com.datastax.spark.connector.
    Definition Classes
    RDDFunctions
  91. def keyBy[K](f: ((L, R)) ⇒ K): RDD[(K, (L, R))]

    Definition Classes
    RDD
  92. def keyByCassandraReplica(keyspaceName: String, tableName: String)(implicit connector: CassandraConnector = ..., currentType: ClassTag[(L, R)], rwf: RowWriterFactory[(L, R)]): RDD[(Set[InetAddress], (L, R))]

    Key every row in the RDD by with the IP Adresses of all of the Cassandra nodes which a contain a replica of the data specified by that row.

    Key every row in the RDD by with the IP Adresses of all of the Cassandra nodes which a contain a replica of the data specified by that row. The calling RDD must have rows that can be converted into the partition key of the given Cassandra Table.

    Implicit information
    This member is added by an implicit conversion from CassandraJoinRDD[L, R] to RDDFunctions[(L, R)] performed by method toRDDFunctions in com.datastax.spark.connector.
    Definition Classes
    RDDFunctions
  93. val keyspaceName: String

  94. implicit val leftClassTag: ClassTag[L]

  95. def limit(rowLimit: Long): Self

    Adds the limit clause to CQL select statement.

    Adds the limit clause to CQL select statement. The limit will be applied for each created Spark partition. In other words, unless the data are fetched from a single Cassandra partition the number of results is unpredictable.

    The main purpose of passing limit clause is to fetch top n rows from a single Cassandra partition when the table is designed so that it uses clustering keys and a partition key predicate is passed to the where clause.

    Definition Classes
    CassandraRDD
  96. val limit: Option[Long]

    Definition Classes
    CassandraJoinRDDCassandraRDD
  97. def log: Logger

    Attributes
    protected
    Definition Classes
    Logging
  98. def logDebug(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging
  99. def logDebug(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging
  100. def logError(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging
  101. def logError(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging
  102. def logInfo(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging
  103. def logInfo(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging
  104. def logName: String

    Attributes
    protected
    Definition Classes
    Logging
  105. def logTrace(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging
  106. def logTrace(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging
  107. def logWarning(msg: ⇒ String, throwable: Throwable): Unit

    Attributes
    protected
    Definition Classes
    Logging
  108. def logWarning(msg: ⇒ String): Unit

    Attributes
    protected
    Definition Classes
    Logging
  109. def map[U](f: ((L, R)) ⇒ U)(implicit arg0: ClassTag[U]): RDD[U]

    Definition Classes
    RDD
  110. def mapPartitions[U](f: (Iterator[(L, R)]) ⇒ Iterator[U], preservesPartitioning: Boolean)(implicit arg0: ClassTag[U]): RDD[U]

    Definition Classes
    RDD
  111. def mapPartitionsWithIndex[U](f: (Int, Iterator[(L, R)]) ⇒ Iterator[U], preservesPartitioning: Boolean)(implicit arg0: ClassTag[U]): RDD[U]

    Definition Classes
    RDD
  112. def max()(implicit ord: Ordering[(L, R)]): (L, R)

    Definition Classes
    RDD
  113. def min()(implicit ord: Ordering[(L, R)]): (L, R)

    Definition Classes
    RDD
  114. var name: String

    Definition Classes
    RDD
  115. def narrowColumnSelection(columns: Seq[SelectableColumnRef]): Seq[SelectableColumnRef]

    Filters currently selected set of columns with a new set of columns

    Filters currently selected set of columns with a new set of columns

    Definition Classes
    CassandraTableRowReaderProvider
  116. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef
  117. final def notify(): Unit

    Definition Classes
    AnyRef
  118. final def notifyAll(): Unit

    Definition Classes
    AnyRef
  119. def on(joinColumns: ColumnSelector): CassandraJoinRDD[L, R]

  120. def parent[U](j: Int)(implicit arg0: ClassTag[U]): RDD[U]

    Attributes
    protected[org.apache.spark]
    Definition Classes
    RDD
  121. val partitioner: Option[Partitioner]

    Definition Classes
    RDD
  122. final def partitions: Array[Partition]

    Definition Classes
    RDD
  123. def persist(): CassandraJoinRDD.this.type

    Definition Classes
    RDD
  124. def persist(newLevel: StorageLevel): CassandraJoinRDD.this.type

    Definition Classes
    RDD
  125. def pipe(command: Seq[String], env: Map[String, String], printPipeContext: ((String) ⇒ Unit) ⇒ Unit, printRDDElement: ((L, R), (String) ⇒ Unit) ⇒ Unit, separateWorkingDir: Boolean): RDD[String]

    Definition Classes
    RDD
  126. def pipe(command: String, env: Map[String, String]): RDD[String]

    Definition Classes
    RDD
  127. def pipe(command: String): RDD[String]

    Definition Classes
    RDD
  128. final def preferredLocations(split: Partition): Seq[String]

    Definition Classes
    RDD
  129. def protocolVersion(session: Session): ProtocolVersion

  130. def quote(name: String): String

    Attributes
    protected
    Definition Classes
    CassandraTableRowReaderProvider
  131. def randomSplit(weights: Array[Double], seed: Long): Array[RDD[(L, R)]]

    Definition Classes
    RDD
  132. val readConf: ReadConf

  133. def reduce(f: ((L, R), (L, R)) ⇒ (L, R)): (L, R)

    Definition Classes
    RDD
  134. def repartition(numPartitions: Int)(implicit ord: Ordering[(L, R)]): RDD[(L, R)]

    Definition Classes
    RDD
  135. def repartitionByCassandraReplica(keyspaceName: String, tableName: String, partitionsPerHost: Int = 10)(implicit connector: CassandraConnector = ..., currentType: ClassTag[(L, R)], rwf: RowWriterFactory[(L, R)]): CassandraPartitionedRDD[(L, R)]

    Repartitions the data (via a shuffle) based upon the replication of the given keyspaceName and tableName.

    Repartitions the data (via a shuffle) based upon the replication of the given keyspaceName and tableName. Calling this method before using joinWithCassandraTable will ensure that requests will be coordinator local. partitionsPerHost Controls the number of Spark Partitions that will be created in this repartitioning event. The calling RDD must have rows that can be converted into the partition key of the given Cassandra Table.

    Implicit information
    This member is added by an implicit conversion from CassandraJoinRDD[L, R] to RDDFunctions[(L, R)] performed by method toRDDFunctions in com.datastax.spark.connector.
    Definition Classes
    RDDFunctions
  136. implicit val rightClassTag: ClassTag[R]

  137. lazy val rowReader: RowReader[R]

    Attributes
    protected
    Definition Classes
    CassandraTableRowReaderProvider
  138. implicit val rowReaderFactory: RowReaderFactory[R]

    RowReaderFactory and ClassTag should be provided from implicit parameters in the constructor of the class implementing this trait

    RowReaderFactory and ClassTag should be provided from implicit parameters in the constructor of the class implementing this trait

    Definition Classes
    CassandraJoinRDDCassandraTableRowReaderProvider
    See also

    CassandraTableScanRDD

  139. lazy val rowWriter: RowWriter[L]

  140. implicit val rowWriterFactory: RowWriterFactory[L]

  141. def sample(withReplacement: Boolean, fraction: Double, seed: Long): RDD[(L, R)]

    Definition Classes
    RDD
  142. def saveAsCassandraTable(keyspaceName: String, tableName: String, columns: ColumnSelector = AllColumns, writeConf: WriteConf = ...)(implicit connector: CassandraConnector = ..., rwf: RowWriterFactory[(L, R)], columnMapper: ColumnMapper[(L, R)]): Unit

    Saves the data from RDD to a new table with definition taken from the ColumnMapper for this class.

    Saves the data from RDD to a new table with definition taken from the ColumnMapper for this class.

    keyspaceName

    keyspace where to create a new table

    tableName

    name of the table to create; the table must not exist

    columns

    Selects the columns to save data to. Uses only the unique column names, and you must select at least all primary key columns. All other fields are discarded. Non-selected property/column names are left unchanged. This parameter does not affect table creation.

    writeConf

    additional configuration object allowing to set consistency level, batch size, etc.

    connector

    optional, implicit connector to Cassandra

    rwf

    factory for obtaining the row writer to be used to extract column values from items of the RDD

    columnMapper

    a column mapper determining the definition of the table

    Implicit information
    This member is added by an implicit conversion from CassandraJoinRDD[L, R] to RDDFunctions[(L, R)] performed by method toRDDFunctions in com.datastax.spark.connector.
    Definition Classes
    RDDFunctions
  143. def saveAsCassandraTableEx(table: TableDef, columns: ColumnSelector = AllColumns, writeConf: WriteConf = ...)(implicit connector: CassandraConnector = ..., rwf: RowWriterFactory[(L, R)]): Unit

    Saves the data from RDD to a new table defined by the given TableDef.

    Saves the data from RDD to a new table defined by the given TableDef.

    First it creates a new table with all columns from the TableDef and then it saves RDD content in the same way as saveToCassandra. The table must not exist prior to this call.

    table

    table definition used to create a new table

    columns

    Selects the columns to save data to. Uses only the unique column names, and you must select at least all primary key columns. All other fields are discarded. Non-selected property/column names are left unchanged. This parameter does not affect table creation.

    writeConf

    additional configuration object allowing to set consistency level, batch size, etc.

    connector

    optional, implicit connector to Cassandra

    rwf

    factory for obtaining the row writer to be used to extract column values from items of the RDD

    Implicit information
    This member is added by an implicit conversion from CassandraJoinRDD[L, R] to RDDFunctions[(L, R)] performed by method toRDDFunctions in com.datastax.spark.connector.
    Definition Classes
    RDDFunctions
  144. def saveAsObjectFile(path: String): Unit

    Definition Classes
    RDD
  145. def saveAsTextFile(path: String, codec: Class[_ <: CompressionCodec]): Unit

    Definition Classes
    RDD
  146. def saveAsTextFile(path: String): Unit

    Definition Classes
    RDD
  147. def saveToCassandra(keyspaceName: String, tableName: String, columns: ColumnSelector = AllColumns, writeConf: WriteConf = ...)(implicit connector: CassandraConnector = ..., rwf: RowWriterFactory[(L, R)]): Unit

    Saves the data from RDD to a Cassandra table.

    Saves the data from RDD to a Cassandra table. Uses the specified column names.

    keyspaceName

    the name of the Keyspace to use

    tableName

    the name of the Table to use

    writeConf

    additional configuration object allowing to set consistency level, batch size, etc.

    Implicit information
    This member is added by an implicit conversion from CassandraJoinRDD[L, R] to RDDFunctions[(L, R)] performed by method toRDDFunctions in com.datastax.spark.connector.
    Definition Classes
    RDDFunctionsWritableToCassandra
    See also

    com.datastax.spark.connector.writer.WritableToCassandra

  148. def select(columns: SelectableColumnRef*): Self

    Narrows down the selected set of columns.

    Narrows down the selected set of columns. Use this for better performance, when you don't need all the columns in the result RDD. When called multiple times, it selects the subset of the already selected columns, so after a column was removed by the previous select call, it is not possible to add it back.

    The selected columns are NamedColumnRef instances. This type allows to specify columns for straightforward retrieval and to read TTL or write time of regular columns as well. Implicit conversions included in com.datastax.spark.connector package make it possible to provide just column names (which is also backward compatible) and optional add .ttl or .writeTime suffix in order to create an appropriate NamedColumnRef instance.

    Definition Classes
    CassandraRDD
  149. def selectedColumnNames: Seq[String]

    Definition Classes
    CassandraRDD
  150. lazy val selectedColumnRefs: Seq[SelectableColumnRef]

    Returns the names of columns to be selected from the table.

    Returns the names of columns to be selected from the table.

    Definition Classes
    CassandraTableRowReaderProvider
  151. def setName(_name: String): CassandraJoinRDD.this.type

    Definition Classes
    RDD
  152. lazy val singleKeyCqlQuery: String

  153. def sortBy[K](f: ((L, R)) ⇒ K, ascending: Boolean, numPartitions: Int)(implicit ord: Ordering[K], ctag: ClassTag[K]): RDD[(L, R)]

    Definition Classes
    RDD
  154. def spanBy[U](f: ((L, R)) ⇒ U): RDD[(U, Iterable[(L, R)])]

    Applies a function to each item, and groups consecutive items having the same value together.

    Applies a function to each item, and groups consecutive items having the same value together. Contrary to groupBy, items from the same group must be already next to each other in the original collection. Works locally on each partition, so items from different partitions will never be placed in the same group.

    Implicit information
    This member is added by an implicit conversion from CassandraJoinRDD[L, R] to RDDFunctions[(L, R)] performed by method toRDDFunctions in com.datastax.spark.connector.
    Definition Classes
    RDDFunctions
  155. def spanByKey: RDD[(L, Seq[R])]

    Groups items with the same key, assuming the items with the same key are next to each other in the collection.

    Groups items with the same key, assuming the items with the same key are next to each other in the collection. It does not perform shuffle, therefore it is much faster than using much more universal Spark RDD groupByKey. For this method to be useful with Cassandra tables, the key must represent a prefix of the primary key, containing at least the partition key of the Cassandra table.

    Implicit information
    This member is added by an implicit conversion from CassandraJoinRDD[L, R] to PairRDDFunctions[L, R] performed by method toPairRDDFunctions in com.datastax.spark.connector.
    Definition Classes
    PairRDDFunctions
  156. def sparkContext: SparkContext

    Definition Classes
    RDD
  157. def splitSize: Int

    Attributes
    protected
    Definition Classes
    CassandraTableRowReaderProvider
  158. def subtract(other: RDD[(L, R)], p: Partitioner)(implicit ord: Ordering[(L, R)]): RDD[(L, R)]

    Definition Classes
    RDD
  159. def subtract(other: RDD[(L, R)], numPartitions: Int): RDD[(L, R)]

    Definition Classes
    RDD
  160. def subtract(other: RDD[(L, R)]): RDD[(L, R)]

    Definition Classes
    RDD
  161. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef
  162. lazy val tableDef: TableDef

  163. val tableName: String

  164. def take(num: Int): Array[(L, R)]

    Definition Classes
    CassandraRDD → RDD
  165. def takeOrdered(num: Int)(implicit ord: Ordering[(L, R)]): Array[(L, R)]

    Definition Classes
    RDD
  166. def takeSample(withReplacement: Boolean, num: Int, seed: Long): Array[(L, R)]

    Definition Classes
    RDD
  167. def toDebugString: String

    Definition Classes
    RDD
  168. def toEmptyCassandraRDD: EmptyCassandraRDD[(L, R)]

    Definition Classes
    CassandraJoinRDDCassandraRDD
  169. def toJavaRDD(): JavaRDD[(L, R)]

    Definition Classes
    RDD
  170. def toLocalIterator: Iterator[(L, R)]

    Definition Classes
    RDD
  171. def toString(): String

    Definition Classes
    RDD → AnyRef → Any
  172. def top(num: Int)(implicit ord: Ordering[(L, R)]): Array[(L, R)]

    Definition Classes
    RDD
  173. def union(other: RDD[(L, R)]): RDD[(L, R)]

    Definition Classes
    RDD
  174. def unpersist(blocking: Boolean): CassandraJoinRDD.this.type

    Definition Classes
    RDD
  175. def verify(): Unit

    Checks for existence of keyspace, table, columns and whether the number of selected columns corresponds to the number of the columns expected by the target type constructor.

    Checks for existence of keyspace, table, columns and whether the number of selected columns corresponds to the number of the columns expected by the target type constructor. If successful, does nothing, otherwise throws appropriate IOException or AssertionError.

    Definition Classes
    CassandraTableRowReaderProvider
  176. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  177. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  178. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  179. def where(cql: String, values: Any*): Self

    Adds a CQL WHERE predicate(s) to the query.

    Adds a CQL WHERE predicate(s) to the query. Useful for leveraging secondary indexes in Cassandra. Implicitly adds an ALLOW FILTERING clause to the WHERE clause, however beware that some predicates might be rejected by Cassandra, particularly in cases when they filter on an unindexed, non-clustering column.

    Definition Classes
    CassandraRDD
  180. val where: CqlWhereClause

    Definition Classes
    CassandraJoinRDDCassandraRDD
  181. def withAscOrder: Self

    Definition Classes
    CassandraRDD
  182. def withConnector(connector: CassandraConnector): Self

    Returns a copy of this Cassandra RDD with specified connector

    Returns a copy of this Cassandra RDD with specified connector

    Definition Classes
    CassandraRDD
  183. def withDescOrder: Self

    Definition Classes
    CassandraRDD
  184. def withReadConf(readConf: ReadConf): Self

    Allows to set custom read configuration, e.

    Allows to set custom read configuration, e.g. consistency level or fetch size.

    Definition Classes
    CassandraRDD
  185. def zip[U](other: RDD[U])(implicit arg0: ClassTag[U]): RDD[((L, R), U)]

    Definition Classes
    RDD
  186. def zipPartitions[B, C, D, V](rdd2: RDD[B], rdd3: RDD[C], rdd4: RDD[D])(f: (Iterator[(L, R)], Iterator[B], Iterator[C], Iterator[D]) ⇒ Iterator[V])(implicit arg0: ClassTag[B], arg1: ClassTag[C], arg2: ClassTag[D], arg3: ClassTag[V]): RDD[V]

    Definition Classes
    RDD
  187. def zipPartitions[B, C, D, V](rdd2: RDD[B], rdd3: RDD[C], rdd4: RDD[D], preservesPartitioning: Boolean)(f: (Iterator[(L, R)], Iterator[B], Iterator[C], Iterator[D]) ⇒ Iterator[V])(implicit arg0: ClassTag[B], arg1: ClassTag[C], arg2: ClassTag[D], arg3: ClassTag[V]): RDD[V]

    Definition Classes
    RDD
  188. def zipPartitions[B, C, V](rdd2: RDD[B], rdd3: RDD[C])(f: (Iterator[(L, R)], Iterator[B], Iterator[C]) ⇒ Iterator[V])(implicit arg0: ClassTag[B], arg1: ClassTag[C], arg2: ClassTag[V]): RDD[V]

    Definition Classes
    RDD
  189. def zipPartitions[B, C, V](rdd2: RDD[B], rdd3: RDD[C], preservesPartitioning: Boolean)(f: (Iterator[(L, R)], Iterator[B], Iterator[C]) ⇒ Iterator[V])(implicit arg0: ClassTag[B], arg1: ClassTag[C], arg2: ClassTag[V]): RDD[V]

    Definition Classes
    RDD
  190. def zipPartitions[B, V](rdd2: RDD[B])(f: (Iterator[(L, R)], Iterator[B]) ⇒ Iterator[V])(implicit arg0: ClassTag[B], arg1: ClassTag[V]): RDD[V]

    Definition Classes
    RDD
  191. def zipPartitions[B, V](rdd2: RDD[B], preservesPartitioning: Boolean)(f: (Iterator[(L, R)], Iterator[B]) ⇒ Iterator[V])(implicit arg0: ClassTag[B], arg1: ClassTag[V]): RDD[V]

    Definition Classes
    RDD
  192. def zipWithIndex(): RDD[((L, R), Long)]

    Definition Classes
    RDD
  193. def zipWithUniqueId(): RDD[((L, R), Long)]

    Definition Classes
    RDD
  194. def [B](y: B): (CassandraJoinRDD[L, R], B)

    Implicit information
    This member is added by an implicit conversion from CassandraJoinRDD[L, R] to ArrowAssoc[CassandraJoinRDD[L, R]] performed by method any2ArrowAssoc in scala.Predef.
    Definition Classes
    ArrowAssoc

Shadowed Implicit Value Members

  1. val self: Any

    Implicit information
    This member is added by an implicit conversion from CassandraJoinRDD[L, R] to StringAdd performed by method any2stringadd in scala.Predef.
    Shadowing
    This implicitly inherited member is ambiguous. One or more implicitly inherited members have similar signatures, so calling this member may produce an ambiguous implicit conversion compiler error.
    To access this member you can use a type ascription:
    (cassandraJoinRDD: StringAdd).self
    Definition Classes
    StringAdd
  2. val self: Any

    Implicit information
    This member is added by an implicit conversion from CassandraJoinRDD[L, R] to StringFormat performed by method any2stringfmt in scala.Predef.
    Shadowing
    This implicitly inherited member is ambiguous. One or more implicitly inherited members have similar signatures, so calling this member may produce an ambiguous implicit conversion compiler error.
    To access this member you can use a type ascription:
    (cassandraJoinRDD: StringFormat).self
    Definition Classes
    StringFormat
  3. val sparkContext: SparkContext

    Implicit information
    This member is added by an implicit conversion from CassandraJoinRDD[L, R] to RDDFunctions[(L, R)] performed by method toRDDFunctions in com.datastax.spark.connector.
    Shadowing
    This implicitly inherited member is shadowed by one or more members in this class.
    To access this member you can use a type ascription:
    (cassandraJoinRDD: RDDFunctions[(L, R)]).sparkContext
    Definition Classes
    RDDFunctionsWritableToCassandra

Deprecated Value Members

  1. def filterWith[A](constructA: (Int) ⇒ A)(p: ((L, R), A) ⇒ Boolean): RDD[(L, R)]

    Definition Classes
    RDD
    Annotations
    @deprecated
    Deprecated

    (Since version 1.0.0) use mapPartitionsWithIndex and filter

  2. def flatMapWith[A, U](constructA: (Int) ⇒ A, preservesPartitioning: Boolean)(f: ((L, R), A) ⇒ Seq[U])(implicit arg0: ClassTag[U]): RDD[U]

    Definition Classes
    RDD
    Annotations
    @deprecated
    Deprecated

    (Since version 1.0.0) use mapPartitionsWithIndex and flatMap

  3. def foreachWith[A](constructA: (Int) ⇒ A)(f: ((L, R), A) ⇒ Unit): Unit

    Definition Classes
    RDD
    Annotations
    @deprecated
    Deprecated

    (Since version 1.0.0) use mapPartitionsWithIndex and foreach

  4. def mapPartitionsWithContext[U](f: (TaskContext, Iterator[(L, R)]) ⇒ Iterator[U], preservesPartitioning: Boolean)(implicit arg0: ClassTag[U]): RDD[U]

    Definition Classes
    RDD
    Annotations
    @DeveloperApi() @deprecated
    Deprecated

    (Since version 1.2.0) use TaskContext.get

  5. def mapPartitionsWithSplit[U](f: (Int, Iterator[(L, R)]) ⇒ Iterator[U], preservesPartitioning: Boolean)(implicit arg0: ClassTag[U]): RDD[U]

    Definition Classes
    RDD
    Annotations
    @deprecated
    Deprecated

    (Since version 0.7.0) use mapPartitionsWithIndex

  6. def mapWith[A, U](constructA: (Int) ⇒ A, preservesPartitioning: Boolean)(f: ((L, R), A) ⇒ U)(implicit arg0: ClassTag[U]): RDD[U]

    Definition Classes
    RDD
    Annotations
    @deprecated
    Deprecated

    (Since version 1.0.0) use mapPartitionsWithIndex

  7. def toArray(): Array[(L, R)]

    Definition Classes
    RDD
    Annotations
    @deprecated
    Deprecated

    (Since version 1.0.0) use collect

  8. def x: CassandraJoinRDD[L, R]

    Implicit information
    This member is added by an implicit conversion from CassandraJoinRDD[L, R] to ArrowAssoc[CassandraJoinRDD[L, R]] performed by method any2ArrowAssoc in scala.Predef.
    Shadowing
    This implicitly inherited member is ambiguous. One or more implicitly inherited members have similar signatures, so calling this member may produce an ambiguous implicit conversion compiler error.
    To access this member you can use a type ascription:
    (cassandraJoinRDD: ArrowAssoc[CassandraJoinRDD[L, R]]).x
    Definition Classes
    ArrowAssoc
    Annotations
    @deprecated
    Deprecated

    (Since version 2.10.0) Use leftOfArrow instead

  9. def x: CassandraJoinRDD[L, R]

    Implicit information
    This member is added by an implicit conversion from CassandraJoinRDD[L, R] to Ensuring[CassandraJoinRDD[L, R]] performed by method any2Ensuring in scala.Predef.
    Shadowing
    This implicitly inherited member is ambiguous. One or more implicitly inherited members have similar signatures, so calling this member may produce an ambiguous implicit conversion compiler error.
    To access this member you can use a type ascription:
    (cassandraJoinRDD: Ensuring[CassandraJoinRDD[L, R]]).x
    Definition Classes
    Ensuring
    Annotations
    @deprecated
    Deprecated

    (Since version 2.10.0) Use resultOfEnsuring instead

Inherited from CassandraRDD[(L, R)]

Inherited from RDD[(L, R)]

Inherited from Logging

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Inherited by implicit conversion toPairRDDFunctions from CassandraJoinRDD[L, R] to PairRDDFunctions[L, R]

Inherited by implicit conversion toRDDFunctions from CassandraJoinRDD[L, R] to RDDFunctions[(L, R)]

Inherited by implicit conversion any2stringadd from CassandraJoinRDD[L, R] to StringAdd

Inherited by implicit conversion any2stringfmt from CassandraJoinRDD[L, R] to StringFormat

Inherited by implicit conversion any2ArrowAssoc from CassandraJoinRDD[L, R] to ArrowAssoc[CassandraJoinRDD[L, R]]

Inherited by implicit conversion any2Ensuring from CassandraJoinRDD[L, R] to Ensuring[CassandraJoinRDD[L, R]]

Ungrouped