class StreamingContextFunctions extends SparkContextFunctions
Provides Cassandra-specific methods on org.apache.spark.streaming.StreamingContext
.
- Alphabetic
- By Inheritance
- StreamingContextFunctions
- SparkContextFunctions
- Serializable
- Serializable
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Instance Constructors
-
new
StreamingContextFunctions(ssc: StreamingContext)
- ssc
the Spark Streaming context
Value Members
-
final
def
!=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
##(): Int
- Definition Classes
- AnyRef → Any
-
final
def
==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
asInstanceOf[T0]: T0
- Definition Classes
- Any
-
def
cassandraTable[T](keyspace: String, table: String)(implicit connector: CassandraConnector = ..., readConf: ReadConf = ReadConf.fromSparkConf(sc.getConf), ct: ClassTag[T], rrf: RowReaderFactory[T], ev: ValidRDDType[T]): CassandraStreamingRDD[T]
Returns a view of a Cassandra table as
CassandraRDD
.Returns a view of a Cassandra table as
CassandraRDD
. This method is made available on SparkContext by importingcom.datastax.spark.connector._
Depending on the type parameter passed to
cassandraTable
, every row is converted to one of the following:- an CassandraRow object (default, if no type given)
- a tuple containing column values in the same order as columns selected by CassandraRDD#select
- object of a user defined class, populated by appropriate ColumnMapper
Example:
CREATE KEYSPACE test WITH replication = {'class': 'SimpleStrategy', 'replication_factor': 1 }; CREATE TABLE test.words (word text PRIMARY KEY, count int); INSERT INTO test.words (word, count) VALUES ('foo', 20); INSERT INTO test.words (word, count) VALUES ('bar', 20); ...
// Obtaining RDD of CassandraRow objects: val rdd1 = sc.cassandraTable("test", "words") rdd1.first.getString("word") // foo rdd1.first.getInt("count") // 20 // Obtaining RDD of tuples: val rdd2 = sc.cassandraTable[(String, Int)]("test", "words").select("word", "count") rdd2.first._1 // foo rdd2.first._2 // 20 // Obtaining RDD of user defined objects: case class WordCount(word: String, count: Int) val rdd3 = sc.cassandraTable[WordCount]("test", "words") rdd3.first.word // foo rdd3.first.count // 20
- Definition Classes
- StreamingContextFunctions → SparkContextFunctions
-
def
clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native() @HotSpotIntrinsicCandidate()
-
def
emptyCassandraTable[T](keyspace: String, table: String)(implicit connector: CassandraConnector = CassandraConnector(sc), readConf: ReadConf = ReadConf.fromSparkConf(sc.getConf), ct: ClassTag[T], rrf: RowReaderFactory[T], ev: ValidRDDType[T]): EmptyCassandraRDD[T]
Produces the empty CassandraRDD which does not perform any validation and it does not even try to return any rows.
Produces the empty CassandraRDD which does not perform any validation and it does not even try to return any rows.
- Definition Classes
- SparkContextFunctions
-
final
def
eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
def
equals(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
getClass(): Class[_]
- Definition Classes
- AnyRef → Any
- Annotations
- @native() @HotSpotIntrinsicCandidate()
-
def
hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native() @HotSpotIntrinsicCandidate()
-
final
def
isInstanceOf[T0]: Boolean
- Definition Classes
- Any
-
final
def
ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
final
def
notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native() @HotSpotIntrinsicCandidate()
-
final
def
notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native() @HotSpotIntrinsicCandidate()
-
val
sc: SparkContext
- Definition Classes
- SparkContextFunctions
-
final
def
synchronized[T0](arg0: ⇒ T0): T0
- Definition Classes
- AnyRef
-
def
toString(): String
- Definition Classes
- AnyRef → Any
-
final
def
wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()
-
final
def
wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
Deprecated Value Members
-
def
finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( classOf[java.lang.Throwable] ) @Deprecated @deprecated
- Deprecated
(Since version ) see corresponding Javadoc for more information.