Package

com.datastax.spark.connector

writer

Permalink

package writer

Contains components for writing RDDs to Cassandra

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. writer
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Type Members

  1. class AsyncExecutor[T, R] extends Logging

    Permalink

    Asynchronously executes tasks but blocks if the limit of unfinished tasks is reached.

  2. sealed trait BatchGroupingKey extends AnyRef

    Permalink
  3. class CassandraRowWriter extends RowWriter[CassandraRow]

    Permalink

    A RowWriter that can write CassandraRow objects.

  4. class DefaultRowWriter[T] extends RowWriter[T]

    Permalink

    A RowWriter suitable for saving objects mappable by a ColumnMapper.

    A RowWriter suitable for saving objects mappable by a ColumnMapper. Can save case class objects, java beans and tuples.

  5. trait LowPriorityRowWriterFactoryImplicits extends AnyRef

    Permalink

    Provides a low-priority implicit RowWriterFactory able to write objects of any class for which a ColumnMapper is defined.

  6. case class NullKeyColumnException(columnName: String) extends NullPointerException with Product with Serializable

    Permalink
  7. case class PerRowWriteOptionValue[T](placeholder: String) extends WriteOptionValue[T] with Product with Serializable

    Permalink
  8. class QueryExecutor extends AsyncExecutor[RichStatement, AsyncResultSet]

    Permalink
  9. class RateLimiter extends Serializable

    Permalink

    A leaking bucket rate limiter.

    A leaking bucket rate limiter. It can be used to limit rate of anything, but typically it is used to limit rate of data transfer.

    It starts with an empty bucket. When packets arrive, they are added to the bucket. The bucket has a constant size and is leaking at a constant rate. If the bucket overflows, the thread is delayed by the amount of time proportional to the amount of the overflow.

    This class is thread safe and lockless.

  10. class ReplicaLocator[T] extends Serializable with Logging

    Permalink

    A utility class for determining the Replica Set (Ip Addresses) of a particular Cassandra Row.

    A utility class for determining the Replica Set (Ip Addresses) of a particular Cassandra Row. Used by the com.datastax.spark.connector.RDDFunctions.keyByCassandraReplica method. Uses the Java Driver to obtain replica information.

  11. trait RichStatement extends AnyRef

    Permalink
  12. trait RowWriter[T] extends Serializable

    Permalink

    RowWriter knows how to extract column names and values from custom row objects and how to convert them to values that can be written to Cassandra.

    RowWriter knows how to extract column names and values from custom row objects and how to convert them to values that can be written to Cassandra. RowWriter is required to apply any user-defined data type conversion.

  13. trait RowWriterFactory[T] extends AnyRef

    Permalink

    Creates instances of RowWriter objects for the given row type T.

    Creates instances of RowWriter objects for the given row type T. RowWriterFactory is the trait you need to implement if you want to support row representations which cannot be simply mapped by a ColumnMapper.

  14. class SqlRowWriter extends RowWriter[Row]

    Permalink

    A RowWriter that can write SparkSQL Row objects.

  15. case class StaticWriteOptionValue[T](value: T) extends WriteOptionValue[T] with Product with Serializable

    Permalink
  16. case class TTLOption(value: WriteOptionValue[Int]) extends WriteOption[Int] with Product with Serializable

    Permalink
  17. class TableWriter[T] extends Serializable with Logging

    Permalink

    Writes RDD data into given Cassandra table.

    Writes RDD data into given Cassandra table. Individual column values are extracted from RDD objects using given RowWriter Then, data are inserted into Cassandra with batches of CQL INSERT statements. Each RDD partition is processed by a single thread.

  18. case class TimestampOption(value: WriteOptionValue[Long]) extends WriteOption[Long] with Product with Serializable

    Permalink
  19. abstract class WritableToCassandra[T] extends AnyRef

    Permalink
  20. case class WriteConf(batchSize: BatchSize = BatchSize.Automatic, batchGroupingBufferSize: Int = ..., batchGroupingKey: BatchGroupingKey = WriteConf.BatchLevelParam.default, consistencyLevel: ConsistencyLevel = ..., ifNotExists: Boolean = WriteConf.IfNotExistsParam.default, ignoreNulls: Boolean = WriteConf.IgnoreNullsParam.default, parallelismLevel: Int = ..., throughputMiBPS: Option[Double] = ..., ttl: TTLOption = TTLOption.defaultValue, timestamp: TimestampOption = TimestampOption.defaultValue, taskMetricsEnabled: Boolean = WriteConf.TaskMetricsParam.default, executeAs: Option[String] = None) extends Product with Serializable

    Permalink

    Write settings for RDD

    Write settings for RDD

    batchSize

    approx. number of bytes to be written in a single batch or exact number of rows to be written in a single batch;

    batchGroupingBufferSize

    the number of distinct batches that can be buffered before they are written to Cassandra

    batchGroupingKey

    which rows can be grouped into a single batch

    consistencyLevel

    consistency level for writes, default LOCAL_QUORUM

    ifNotExists

    inserting a row should happen only if it does not already exist

    parallelismLevel

    number of batches to be written in parallel

    ttl

    the default TTL value which is used when it is defined (in seconds)

    timestamp

    the default timestamp value which is used when it is defined (in microseconds)

    taskMetricsEnabled

    whether or not enable task metrics updates (requires Spark 1.2+)

  21. sealed trait WriteOption[+T] extends AnyRef

    Permalink
  22. sealed trait WriteOptionValue[+T] extends AnyRef

    Permalink

Value Members

  1. object AsyncExecutor

    Permalink
  2. object BatchGroupingKey

    Permalink
  3. object CassandraRowWriter extends Serializable

    Permalink
  4. object DefaultRowWriter extends Serializable

    Permalink
  5. object DefaultValue extends WriteOptionValue[Nothing] with Product with Serializable

    Permalink
  6. object ObjectSizeEstimator

    Permalink

    Estimates amount of memory required to serialize Java/Scala objects

  7. object QueryExecutor

    Permalink
  8. object ReplicaLocator extends Serializable

    Permalink

    Helper methods for mapping a set of data to their relative locations in a Cassandra Cluster.

  9. object RichStatement

    Permalink
  10. object RowWriterFactory extends LowPriorityRowWriterFactoryImplicits

    Permalink

    Provides an implicit RowWriterFactory for saving CassandraRow objects.

  11. object SqlRowWriter extends Serializable

    Permalink
  12. object TTLOption extends Serializable

    Permalink
  13. object TableWriter extends Serializable

    Permalink
  14. object TimestampOption extends Serializable

    Permalink
  15. object WriteConf extends Serializable

    Permalink
  16. object WriteOption

    Permalink

Inherited from AnyRef

Inherited from Any

Ungrouped