Packages

class CassandraCatalog extends CatalogPlugin with TableCatalog with SupportsNamespaces with Logging

A Spark Sql Catalog for inter-operation with Cassandra

Namespaces naturally map to C* Keyspaces, but they are always only a single element deep.

Linear Supertypes
Logging, SupportsNamespaces, TableCatalog, CatalogPlugin, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. CassandraCatalog
  2. Logging
  3. SupportsNamespaces
  4. TableCatalog
  5. CatalogPlugin
  6. AnyRef
  7. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new CassandraCatalog()

Abstract Value Members

  1. abstract def alterNamespace(arg0: Array[String], arg1: <repeated...>[NamespaceChange]): Unit
    Definition Classes
    SupportsNamespaces
    Annotations
    @throws( ... ) @transient()
  2. abstract def alterTable(arg0: Identifier, arg1: <repeated...>[TableChange]): Table
    Definition Classes
    TableCatalog
    Annotations
    @throws( ... ) @transient()

Concrete Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. def alterNamespace(namespace: Array[String], changes: NamespaceChange*): Unit
  5. def alterTable(ident: Identifier, changes: TableChange*): Table

    Limited to Adding and Removing Normal Columns and Setting Properties

    Limited to Adding and Removing Normal Columns and Setting Properties

    The API expects that if any change is rejected, we should not apply any changes. This is basically impossible for us since we cannot batch together all of our DDL changes into a single CQL statement. To ameliorate this we perform as many checks as we can up front to try to avoid beginning our DDL requests which we know will fail.

    We break up DDL into 3 phases Set Properties Remove Columns Add Columns

  6. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  7. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native() @HotSpotIntrinsicCandidate()
  8. def createNamespace(namespace: Array[String], metadata: Map[String, String]): Unit

    Builds a Cassandra Keyspace based off the metadata provided by SparkSQL.

    Builds a Cassandra Keyspace based off the metadata provided by SparkSQL. There are currently only two available replication strategies and their syntax mimics CQL

    "class" -> "NetworkTopologyStrategy", "dcName" -> "rfNum"

    All additional NTS properties will be considered datacenter Names (except durable_writes and replication_factor)

    or

    "class" -> "SimpleStrategy", "replication_factor" -> "rfNum"

    In either case

    "durable_writes" -> "true|false"

    is also allowed to disable the commitlog (see Cassandra Docs defaults to true)

    Definition Classes
    CassandraCatalog → SupportsNamespaces
  9. def createTable(ident: Identifier, schema: StructType, partitions: Array[Transform], properties: Map[String, String]): Table

    Creates a Cassandra Table

    Creates a Cassandra Table

    Uses properties "partition_key" and "clustering_key" to set partition key and clustering key respectively for each of these properties.

    Partition key is defined as a string of comma-separated column identifiers: "col_a, col_b, col_b" Clustering key is defined as a string of comma-separated column identifiers optionally marked with clustering order: "col_a.ASC, col_b.DESC"

    Additional options can be specified as table properties For Example caching='{keys=ALL,rows_per_partition=42}', default_time_to_live='33', compaction='{class=SizeTieredCompactionStrategy,bucket_high=42}'

    Definition Classes
    CassandraCatalog → TableCatalog
  10. def defaultNamespace(): Array[String]
    Definition Classes
    CatalogPlugin
  11. def dropNamespace(namespace: Array[String]): Boolean
    Definition Classes
    CassandraCatalog → SupportsNamespaces
  12. def dropTable(ident: Identifier): Boolean
    Definition Classes
    CassandraCatalog → TableCatalog
  13. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  14. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  15. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  16. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  17. def initialize(name: String, options: CaseInsensitiveStringMap): Unit
    Definition Classes
    CassandraCatalog → CatalogPlugin
  18. def invalidateTable(arg0: Identifier): Unit
    Definition Classes
    TableCatalog
  19. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  20. def isTraceEnabled(): Boolean
    Attributes
    protected
    Definition Classes
    Logging
  21. def listNamespaces(namespace: Array[String]): Array[Array[String]]

    Since we only allow single depth keyspace identifiers in C* we always either return an empty list of namespaces or throw a NoSuchNamespaceException

    Since we only allow single depth keyspace identifiers in C* we always either return an empty list of namespaces or throw a NoSuchNamespaceException

    Definition Classes
    CassandraCatalog → SupportsNamespaces
  22. def listNamespaces(): Array[Array[String]]
    Definition Classes
    CassandraCatalog → SupportsNamespaces
  23. def listTables(namespace: Array[String]): Array[Identifier]
    Definition Classes
    CassandraCatalog → TableCatalog
  24. def loadNamespaceMetadata(namespace: Array[String]): Map[String, String]
    Definition Classes
    CassandraCatalog → SupportsNamespaces
  25. def loadTable(ident: Identifier): Table
    Definition Classes
    CassandraCatalog → TableCatalog
  26. def log: Logger
    Attributes
    protected
    Definition Classes
    Logging
  27. def logDebug(msg: ⇒ String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  28. def logDebug(msg: ⇒ String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  29. def logError(msg: ⇒ String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  30. def logError(msg: ⇒ String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  31. def logInfo(msg: ⇒ String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  32. def logInfo(msg: ⇒ String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  33. def logName: String
    Attributes
    protected
    Definition Classes
    Logging
  34. def logTrace(msg: ⇒ String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  35. def logTrace(msg: ⇒ String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  36. def logWarning(msg: ⇒ String, throwable: Throwable): Unit
    Attributes
    protected
    Definition Classes
    Logging
  37. def logWarning(msg: ⇒ String): Unit
    Attributes
    protected
    Definition Classes
    Logging
  38. def name(): String
    Definition Classes
    CassandraCatalog → CatalogPlugin
  39. def namespaceExists(arg0: Array[String]): Boolean
    Definition Classes
    SupportsNamespaces
  40. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  41. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  42. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @HotSpotIntrinsicCandidate()
  43. def purgeTable(arg0: Identifier): Boolean
    Definition Classes
    TableCatalog
    Annotations
    @throws( ... )
  44. def renameTable(oldIdent: Identifier, newIdent: Identifier): Unit
    Definition Classes
    CassandraCatalog → TableCatalog
  45. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  46. def tableExists(arg0: Identifier): Boolean
    Definition Classes
    TableCatalog
  47. def toString(): String
    Definition Classes
    AnyRef → Any
  48. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  49. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  50. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Deprecated Value Members

  1. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] ) @Deprecated @deprecated
    Deprecated

    (Since version ) see corresponding Javadoc for more information.

Inherited from Logging

Inherited from SupportsNamespaces

Inherited from TableCatalog

Inherited from CatalogPlugin

Inherited from AnyRef

Inherited from Any

Ungrouped