Packages

package util

Useful stuff that didn't fit elsewhere.

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. util
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Type Members

  1. class BufferedIterator2[T] extends Iterator[T]

    Serves the same purpose as BufferedIterator in Scala, but its takeWhile method properly doesn't consume the next element.

  2. class ConfigParameter[T] extends DataFrameOption with Serializable
  3. class CountingIterator[T] extends Iterator[T]

    Counts elements fetched form the underlying iterator.

    Counts elements fetched form the underlying iterator. Limit causes iterator to terminate early

  4. trait DataFrameOption extends AnyRef
  5. class DeprecatedConfigParameter[N] extends Logging with Serializable

    Class representing a Config Parameter no longer in use and it's replacement if any.

    Class representing a Config Parameter no longer in use and it's replacement if any. Use rational to display more information about the deprecation and what to do for the end user.

  6. class MergeJoinIterator[L, R, K] extends Iterator[(K, Seq[L], Seq[R])]

    An iterator that preforms a mergeJoin between two ordered iterators joining on a given key.

    An iterator that preforms a mergeJoin between two ordered iterators joining on a given key. The input iterators are assumed to be ordered so we can do a greedy merge join. Since our iterators are lazy we cannot check that they are ordered before starting.

    Example:

    val list1 = Seq( (1, "a"), (2, "a") , (3, "a") )
    val list2 = Seq( (1, "b"), (2, "b") , (3, "b") )
    val iterator = new MergeJoinIterator(
      list1.iterator,
      list2.iterator,
      (x: (Int, String)) => x._1,
      (y: (Int, String)) => y._1
    )
    val result = iterator.toSeq
    // (1, Seq((1, "a")), Seq((1, "b"))),
    // (2, Seq((2, "a")), Seq((2, "b"))),
    // (3, Seq((3, "a")), Seq((3, "b")))
  7. class MultiMergeJoinIterator[T, K] extends Iterator[Seq[Seq[T]]]

    An iterator that preforms a mergeJoin among ordered iterators joining on a given key.

    An iterator that preforms a mergeJoin among ordered iterators joining on a given key. The input iterators are assumed to be ordered so we can do a greedy merge join. Since our iterators are lazy we cannot check that they are ordered before starting.

    Example:

    val list1 = Seq( (1, "a"), (2, "a") , (3, "a") )
    val list2 = Seq( (1, "b"), (2, "b") , (3, "b") )
    val iterator = new MultiMergeJoinIterator(
      Seq(list1.iterator,list2.iterator),
      (x: (Int, String)) => x._1,
    )
    val result = iterator.toSeq
    // (Seq((1, "a")), Seq((1, "b"))),
    // (Seq((2, "a")), Seq((2, "b"))),
    // (Seq((3, "a")), Seq((3, "b")))
  8. final class PriorityHashMap[K, V] extends AnyRef

    A HashMap and a PriorityQueue hybrid.

    A HashMap and a PriorityQueue hybrid. Works like a HashMap but offers additional O(1) access to the entry with the highest value. As in a standard HashMap, entries can be looked up by key in O(1) time. Adding, removing and updating items by key is handled in O(log n) time.

    Keys must not be changed externally and must implement proper equals and hashCode. It is advised to use immutable classes for keys.

    Values must be properly comparable. Values may be externally mutated as long as a proper immediate call to put is issued to notify the PriorityHashMap that the value associated with the given key has changed, after each value mutation. It is not allowed to externally mutate more than one value at a time or to mutate a value associated with multiple keys. Therefore, it is advised to use immutable classes for values, and updating values only by calls to put.

    Contrary to standard Java HashMap implementation, PriorityHashMap does not allocate memory on adding / removing / updating items and stores all data in flat, non-resizable arrays instead. Therefore its capacity cannot be modified after construction. It is technically possible to remove this limitation in the future.

    PriorityHashMap is mutable and not thread-safe.

    Internally, PriorityHashMap is composed of the following data arrays: - an array storing references to keys, forming a heap-based priority queue; - an array storing corresponding references to values, always in the same order as keys; - an array storing indexes into the first two arrays, used as an inline hash-table allowing to quickly locate keys in the heap in constant time; - an array for fast translating indexes in the heap into indexes into hash-table, so after moving a key/value in the heap, the corresponding index in the hash-table can be quickly updated, without hashing.

    The indexes hash-table doesn't use overflow lists for dealing with hash collisions. The overflow entries are placed in the main hash-table array in the first not-taken entry to the right from the original position pointed by key hash. On search, if the key is not found immediately at a position pointed by key hash, it is searched to the right, until it is found or an empty array entry is found.

    K

    type of keys

    V

    type of values; values must be comparable

  9. class SpanningIterator[K, T] extends Iterator[(K, Seq[T])]

    An iterator that groups items having the same value of the given function (key).

    An iterator that groups items having the same value of the given function (key). To be included in the same group, items with the same key must be next to each other in the original collection.

    SpanningIterator buffers internally one group at a time and the wrapped iterator is consumed in a lazy way.

    Example:

    val collection = Seq(1 -> "a", 1 -> "b", 1 -> "c", 2 -> "d", 2 -> "e")
    val iterator = new SpanningIterator(collection.iterator, (x: (Int, String)) => x._1)
    val result = iterator.toSeq  // Seq(1 -> Seq("a", "b", "c"), 2 -> Seq("d", "e"))

Value Members

  1. def maybeExecutingAs[StatementT <: Statement[StatementT]](stmt: StatementT, proxyUser: Option[String]): StatementT
  2. def schemaFromCassandra(connector: CassandraConnector, keyspaceName: Option[String] = None, tableName: Option[String] = None): Schema
  3. def tableFromCassandra(connector: CassandraConnector, keyspaceName: String, tableName: String): TableDef
  4. object ClassLoaderCheck

    Do not remove.

    Do not remove. This is a temporary marker class that is loaded by name during application shutdown. See SPARKC-620 for details.

  5. object CodecRegistryUtil
  6. object ConfigCheck

    Helper class to throw exceptions if there are environment variables in the spark.cassandra namespace which don't map to Spark Cassandra Connector known properties.

  7. object ConfigParameter extends Serializable
  8. object CqlWhereParser extends RegexParsers with Logging
  9. object DeprecatedConfigParameter extends Serializable
  10. object JavaApiHelper

    A helper class to make it possible to access components written in Scala from Java code.

    A helper class to make it possible to access components written in Scala from Java code. INTERNAL API

  11. object MagicalTypeTricks
  12. object PatitionKeyTools
  13. object RefBuilder
  14. object Threads extends Logging

Inherited from AnyRef

Inherited from Any

Ungrouped