org.apache.spark.streaming.api

JavaSnappyStreamingContext

class JavaSnappyStreamingContext extends JavaStreamingContext

Linear Supertypes
JavaStreamingContext, Closeable, AutoCloseable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By inheritance
Inherited
  1. JavaSnappyStreamingContext
  2. JavaStreamingContext
  3. Closeable
  4. AutoCloseable
  5. AnyRef
  6. Any
  1. Hide All
  2. Show all
Learn more about member selection
Visibility
  1. Public
  2. All

Instance Constructors

  1. new JavaSnappyStreamingContext(path: String, sparkContext: JavaSparkContext)

    Recreate a JavaSnappyStreamingContext from a checkpoint file using an existing SparkContext.

    Recreate a JavaSnappyStreamingContext from a checkpoint file using an existing SparkContext.

    path

    Path to the directory that was specified as the checkpoint directory

    sparkContext

    Existing SparkContext

  2. new JavaSnappyStreamingContext(path: String)

    Recreate a JavaSnappyStreamingContext from a checkpoint file.

    Recreate a JavaSnappyStreamingContext from a checkpoint file.

    path

    Path to the directory that was specified as the checkpoint directory

  3. new JavaSnappyStreamingContext(path: String, hadoopConf: Configuration)

    Recreate a JavaSnappyStreamingContext from a checkpoint file.

    Recreate a JavaSnappyStreamingContext from a checkpoint file.

    path

    Path to the directory that was specified as the checkpoint directory

    hadoopConf

    Optional, configuration object if necessary for reading from HDFS compatible filesystems

  4. new JavaSnappyStreamingContext(conf: SparkConf, batchDuration: Duration)

    Create a JavaSnappyStreamingContext by providing the configuration necessary for a new SparkContext.

    Create a JavaSnappyStreamingContext by providing the configuration necessary for a new SparkContext.

    conf

    a org.apache.spark.SparkConf object specifying Spark parameters

    batchDuration

    the time interval at which streaming data will be divided into batches

  5. new JavaSnappyStreamingContext(sparkContext: JavaSparkContext, batchDuration: Duration)

    Create a JavaSnappyStreamingContext using an existing SparkContext.

    Create a JavaSnappyStreamingContext using an existing SparkContext.

    sparkContext

    existing SparkContext

    batchDuration

    the time interval at which streaming data will be divided into batches

  6. new JavaSnappyStreamingContext(sparkContext: JavaSparkContext, checkpoint: Checkpoint, batchDuration: Duration)

    Create a JavaSnappyStreamingContext using an existing SparkContext.

    Create a JavaSnappyStreamingContext using an existing SparkContext.

    sparkContext

    existing SparkContext

    checkpoint

    checkpoint directory

    batchDuration

    the time interval at which streaming data will be divided into batches

  7. new JavaSnappyStreamingContext(snsc: SnappyStreamingContext)

Value Members

  1. final def !=(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  2. final def !=(arg0: Any): Boolean

    Definition Classes
    Any

  3. final def ##(): Int

    Definition Classes
    AnyRef → Any

  4. final def ==(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  5. final def ==(arg0: Any): Boolean

    Definition Classes
    Any

  6. def actorStream[T](props: Props, name: String): JavaReceiverInputDStream[T]

    Definition Classes
    JavaStreamingContext

  7. def actorStream[T](props: Props, name: String, storageLevel: StorageLevel): JavaReceiverInputDStream[T]

    Definition Classes
    JavaStreamingContext

  8. def actorStream[T](props: Props, name: String, storageLevel: StorageLevel, supervisorStrategy: SupervisorStrategy): JavaReceiverInputDStream[T]

    Definition Classes
    JavaStreamingContext

  9. def addStreamingListener(streamingListener: StreamingListener): Unit

    Definition Classes
    JavaStreamingContext

  10. final def asInstanceOf[T0]: T0

    Definition Classes
    Any

  11. def awaitTermination(): Unit

    Definition Classes
    JavaStreamingContext

  12. def awaitTerminationOrTimeout(timeout: Long): Boolean

    Definition Classes
    JavaStreamingContext

  13. def binaryRecordsStream(directory: String, recordLength: Int): JavaDStream[Array[Byte]]

    Definition Classes
    JavaStreamingContext

  14. def checkpoint(directory: String): Unit

    Definition Classes
    JavaStreamingContext

  15. def clone(): AnyRef

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  16. def close(): Unit

    Definition Classes
    JavaStreamingContext → Closeable → AutoCloseable

  17. def createSchemaDStream(rowStream: JavaDStream[_], beanClass: Class[_]): SchemaDStream

  18. final def eq(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  19. def equals(arg0: Any): Boolean

    Definition Classes
    AnyRef → Any

  20. def fileStream[K, V, F <: InputFormat[K, V]](directory: String, kClass: Class[K], vClass: Class[V], fClass: Class[F], filter: Function[Path, Boolean], newFilesOnly: Boolean, conf: Configuration): JavaPairInputDStream[K, V]

    Definition Classes
    JavaStreamingContext

  21. def fileStream[K, V, F <: InputFormat[K, V]](directory: String, kClass: Class[K], vClass: Class[V], fClass: Class[F], filter: Function[Path, Boolean], newFilesOnly: Boolean): JavaPairInputDStream[K, V]

    Definition Classes
    JavaStreamingContext

  22. def fileStream[K, V, F <: InputFormat[K, V]](directory: String, kClass: Class[K], vClass: Class[V], fClass: Class[F]): JavaPairInputDStream[K, V]

    Definition Classes
    JavaStreamingContext

  23. def finalize(): Unit

    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )

  24. final def getClass(): Class[_]

    Definition Classes
    AnyRef → Any

  25. def getSchemaDStream(tableName: String): SchemaDStream

  26. def getState(): StreamingContextState

    Definition Classes
    JavaStreamingContext

  27. def hashCode(): Int

    Definition Classes
    AnyRef → Any

  28. final def isInstanceOf[T0]: Boolean

    Definition Classes
    Any

  29. final def ne(arg0: AnyRef): Boolean

    Definition Classes
    AnyRef

  30. final def notify(): Unit

    Definition Classes
    AnyRef

  31. final def notifyAll(): Unit

    Definition Classes
    AnyRef

  32. def queueStream[T](queue: Queue[JavaRDD[T]], oneAtATime: Boolean, defaultRDD: JavaRDD[T]): JavaInputDStream[T]

    Definition Classes
    JavaStreamingContext

  33. def queueStream[T](queue: Queue[JavaRDD[T]], oneAtATime: Boolean): JavaInputDStream[T]

    Definition Classes
    JavaStreamingContext

  34. def queueStream[T](queue: Queue[JavaRDD[T]]): JavaDStream[T]

    Definition Classes
    JavaStreamingContext

  35. def rawSocketStream[T](hostname: String, port: Int): JavaReceiverInputDStream[T]

    Definition Classes
    JavaStreamingContext

  36. def rawSocketStream[T](hostname: String, port: Int, storageLevel: StorageLevel): JavaReceiverInputDStream[T]

    Definition Classes
    JavaStreamingContext

  37. def receiverStream[T](receiver: Receiver[T]): JavaReceiverInputDStream[T]

    Definition Classes
    JavaStreamingContext

  38. def registerCQ(queryStr: String): SchemaDStream

    Registers and executes given SQL query and returns SchemaDStream to consume the results

    Registers and executes given SQL query and returns SchemaDStream to consume the results

    queryStr
    returns

  39. def remember(duration: Duration): Unit

    Definition Classes
    JavaStreamingContext

  40. val snsc: SnappyStreamingContext

  41. def socketStream[T](hostname: String, port: Int, converter: Function[InputStream, Iterable[T]], storageLevel: StorageLevel): JavaReceiverInputDStream[T]

    Definition Classes
    JavaStreamingContext

  42. def socketTextStream(hostname: String, port: Int): JavaReceiverInputDStream[String]

    Definition Classes
    JavaStreamingContext

  43. def socketTextStream(hostname: String, port: Int, storageLevel: StorageLevel): JavaReceiverInputDStream[String]

    Definition Classes
    JavaStreamingContext

  44. val sparkContext: JavaSparkContext

    Definition Classes
    JavaStreamingContext

  45. def sql(sqlText: String): DataFrame

  46. val ssc: StreamingContext

    Definition Classes
    JavaStreamingContext

  47. def start(): Unit

    Start the execution of the streams.

    Start the execution of the streams. Also registers population of AQP tables from stream tables if present.

    Definition Classes
    JavaSnappyStreamingContext → JavaStreamingContext
    Exceptions thrown
    IllegalStateException

    if the JavaSnappyStreamingContext is already stopped

  48. def stop(stopSparkContext: Boolean, stopGracefully: Boolean): Unit

    Definition Classes
    JavaSnappyStreamingContext → JavaStreamingContext

  49. def stop(stopSparkContext: Boolean): Unit

    Definition Classes
    JavaStreamingContext

  50. def stop(): Unit

    Definition Classes
    JavaStreamingContext

  51. final def synchronized[T0](arg0: ⇒ T0): T0

    Definition Classes
    AnyRef

  52. def textFileStream(directory: String): JavaDStream[String]

    Definition Classes
    JavaStreamingContext

  53. def toString(): String

    Definition Classes
    AnyRef → Any

  54. def transform[T](dstreams: List[JavaDStream[_]], transformFunc: Function2[List[JavaRDD[_]], Time, JavaRDD[T]]): JavaDStream[T]

    Definition Classes
    JavaStreamingContext

  55. def transformToPair[K, V](dstreams: List[JavaDStream[_]], transformFunc: Function2[List[JavaRDD[_]], Time, JavaPairRDD[K, V]]): JavaPairDStream[K, V]

    Definition Classes
    JavaStreamingContext

  56. def union[K, V](first: JavaPairDStream[K, V], rest: List[JavaPairDStream[K, V]]): JavaPairDStream[K, V]

    Definition Classes
    JavaStreamingContext

  57. def union[T](first: JavaDStream[T], rest: List[JavaDStream[T]]): JavaDStream[T]

    Definition Classes
    JavaStreamingContext

  58. final def wait(): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  59. final def wait(arg0: Long, arg1: Int): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

  60. final def wait(arg0: Long): Unit

    Definition Classes
    AnyRef
    Annotations
    @throws( ... )

Deprecated Value Members

  1. def awaitTermination(timeout: Long): Unit

    Definition Classes
    JavaStreamingContext
    Annotations
    @deprecated
    Deprecated

    (Since version 1.3.0) Use awaitTerminationOrTimeout(Long) instead

  2. val sc: JavaSparkContext

    Definition Classes
    JavaStreamingContext
    Annotations
    @deprecated
    Deprecated

    (Since version 0.9.0) use sparkContext

Inherited from JavaStreamingContext

Inherited from Closeable

Inherited from AutoCloseable

Inherited from AnyRef

Inherited from Any

Ungrouped