flink-scala-api
API
Generated with
flink-scala-api/org.apache.flinkx.api

org.apache.flinkx.api

package org.apache.flinkx.api

Members list

Packages

package org.apache.flinkx.api.async
package org.apache.flinkx.api.extension
package org.apache.flinkx.api.extensions.impl.acceptPartialFunctions
package org.apache.flinkx.api.function
package org.apache.flinkx.api.mapper
package org.apache.flinkx.api.serializer
package org.apache.flinkx.api.typeinfo

Type members

Classlikes

@Public
class AllWindowedStream[T, W <: Window](javaStream: AllWindowedStream[T, W])

A AllWindowedStream represents a data stream where the stream of elements is split into windows based on a org.apache.flink.streaming.api.windowing.assigners.WindowAssigner. Window emission is triggered based on a Trigger.

A AllWindowedStream represents a data stream where the stream of elements is split into windows based on a org.apache.flink.streaming.api.windowing.assigners.WindowAssigner. Window emission is triggered based on a Trigger.

If an Evictor is specified it will be used to evict elements from the window after evaluation was triggered by the Trigger but before the actual evaluation of the window. When using an evictor window performance will degrade significantly, since pre-aggregation of window results cannot be used.

Note that the AllWindowedStream is purely and API construct, during runtime the AllWindowedStream will be collapsed together with the operation over the window into one single operation.

Type parameters

T

The type of elements in the stream.

W

The type of Window that the org.apache.flink.streaming.api.windowing.assigners.WindowAssigner assigns the elements to.

Attributes

Supertypes
class Object
trait Matchable
class Any
@PublicEvolving
object AsyncDataStream

A helper class to apply AsyncFunction to a data stream.

A helper class to apply AsyncFunction to a data stream.

Example:

 val input: DataStream[String] = ...
 val asyncFunction: (String, ResultFuture[String]) => Unit = ...

 AsyncDataStream.orderedWait(input, asyncFunction, timeout, TimeUnit.MILLISECONDS, 100)

Attributes

Supertypes
class Object
trait Matchable
class Any
Self type
class BroadcastConnectedStream[IN1, IN2](javaStream: BroadcastConnectedStream[IN1, IN2])

Attributes

Supertypes
class Object
trait Matchable
class Any
trait CloseableIterator[T] extends Iterator[T], AutoCloseable

This interface represents an Iterator that is also AutoCloseable. A typical use-case for this interface are iterators that are based on native-resources such as files, network, or database connections. Clients must call close after using the iterator.

This interface represents an Iterator that is also AutoCloseable. A typical use-case for this interface are iterators that are based on native-resources such as files, network, or database connections. Clients must call close after using the iterator.

Attributes

Companion
object
Supertypes
trait AutoCloseable
trait Iterator[T]
trait IterableOnceOps[T, Iterator, Iterator[T]]
trait IterableOnce[T]
class Object
trait Matchable
class Any
Show all
object CloseableIterator

Attributes

Companion
trait
Supertypes
class Object
trait Matchable
class Any
Self type
object ClosureCleaner

A cleaner that renders closures serializable if they can be done so safely.

A cleaner that renders closures serializable if they can be done so safely.

Attributes

Supertypes
class Object
trait Matchable
class Any
Self type
@Public
class CoGroupedStreams[T1, T2](input1: DataStream[T1], input2: DataStream[T2])

CoGroupedStreams represents two DataStreams that have been co-grouped. A streaming co-group operation is evaluated over elements in a window.

CoGroupedStreams represents two DataStreams that have been co-grouped. A streaming co-group operation is evaluated over elements in a window.

To finalize the co-group operation you also need to specify a KeySelector for both the first and second input and a WindowAssigner

Note: Right now, the groups are being built in memory so you need to ensure that they don't get too big. Otherwise the JVM might crash.

Example:

val one: DataStream[(String, Int)]  = ...
val two: DataStream[(String, Int)] = ...

val result = one.coGroup(two)
   .where(new MyFirstKeySelector())
   .equalTo(new MyFirstKeySelector())
   .window(TumblingEventTimeWindows.of(Time.of(5, TimeUnit.SECONDS)))
   .apply(new MyCoGroupFunction())
}

Attributes

Supertypes
class Object
trait Matchable
class Any
trait CommonTaggedDerivation[TypeClass[_]]

Attributes

Supertypes
class Object
trait Matchable
class Any
Known subtypes
trait TaggedDerivation[TypeClass]
@Public
class ConnectedStreams[IN1, IN2](javaStream: ConnectedStreams[IN1, IN2])

ConnectedStreams represents two connected streams of (possibly) different data types. Connected streams are useful for cases where operations on one stream directly affect the operations on the other stream, usually via shared state between the streams.

ConnectedStreams represents two connected streams of (possibly) different data types. Connected streams are useful for cases where operations on one stream directly affect the operations on the other stream, usually via shared state between the streams.

An example for the use of connected streams would be to apply rules that change over time onto another stream. One of the connected streams has the rules, the other stream the elements to apply the rules to. The operation on the connected stream maintains the current set of rules in the state. It may receive either a rule update and update the state or a data element and apply the rules in the state to the element.

The connected stream can be conceptually viewed as a union stream of an Either type, that holds either the first stream's type or the second stream's type.

Attributes

Supertypes
class Object
trait Matchable
class Any
@Public
class DataStream[T](stream: DataStream[T])

Attributes

Supertypes
class Object
trait Matchable
class Any
Known subtypes
class KeyedStream[T, K]
@Experimental
class DataStreamUtils[T](val self: DataStream[T])(implicit evidence$1: TypeInformation[T], evidence$2: ClassTag[T])

This class provides simple utility methods for collecting a DataStream, effectively enriching it with the functionality encapsulated by DataStreamUtils.

This class provides simple utility methods for collecting a DataStream, effectively enriching it with the functionality encapsulated by DataStreamUtils.

This experimental class is relocated from flink-streaming-contrib.

Value parameters

self

DataStream

Attributes

Supertypes
class Object
trait Matchable
class Any
object IndylambdaScalaClosures

Attributes

Supertypes
class Object
trait Matchable
class Any
Self type
@Internal
class JavaKeySelector[IN, K](fun: IN => K) extends KeySelector[IN, K]

Attributes

Supertypes
trait KeySelector[IN, K]
trait Function
trait Serializable
class Object
trait Matchable
class Any
Show all
@Public
class JoinedStreams[T1, T2](input1: DataStream[T1], input2: DataStream[T2])

JoinedStreams represents two DataStreams that have been joined. A streaming join operation is evaluated over elements in a window.

JoinedStreams represents two DataStreams that have been joined. A streaming join operation is evaluated over elements in a window.

To finalize the join operation you also need to specify a KeySelector for both the first and second input and a WindowAssigner

Note: Right now, the groups are being built in memory so you need to ensure that they don't get too big. Otherwise the JVM might crash.

Example:

val one: DataStream[(String, Int)]  = ...
val two: DataStream[(String, Int)] = ...

val result = one.join(two)
   .where {t => ... }
   .equal {t => ... }
   .window(TumblingEventTimeWindows.of(Time.of(5, TimeUnit.SECONDS)))
   .apply(new MyJoinFunction())
}

Attributes

Supertypes
class Object
trait Matchable
class Any
@Public
class KeyedStream[T, K](javaStream: KeyedStream[T, K]) extends DataStream[T]

Attributes

Supertypes
class DataStream[T]
class Object
trait Matchable
class Any
@PublicEvolving
class OutputTag[T](id: String)(implicit evidence$1: TypeInformation[T]) extends OutputTag[T]

An OutputTag is a typed and named tag to use for tagging side outputs of an operator.

An OutputTag is a typed and named tag to use for tagging side outputs of an operator.

Example:

 val outputTag = OutputTag[String]("late-data")

Type parameters

T

the type of elements in the side-output stream.

Attributes

Companion
object
Supertypes
class OutputTag[T]
trait Serializable
class Object
trait Matchable
class Any
object OutputTag

Attributes

Companion
class
Supertypes
class Object
trait Matchable
class Any
Self type
OutputTag.type
class ReturnStatementInClosureException extends FlinkException

Attributes

Supertypes
class FlinkException
class Exception
class Throwable
trait Serializable
class Object
trait Matchable
class Any
Show all
object ScalaStreamOps

Attributes

Supertypes
class Object
trait Matchable
class Any
Self type
@Public
class StreamExecutionEnvironment(javaEnv: StreamExecutionEnvironment)

Attributes

Companion
object
Supertypes
class Object
trait Matchable
class Any
object StreamExecutionEnvironment

Attributes

Companion
class
Supertypes
class Object
trait Matchable
class Any
Self type
trait TaggedDerivation[TypeClass[_]] extends CommonTaggedDerivation[TypeClass]

Attributes

Supertypes
trait CommonTaggedDerivation[TypeClass]
class Object
trait Matchable
class Any
trait TypeTag[A]

Attributes

Companion
object
Supertypes
class Object
trait Matchable
class Any
object TypeTag

Attributes

Companion
trait
Supertypes
class Object
trait Matchable
class Any
Self type
TypeTag.type
object TypeTagMacro

Attributes

Supertypes
class Object
trait Matchable
class Any
Self type
@Public
class WindowedStream[T, K, W <: Window](javaStream: WindowedStream[T, K, W])

A WindowedStream represents a data stream where elements are grouped by key, and for each key, the stream of elements is split into windows based on a org.apache.flink.streaming.api.windowing.assigners.WindowAssigner. Window emission is triggered based on a Trigger.

A WindowedStream represents a data stream where elements are grouped by key, and for each key, the stream of elements is split into windows based on a org.apache.flink.streaming.api.windowing.assigners.WindowAssigner. Window emission is triggered based on a Trigger.

The windows are conceptually evaluated for each key individually, meaning windows can trigger at different points for each key.

If an org.apache.flink.streaming.api.windowing.evictors.Evictor is specified it will be used to evict elements from the window after evaluation was triggered by the Trigger but before the actual evaluation of the window. When using an evictor window performance will degrade significantly, since pre-aggregation of window results cannot be used.

Note that the WindowedStream is purely and API construct, during runtime the WindowedStream will be collapsed together with the KeyedStream and the operation over the window into one single operation.

Type parameters

K

The type of the key by which elements are grouped.

T

The type of elements in the stream.

W

The type of Window that the org.apache.flink.streaming.api.windowing.assigners.WindowAssigner assigns the elements to.

Attributes

Supertypes
class Object
trait Matchable
class Any
object serializers

Attributes

Supertypes
trait TaggedDerivation[[T <: <FromJavaObject>] =>> TypeInformation[T]]
trait CommonTaggedDerivation[[T <: <FromJavaObject>] =>> TypeInformation[T]]
class Object
trait Matchable
class Any
Self type
Generated with