org.apache.pulsar.io.core

Class PushSource<T>

java.lang.Object

org.apache.pulsar.io.core.PushSource<T>

All Implemented Interfaces:

AutoCloseable, Source<T>

@InterfaceAudience.Public
 @InterfaceStability.Stable
public abstract class PushSource<T>
extends Object
implements Source<T>

Pulsar's Push Source interface. PushSource read data from external sources (database changes, twitter firehose, etc) and publish to a Pulsar topic. The reason its called Push is because PushSources get passed a consumer that they invoke whenever they have data to be published to Pulsar. The lifecycle of a PushSource is to open it passing any config needed by it to initialize(like open network connection, authenticate, etc). A consumer is then to it which is invoked by the source whenever there is data to be published. Once all data has been read, one can use close at the end of the session to do any cleanup

Constructor Summary

Constructors

Constructor and Description
PushSource()

Method Summary

Modifier and Type	Method and Description
void	consume(Record<T> record) Attach a consumer function to this Source.
int	getQueueLength() Get length of the queue that records are push onto Users can override this method to customize the queue length
abstract void	open(Map<String,Object> config, SourceContext sourceContext) Open connector with configuration.
Record<T>	read() Reads the next message from source.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.lang.AutoCloseable

close

Constructor Detail

PushSource

public PushSource()

Method Detail

read

public Record<T> read()
               throws Exception

Description copied from interface: Source

Reads the next message from source. If source does not have any new messages, this call should block.

Specified by:

read in interface Source<T>

Returns:

next message from source. The return result should never be null

Throws:

Exception

open

public abstract void open(Map<String,Object> config,
                          SourceContext sourceContext)
                   throws Exception

Open connector with configuration.

Specified by:

open in interface Source<T>

Parameters:

config - initialization config

sourceContext - environment where the source connector is running

Throws:

Exception - IO type exceptions when opening a connector

consume

public void consume(Record<T> record)

Attach a consumer function to this Source. This is invoked by the implementation to pass messages whenever there is data to be pushed to Pulsar.

Parameters:

record - next message from source which should be sent to a Pulsar topic

getQueueLength

public int getQueueLength()

Get length of the queue that records are push onto Users can override this method to customize the queue length

Returns:

queue length