io.cdap.delta.api

Interface EventConsumer

public interface EventConsumer

Consumes events emitted by a source and applies them to the target system.

Method Summary

Modifier and Type	Method and Description
void	applyDDL(Sequenced<DDLEvent> event) Apply a DDL event, such as creating a table.
void	applyDML(Sequenced<DMLEvent> event) Apply a DML event.
void	start() Start the event consumer.
default void	stop() Stop the event consumer.

Method Detail

start

void start()

Start the event consumer. This will be called before any other method is called.

stop

default void stop()
           throws InterruptedException

Stop the event consumer.

Throws:

InterruptedException - if the consumer was interrupted while stopping

applyDDL

void applyDDL(Sequenced<DDLEvent> event)
       throws Exception

Apply a DDL event, such as creating a table. This method must be idempotent. For example, if the event is a table creation and the table already exists, this method should not fail due to an attempt to create a table that already exists. Idempotency is required because the event can be applied multiple times in failure scenarios. During normal operation, an event will be applied exactly once. In failure scenarios the event will be applied at least once. If this method throws a DeltaFailureException, the pipeline will fail immediately. If any other type of Exception is thrown, pipeline offset and sequence number will be reset to the last commit, and events will be re-applied. If exceptions are consecutively thrown for longer than the configured pipeline timeout, retries will be abandoned and the pipeline will be fail.

Parameters:

event - ddl event to apply

Throws:

DeltaFailureException - if there was an error and the pipeline should immediately be failed

Exception - if there was an error applying the change, but it might succeed in the future

applyDML

void applyDML(Sequenced<DMLEvent> event)
       throws Exception

Apply a DML event. This method must be idempotent. For example, if there is an insert and the row already exists, this method should not fail due to an attempt to insert a row that already exists, and it should not write duplicate data. Idempotency is required because the batch of events can be applied multiple times in failure scenarios. During normal operation, each batch will be applied exactly once. In failure scenarios the batch will be applied at least once. If this method throws a DeltaFailureException, the pipeline will fail immediately. If any other type of Exception is thrown, pipeline offset and sequence number will be reset to the last commit, and events will be re-applied. If exceptions are consecutively thrown for longer than the configured pipeline timeout, retries will be abandoned and the pipeline will be fail.

Parameters:

event - DML event to apply

Throws:

DeltaFailureException - if there was an error and the pipeline should immediately be failed

Exception - if there was an error applying the change, but it might succeed in the future