io.cdap.delta.api

Interface DeltaTargetContext

All Superinterfaces:

DeltaRuntimeContext, io.cdap.cdap.api.plugin.PluginContext

public interface DeltaTargetContext
extends DeltaRuntimeContext

Context for a CDC Target.

Method Summary

Modifier and Type	Method and Description
void	commitOffset(Offset offset, long sequenceNumber) Commit changes up to the given offset.
void	dropTableState(String database, String table) Record that the table was dropped and its state can be removed
SourceProperties	getSourceProperties()
void	incrementCount(DDLOperation op) Record that a DDL operation was processed.
void	incrementCount(DMLOperation op) Record that a DML operation was processed.
default void	initializeSequenceNumber(long sequenceNumber) Initialize the sequence number generated by the replicator pipeline.
void	setTableError(String database, String table, ReplicationError error) Record that there are currently errors applying events to a specified table.
void	setTableError(String database, String schema, String table, ReplicationError error) Record that there are currently errors applying events to a specified table.
void	setTableReplicating(String database, String table) Record that a table is being replicated.
void	setTableSnapshotting(String database, String table) Record that the initial snapshot for the table is being applied

Methods inherited from interface io.cdap.delta.api.DeltaRuntimeContext

getAllTables, getApplicationName, getInstanceCount, getInstanceId, getMaxRetrySeconds, getMetrics, getPipelineId, getRunId, getRuntimeArguments, getState, putState

Methods inherited from interface io.cdap.cdap.api.plugin.PluginContext

getPluginProperties, getPluginProperties, loadPluginClass, newPluginInstance, newPluginInstance

Method Detail

incrementCount

void incrementCount(DMLOperation op)

Record that a DML operation was processed. Metrics on the number of operations processed are stored in memory and actually written out when commitOffset(Offset, long) is called.

Parameters:

op - a DML operation

incrementCount

void incrementCount(DDLOperation op)

Record that a DDL operation was processed. Metrics on the number of operations processed are stored in memory and actually written out when commitOffset(Offset, long) is called.

Parameters:

op - a DDL operation

commitOffset

void commitOffset(Offset offset,
                  long sequenceNumber)
           throws IOException

Commit changes up to the given offset. Once an offset is successfully committed, events up to that offset are considered complete. When the program starts up, events from the last committed offset will be read. If the program died before committing an offset, events may be replayed. The sequence number given here must match the sequence number of the event with the given offset in order to ensure that events with a particular offset always have the same corresponding sequence number. Also writes out metrics on the number of DML and DDL operations that were applied since the last successful commit, as recorded by calls to incrementCount(DDLOperation) and incrementCount(DMLOperation).

Parameters:

offset - offset to commitOffset

sequenceNumber - the sequence number for the offset

Throws:

IOException - if there was an error persisting the new offset

setTableError

void setTableError(String database,
                   String table,
                   ReplicationError error)
            throws IOException

Record that there are currently errors applying events to a specified table.

Parameters:

database - database that the table is in

table - table that is having errors

error - information about the error

Throws:

IOException - if there was an error persisting the change in state. This can only be thrown if there was a change in state.

setTableError

void setTableError(String database,
                   @Nullable
                   String schema,
                   String table,
                   ReplicationError error)
            throws IOException

Record that there are currently errors applying events to a specified table.

Parameters:

database - database that the table is in

schema - schema in which table is present. Can be null for the databases which don't support schema

table - table that is having errors

error - information about the error

Throws:

IOException - if there was an error persisting the change in state. This can only be thrown if there was a change in state.

setTableReplicating

void setTableReplicating(String database,
                         String table)
                  throws IOException

Record that a table is being replicated. This should be called after the initial table snapshot has been applied, and if there was a recovery from an error.

Parameters:

database - database that the table is in

table - table name

Throws:

IOException - if there was an error persisting the change in state. This can only be thrown if there was a change in state.

setTableSnapshotting

void setTableSnapshotting(String database,
                          String table)
                   throws IOException

Record that the initial snapshot for the table is being applied

Parameters:

database - database that the table is in

table - table name

Throws:

IOException - if there was an error persisting the change in state. This can only be thrown if there was a change in state.

dropTableState

void dropTableState(String database,
                    String table)
             throws IOException

Record that the table was dropped and its state can be removed

Parameters:

database - database that the table is in

table - table name

Throws:

IOException - if there was an error persisting the change in state. This can only be thrown if there was a change in state.

getSourceProperties

@Nullable
SourceProperties getSourceProperties()

Returns:

the properties about source in the replicator pipeline which will be useful to targets. If no such properties are provided by source null is returned

initializeSequenceNumber

default void initializeSequenceNumber(long sequenceNumber)

Initialize the sequence number generated by the replicator pipeline.

Parameters:

sequenceNumber - the sequence number to be set for the first record seen by replication pipeline