io.debezium.pipeline.txmetadata

Class TransactionMonitor

java.lang.Object

io.debezium.pipeline.txmetadata.TransactionMonitor

@NotThreadSafe
public class TransactionMonitor
extends Object

The class has externalized its state in TransactionContext context class so it can be stored in and recovered from offsets. The class receives all processed events and keeps the transaction tracking depending on transaction id. Upon transaction change the metadata events are delivered to a dedicated topic informing about START/END of the transaction, including transaction id and in case of END event the amount of events generated by the transaction.

Every event seen has its source block enriched to contain

• transaction id
• the total event order in the transaction
• the order of event per table/collection source in the transaction

Author:

Jiri Pechanec

Field Summary

Fields

Modifier and Type	Field and Description
private CommonConnectorConfig	connectorConfig
static String	DEBEZIUM_TRANSACTION_COLLECTION_KEY
static String	DEBEZIUM_TRANSACTION_DATA_COLLECTION_ORDER_KEY
static String	DEBEZIUM_TRANSACTION_DATA_COLLECTIONS_KEY
static String	DEBEZIUM_TRANSACTION_EVENT_COUNT_KEY
static String	DEBEZIUM_TRANSACTION_ID_KEY
static String	DEBEZIUM_TRANSACTION_KEY
static String	DEBEZIUM_TRANSACTION_STATUS_KEY
static String	DEBEZIUM_TRANSACTION_TOTAL_ORDER_KEY
private static org.apache.kafka.connect.data.Schema	EVENT_COUNT_PER_DATA_COLLECTION_SCHEMA
private EventMetadataProvider	eventMetadataProvider
private static org.slf4j.Logger	LOGGER
private static SchemaNameAdjuster	schemaNameAdjuster
private BlockingConsumer<org.apache.kafka.connect.source.SourceRecord>	sender
private static String	TOPIC_SUFFIX
private String	topicName
static org.apache.kafka.connect.data.Schema	TRANSACTION_BLOCK_SCHEMA
private static org.apache.kafka.connect.data.Schema	TRANSACTION_KEY_SCHEMA
private static org.apache.kafka.connect.data.Schema	TRANSACTION_VALUE_SCHEMA

Constructor Summary

Constructors

Constructor and Description
TransactionMonitor(CommonConnectorConfig connectorConfig, EventMetadataProvider eventMetadataProvider, BlockingConsumer<org.apache.kafka.connect.source.SourceRecord> sender)

Method Summary

Modifier and Type	Method and Description
private void	beginTransaction(OffsetContext offsetContext)
void	dataEvent(DataCollectionId source, OffsetContext offset, Object key, org.apache.kafka.connect.data.Struct value)
private void	endTransaction(OffsetContext offsetContext)
void	transactionComittedEvent(OffsetContext offset)
private void	transactionEvent(OffsetContext offsetContext, DataCollectionId source, org.apache.kafka.connect.data.Struct value)
void	transactionStartedEvent(String transactionId, OffsetContext offset)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOGGER

private static final org.slf4j.Logger LOGGER

schemaNameAdjuster

private static final SchemaNameAdjuster schemaNameAdjuster

TOPIC_SUFFIX

private static final String TOPIC_SUFFIX

See Also:

Constant Field Values

DEBEZIUM_TRANSACTION_KEY

public static final String DEBEZIUM_TRANSACTION_KEY

See Also:

Constant Field Values

DEBEZIUM_TRANSACTION_ID_KEY

public static final String DEBEZIUM_TRANSACTION_ID_KEY

See Also:

Constant Field Values

DEBEZIUM_TRANSACTION_TOTAL_ORDER_KEY

public static final String DEBEZIUM_TRANSACTION_TOTAL_ORDER_KEY

See Also:

Constant Field Values

DEBEZIUM_TRANSACTION_DATA_COLLECTION_ORDER_KEY

public static final String DEBEZIUM_TRANSACTION_DATA_COLLECTION_ORDER_KEY

See Also:

Constant Field Values

DEBEZIUM_TRANSACTION_STATUS_KEY

public static final String DEBEZIUM_TRANSACTION_STATUS_KEY

See Also:

Constant Field Values

DEBEZIUM_TRANSACTION_EVENT_COUNT_KEY

public static final String DEBEZIUM_TRANSACTION_EVENT_COUNT_KEY

See Also:

Constant Field Values

DEBEZIUM_TRANSACTION_COLLECTION_KEY

public static final String DEBEZIUM_TRANSACTION_COLLECTION_KEY

See Also:

Constant Field Values

DEBEZIUM_TRANSACTION_DATA_COLLECTIONS_KEY

public static final String DEBEZIUM_TRANSACTION_DATA_COLLECTIONS_KEY

See Also:

Constant Field Values

TRANSACTION_BLOCK_SCHEMA

public static final org.apache.kafka.connect.data.Schema TRANSACTION_BLOCK_SCHEMA

TRANSACTION_KEY_SCHEMA

private static final org.apache.kafka.connect.data.Schema TRANSACTION_KEY_SCHEMA

EVENT_COUNT_PER_DATA_COLLECTION_SCHEMA

private static final org.apache.kafka.connect.data.Schema EVENT_COUNT_PER_DATA_COLLECTION_SCHEMA

TRANSACTION_VALUE_SCHEMA

private static final org.apache.kafka.connect.data.Schema TRANSACTION_VALUE_SCHEMA

eventMetadataProvider

private final EventMetadataProvider eventMetadataProvider

topicName

private final String topicName

sender

private final BlockingConsumer<org.apache.kafka.connect.source.SourceRecord> sender

connectorConfig

private final CommonConnectorConfig connectorConfig

Constructor Detail

TransactionMonitor

public TransactionMonitor(CommonConnectorConfig connectorConfig,
                          EventMetadataProvider eventMetadataProvider,
                          BlockingConsumer<org.apache.kafka.connect.source.SourceRecord> sender)

Method Detail

dataEvent

public void dataEvent(DataCollectionId source,
                      OffsetContext offset,
                      Object key,
                      org.apache.kafka.connect.data.Struct value)
               throws InterruptedException

Throws:

InterruptedException

transactionComittedEvent

public void transactionComittedEvent(OffsetContext offset)
                              throws InterruptedException

Throws:

InterruptedException

transactionStartedEvent

public void transactionStartedEvent(String transactionId,
                                    OffsetContext offset)
                             throws InterruptedException

Throws:

InterruptedException

transactionEvent

private void transactionEvent(OffsetContext offsetContext,
                              DataCollectionId source,
                              org.apache.kafka.connect.data.Struct value)

beginTransaction

private void beginTransaction(OffsetContext offsetContext)
                       throws InterruptedException

Throws:

InterruptedException

endTransaction

private void endTransaction(OffsetContext offsetContext)
                     throws InterruptedException

Throws:

InterruptedException