Package io.debezium.pipeline.txmetadata

Class TransactionMonitor

java.lang.Object

io.debezium.pipeline.txmetadata.TransactionMonitor

@NotThreadSafe
public class TransactionMonitor
extends Object

The class has externalized its state in TransactionContext context class so it can be stored in and recovered from offsets. The class receives all processed events and keeps the transaction tracking depending on transaction id. Upon transaction change the metadata events are delivered to a dedicated topic informing about START/END of the transaction, including transaction id and in case of END event the amount of events generated by the transaction.

Every event seen has its source block enriched to contain

• transaction id
• the total event order in the transaction
• the order of event per table/collection source in the transaction

Author:

Jiri Pechanec

Field Summary

Fields

Modifier and Type	Field	Description
private final CommonConnectorConfig	connectorConfig
static final String	DEBEZIUM_TRANSACTION_COLLECTION_KEY
static final String	DEBEZIUM_TRANSACTION_DATA_COLLECTION_ORDER_KEY
static final String	DEBEZIUM_TRANSACTION_DATA_COLLECTIONS_KEY
static final String	DEBEZIUM_TRANSACTION_EVENT_COUNT_KEY
static final String	DEBEZIUM_TRANSACTION_ID_KEY
static final String	DEBEZIUM_TRANSACTION_KEY
static final String	DEBEZIUM_TRANSACTION_STATUS_KEY
static final String	DEBEZIUM_TRANSACTION_TOTAL_ORDER_KEY
static final String	DEBEZIUM_TRANSACTION_TS_MS
private static final org.apache.kafka.connect.data.Schema	EVENT_COUNT_PER_DATA_COLLECTION_SCHEMA
private final EventMetadataProvider	eventMetadataProvider
private static final org.slf4j.Logger	LOGGER
private final BlockingConsumer<org.apache.kafka.connect.source.SourceRecord>	sender
private final String	topicName
static final org.apache.kafka.connect.data.Schema	TRANSACTION_BLOCK_SCHEMA
protected final org.apache.kafka.connect.data.Schema	transactionKeySchema
protected final org.apache.kafka.connect.data.Schema	transactionValueSchema

Constructor Summary

Constructors

Constructor	Description
TransactionMonitor(CommonConnectorConfig connectorConfig, EventMetadataProvider eventMetadataProvider, SchemaNameAdjuster schemaNameAdjuster, BlockingConsumer<org.apache.kafka.connect.source.SourceRecord> sender, String topicName)

Method Summary

Modifier and Type	Method	Description
private void	beginTransaction(Partition partition, OffsetContext offsetContext, Instant timestamp)
void	dataEvent(Partition partition, DataCollectionId source, OffsetContext offset, Object key, org.apache.kafka.connect.data.Struct value)
private void	endTransaction(Partition partition, OffsetContext offsetContext, Instant timestamp)
protected org.apache.kafka.connect.data.Struct	prepareTxBeginValue(OffsetContext offsetContext, Instant timestamp)
protected org.apache.kafka.connect.data.Struct	prepareTxEndValue(OffsetContext offsetContext, Instant timestamp)
protected org.apache.kafka.connect.data.Struct	prepareTxKey(OffsetContext offsetContext)
protected org.apache.kafka.connect.data.Struct	prepareTxStruct(OffsetContext offsetContext, long dataCollectionEventOrder, org.apache.kafka.connect.data.Struct value)
void	transactionComittedEvent(Partition partition, OffsetContext offset, Instant timestamp)
private void	transactionEvent(OffsetContext offsetContext, DataCollectionId source, org.apache.kafka.connect.data.Struct value)
void	transactionStartedEvent(Partition partition, String transactionId, OffsetContext offset, Instant timestamp)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOGGER

private static final org.slf4j.Logger LOGGER

DEBEZIUM_TRANSACTION_KEY

public static final String DEBEZIUM_TRANSACTION_KEY

See Also:

• Constant Field Values

DEBEZIUM_TRANSACTION_ID_KEY

public static final String DEBEZIUM_TRANSACTION_ID_KEY

See Also:

• Constant Field Values

DEBEZIUM_TRANSACTION_TOTAL_ORDER_KEY

public static final String DEBEZIUM_TRANSACTION_TOTAL_ORDER_KEY

See Also:

• Constant Field Values

DEBEZIUM_TRANSACTION_DATA_COLLECTION_ORDER_KEY

public static final String DEBEZIUM_TRANSACTION_DATA_COLLECTION_ORDER_KEY

See Also:

• Constant Field Values

DEBEZIUM_TRANSACTION_STATUS_KEY

public static final String DEBEZIUM_TRANSACTION_STATUS_KEY

See Also:

• Constant Field Values

DEBEZIUM_TRANSACTION_EVENT_COUNT_KEY

public static final String DEBEZIUM_TRANSACTION_EVENT_COUNT_KEY

See Also:

• Constant Field Values

DEBEZIUM_TRANSACTION_COLLECTION_KEY

public static final String DEBEZIUM_TRANSACTION_COLLECTION_KEY

See Also:

• Constant Field Values

DEBEZIUM_TRANSACTION_DATA_COLLECTIONS_KEY

public static final String DEBEZIUM_TRANSACTION_DATA_COLLECTIONS_KEY

See Also:

• Constant Field Values

DEBEZIUM_TRANSACTION_TS_MS

public static final String DEBEZIUM_TRANSACTION_TS_MS

See Also:

• Constant Field Values

TRANSACTION_BLOCK_SCHEMA

public static final org.apache.kafka.connect.data.Schema TRANSACTION_BLOCK_SCHEMA

EVENT_COUNT_PER_DATA_COLLECTION_SCHEMA

private static final org.apache.kafka.connect.data.Schema EVENT_COUNT_PER_DATA_COLLECTION_SCHEMA

transactionKeySchema

protected final org.apache.kafka.connect.data.Schema transactionKeySchema

transactionValueSchema

protected final org.apache.kafka.connect.data.Schema transactionValueSchema

eventMetadataProvider

private final EventMetadataProvider eventMetadataProvider

topicName

private final String topicName

sender

private final BlockingConsumer<org.apache.kafka.connect.source.SourceRecord> sender

connectorConfig

private final CommonConnectorConfig connectorConfig

Constructor Details

TransactionMonitor

public TransactionMonitor(CommonConnectorConfig connectorConfig,
 EventMetadataProvider eventMetadataProvider,
 SchemaNameAdjuster schemaNameAdjuster,
 BlockingConsumer<org.apache.kafka.connect.source.SourceRecord> sender,
 String topicName)

Method Details

dataEvent

public void dataEvent(Partition partition,
 DataCollectionId source,
 OffsetContext offset,
 Object key,
 org.apache.kafka.connect.data.Struct value)
               throws InterruptedException

Throws:

InterruptedException

transactionComittedEvent

public void transactionComittedEvent(Partition partition,
 OffsetContext offset,
 Instant timestamp)
                              throws InterruptedException

Throws:

InterruptedException

transactionStartedEvent

public void transactionStartedEvent(Partition partition,
 String transactionId,
 OffsetContext offset,
 Instant timestamp)
                             throws InterruptedException

Throws:

InterruptedException

prepareTxKey

protected org.apache.kafka.connect.data.Struct prepareTxKey(OffsetContext offsetContext)

prepareTxBeginValue

protected org.apache.kafka.connect.data.Struct prepareTxBeginValue(OffsetContext offsetContext,
 Instant timestamp)

prepareTxEndValue

protected org.apache.kafka.connect.data.Struct prepareTxEndValue(OffsetContext offsetContext,
 Instant timestamp)

prepareTxStruct

protected org.apache.kafka.connect.data.Struct prepareTxStruct(OffsetContext offsetContext,
 long dataCollectionEventOrder,
 org.apache.kafka.connect.data.Struct value)

transactionEvent

private void transactionEvent(OffsetContext offsetContext,
 DataCollectionId source,
 org.apache.kafka.connect.data.Struct value)

beginTransaction

private void beginTransaction(Partition partition,
 OffsetContext offsetContext,
 Instant timestamp)
                       throws InterruptedException

Throws:

InterruptedException

endTransaction

private void endTransaction(Partition partition,
 OffsetContext offsetContext,
 Instant timestamp)
                     throws InterruptedException

Throws:

InterruptedException