Package io.debezium.connector.oracle.logminer.processor.infinispan

Class InfinispanLogMinerEventProcessor

java.lang.Object

io.debezium.connector.oracle.logminer.processor.AbstractLogMinerEventProcessor

io.debezium.connector.oracle.logminer.processor.infinispan.InfinispanLogMinerEventProcessor

All Implemented Interfaces:

LogMinerEventProcessor, AutoCloseable

public class InfinispanLogMinerEventProcessor
extends AbstractLogMinerEventProcessor

An implementation of LogMinerEventProcessor that uses Infinispan to persist the transaction cache across restarts on disk.

Author:

Chris Cranford

Nested Class Summary

Nested classes/interfaces inherited from class io.debezium.connector.oracle.logminer.processor.AbstractLogMinerEventProcessor

AbstractLogMinerEventProcessor.Counters

Field Summary

Fields

Modifier and Type	Field	Description
private ChangeEventSource.ChangeEventSourceContext	context
private Scn	currentOffsetCommitScn
private Scn	currentOffsetScn
private EventDispatcher<TableId>	dispatcher
private OracleConnection	jdbcConnection
private Scn	lastCommittedScn
private static org.slf4j.Logger	LOGGER
private Scn	maxCommittedScn
private OracleStreamingChangeEventSourceMetrics	metrics
private OracleOffsetContext	offsetContext
private OraclePartition	partition
private org.infinispan.Cache<String,String>	recentlyCommittedTransactionsCache	A cache storing recently committed transactions key by the unique transaction id and the event's system change number.
private org.infinispan.Cache<String,String>	rollbackTransactionsCache	A cache storing recently rolled back transactions keyed by the unique transaction id and the event's system change number.
private org.infinispan.Cache<String,String>	schemaChangesCache	A cache storing recently emitted schema changes keyed by the system change number of the schema change and the associated fully qualified TableId identifier value for the change.
private InfinispanTransactionCache	transactionCache	A cache that stores the complete Transaction object keyed by the unique transaction id.

Fields inherited from class io.debezium.connector.oracle.logminer.processor.AbstractLogMinerEventProcessor

counters

Constructor Summary

Constructors

Constructor	Description
InfinispanLogMinerEventProcessor(ChangeEventSource.ChangeEventSourceContext context, OracleConnectorConfig connectorConfig, OracleConnection jdbcConnection, EventDispatcher<TableId> dispatcher, OraclePartition partition, OracleOffsetContext offsetContext, OracleDatabaseSchema schema, OracleStreamingChangeEventSourceMetrics metrics)

Method Summary

Modifier and Type	Method	Description
void	abandonTransactions(Duration retention)	A callback for the event processor to abandon long running transactions.
protected void	addToTransaction(String transactionId, LogMinerEventRow row, Supplier<LogMinerEvent> eventSupplier)	Add a transaction to the transaction map if allowed.
private Scn	calculateNewStartScn(Scn endScn)
void	close()
private <K,V> org.infinispan.Cache<K,V>	createCache(org.infinispan.manager.EmbeddedCacheManager manager, OracleConnectorConfig connectorConfig, String name)
private PreparedStatement	createQueryStatement()
protected TransactionCache<?>	getTransactionCache()	Returns the TransactionCache implementation.
protected void	handleCommit(LogMinerEventRow row)	Handle processing a LogMinerEventRow for a COMMIT event.
protected void	handleRollback(LogMinerEventRow row)	Handle processing a LogMinerEventRow for a ROLLBACK event.
protected void	handleSchemaChange(LogMinerEventRow row)	Handle processing a LogMinerEventRow for a DDL event.
protected boolean	hasSchemaChangeBeenSeen(LogMinerEventRow row)	Checks whether the LogMinerEvent row for a schema change can be emitted.
protected boolean	isTransactionIdAllowed(String transactionId)	Return whether a give transaction can be added to the processor's buffer.
Scn	process(Scn startScn, Scn endScn)	Process Oracle LogMiner events for a given system change number range.
protected void	processRow(LogMinerEventRow row)	Processes a single LogMinerEventRow.

Methods inherited from class io.debezium.connector.oracle.logminer.processor.AbstractLogMinerEventProcessor

getConfig, getLastProcessedScn, getReconciliation, getSchema, handleDataEvent, handleLobWrite, handleMissingScn, handleSelectLobLocator, handleStart, isRecentlyCommitted, isTrxIdRawValue, processResults, warnPotentiallyStuckScn

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOGGER

private static final org.slf4j.Logger LOGGER

jdbcConnection

private final OracleConnection jdbcConnection

metrics

private final OracleStreamingChangeEventSourceMetrics metrics

partition

private final OraclePartition partition

offsetContext

private final OracleOffsetContext offsetContext

dispatcher

private final EventDispatcher<TableId> dispatcher

context

private final ChangeEventSource.ChangeEventSourceContext context

transactionCache

private final InfinispanTransactionCache transactionCache

A cache that stores the complete Transaction object keyed by the unique transaction id.

recentlyCommittedTransactionsCache

private final org.infinispan.Cache<String,String> recentlyCommittedTransactionsCache

A cache storing recently committed transactions key by the unique transaction id and the event's system change number. This cache is used to filter events during re-mining when LOB support is enabled to skip the processing of already emitted transactions. Entries in this cache are removed when the offset low watermark (scn) advances beyond the system change number associated with the transaction.

rollbackTransactionsCache

private final org.infinispan.Cache<String,String> rollbackTransactionsCache

A cache storing recently rolled back transactions keyed by the unique transaction id and the event's system change number. This cache is used to filter events during re-mining when LOB support is enabled to skip the processing of already discarded transactions. Entries in this cache are removed when the offset low watermark (scn) advances beyond the system change number associated with the transaction.

schemaChangesCache

private final org.infinispan.Cache<String,String> schemaChangesCache

A cache storing recently emitted schema changes keyed by the system change number of the schema change and the associated fully qualified TableId identifier value for the change. This cache is used to filter events during re-mining when LOB support is enabled to skip the processing of already emitted schema changes. Entries in this cache are removed when the offset low watermark (scn) advances beyond the system change number associated with the schema change event.

currentOffsetScn

private Scn currentOffsetScn

currentOffsetCommitScn

private Scn currentOffsetCommitScn

lastCommittedScn

private Scn lastCommittedScn

maxCommittedScn

private Scn maxCommittedScn

Constructor Detail

InfinispanLogMinerEventProcessor

public InfinispanLogMinerEventProcessor(ChangeEventSource.ChangeEventSourceContext context,
                                        OracleConnectorConfig connectorConfig,
                                        OracleConnection jdbcConnection,
                                        EventDispatcher<TableId> dispatcher,
                                        OraclePartition partition,
                                        OracleOffsetContext offsetContext,
                                        OracleDatabaseSchema schema,
                                        OracleStreamingChangeEventSourceMetrics metrics)

Method Detail

createCache

private <K,V> org.infinispan.Cache<K,V> createCache(org.infinispan.manager.EmbeddedCacheManager manager,
                                                                OracleConnectorConfig connectorConfig,
                                                                String name)

getTransactionCache

protected TransactionCache<?> getTransactionCache()

Description copied from class: AbstractLogMinerEventProcessor

Returns the TransactionCache implementation.

Specified by:

getTransactionCache in class AbstractLogMinerEventProcessor

Returns:

the transaction cache, never null

close

public void close()
           throws Exception

Throws:

Exception

process

public Scn process(Scn startScn,
                   Scn endScn)
            throws SQLException,
                   InterruptedException

Description copied from interface: LogMinerEventProcessor

Process Oracle LogMiner events for a given system change number range.

Parameters:

startScn - the starting system change number, must not be null

endScn - the ending system change number, must not be null

Returns:

the next iteration's starting system change number, never null

Throws:

SQLException

InterruptedException

processRow

protected void processRow(LogMinerEventRow row)
                   throws SQLException,
                          InterruptedException

Description copied from class: AbstractLogMinerEventProcessor

Processes a single LogMinerEventRow.

Overrides:

processRow in class AbstractLogMinerEventProcessor

Parameters:

row - the event row, must not be null

Throws:

SQLException - if a database exception occurred

InterruptedException - if the dispatcher was interrupted sending an event

abandonTransactions

public void abandonTransactions(Duration retention)

Description copied from interface: LogMinerEventProcessor

A callback for the event processor to abandon long running transactions.

Parameters:

retention - the maximum duration in which long running transactions are allowed.

isTransactionIdAllowed

protected boolean isTransactionIdAllowed(String transactionId)

Description copied from class: AbstractLogMinerEventProcessor

Return whether a give transaction can be added to the processor's buffer. The default implementation is to allow all transaction ids.

Overrides:

isTransactionIdAllowed in class AbstractLogMinerEventProcessor

Parameters:

transactionId - the unique transaction id

Returns:

whether a transaction id can be added to the processor's buffer

hasSchemaChangeBeenSeen

protected boolean hasSchemaChangeBeenSeen(LogMinerEventRow row)

Description copied from class: AbstractLogMinerEventProcessor

Checks whether the LogMinerEvent row for a schema change can be emitted.

Overrides:

hasSchemaChangeBeenSeen in class AbstractLogMinerEventProcessor

Parameters:

row - the result set row

Returns:

true if the schema change has been seen, false otherwise.

handleCommit

protected void handleCommit(LogMinerEventRow row)
                     throws InterruptedException

Description copied from class: AbstractLogMinerEventProcessor

Handle processing a LogMinerEventRow for a COMMIT event.

Specified by:

handleCommit in class AbstractLogMinerEventProcessor

Parameters:

row - the result set row

Throws:

InterruptedException - if the event dispatcher was interrupted sending events

handleRollback

protected void handleRollback(LogMinerEventRow row)

Description copied from class: AbstractLogMinerEventProcessor

Handle processing a LogMinerEventRow for a ROLLBACK event. The default implementation is a no-op.

Overrides:

handleRollback in class AbstractLogMinerEventProcessor

Parameters:

row - the result set row

handleSchemaChange

protected void handleSchemaChange(LogMinerEventRow row)
                           throws InterruptedException

Description copied from class: AbstractLogMinerEventProcessor

Handle processing a LogMinerEventRow for a DDL event.

Overrides:

handleSchemaChange in class AbstractLogMinerEventProcessor

Parameters:

row - the result set row

Throws:

InterruptedException - if the event dispatcher is interrupted sending the event

addToTransaction

protected void addToTransaction(String transactionId,
                                LogMinerEventRow row,
                                Supplier<LogMinerEvent> eventSupplier)

Description copied from class: AbstractLogMinerEventProcessor

Add a transaction to the transaction map if allowed.

Overrides:

addToTransaction in class AbstractLogMinerEventProcessor

Parameters:

transactionId - the unqiue transaction id

row - the LogMiner event row

eventSupplier - the supplier of the event to create if the event is allowed to be added

createQueryStatement

private PreparedStatement createQueryStatement()
                                        throws SQLException

Throws:

SQLException

calculateNewStartScn

private Scn calculateNewStartScn(Scn endScn)
                          throws InterruptedException

Throws:

InterruptedException