Package io.debezium.connector.oracle.logminer.processor.memory

Class MemoryLogMinerEventProcessor

java.lang.Object

io.debezium.connector.oracle.logminer.processor.AbstractLogMinerEventProcessor<MemoryTransaction>

io.debezium.connector.oracle.logminer.processor.memory.MemoryLogMinerEventProcessor

All Implemented Interfaces:

LogMinerEventProcessor, AutoCloseable

public class MemoryLogMinerEventProcessor
extends AbstractLogMinerEventProcessor<MemoryTransaction>

A LogMinerEventProcessor that uses the JVM heap to store events as they're being processed and emitted from Oracle LogMiner.

Author:

Chris Cranford

Nested Class Summary

Nested classes/interfaces inherited from class io.debezium.connector.oracle.logminer.processor.AbstractLogMinerEventProcessor

AbstractLogMinerEventProcessor.Counters

Field Summary

Fields

Modifier and Type	Field	Description
private final EventDispatcher<OraclePartition,TableId>	dispatcher
private static final org.slf4j.Logger	LOGGER
private final LogMinerStreamingChangeEventSourceMetrics	metrics
private final OracleOffsetContext	offsetContext
private final OraclePartition	partition
private final Map<String,Scn>	recentlyProcessedTransactionsCache	Cache of processed transactions (committed or rolled back), keyed based on the transaction's unique identifier.
private final Set<Scn>	schemaChangesCache
private final Map<String,MemoryTransaction>	transactionCache	Cache of transactions, keyed based on the transaction's unique identifier

Fields inherited from class io.debezium.connector.oracle.logminer.processor.AbstractLogMinerEventProcessor

counters, jdbcConnection, sqlQuery

Constructor Summary

Constructors

Constructor	Description
MemoryLogMinerEventProcessor(ChangeEventSource.ChangeEventSourceContext context, OracleConnectorConfig connectorConfig, OracleConnection jdbcConnection, EventDispatcher<OraclePartition,TableId> dispatcher, OraclePartition partition, OracleOffsetContext offsetContext, OracleDatabaseSchema schema, LogMinerStreamingChangeEventSourceMetrics metrics)

Method Summary

Modifier and Type	Method	Description
protected void	addToTransaction(String transactionId, LogMinerEventRow row, Supplier<LogMinerEvent> eventSupplier)	Add a transaction to the transaction map if allowed.
protected Scn	calculateNewStartScn(Scn endScn, Scn maxCommittedScn)	Calculates the new starting system change number based on the current processing range.
void	close()
protected PreparedStatement	createQueryStatement()	Create the JDBC query that will be used to fetch the mining result set.
protected MemoryTransaction	createTransaction(LogMinerEventRow row)	Creates a new transaction based on the supplied START event.
protected void	finalizeTransactionCommit(String transactionId, Scn commitScn)	Finalizes the commit of a transaction.
protected void	finalizeTransactionRollback(String transactionId, Scn rollbackScn)	Finalizes the rollback the specified transaction
protected MemoryTransaction	getAndRemoveTransactionFromCache(String transactionId)	Gets a transaction instance from the transaction cache while also removing its cache entry.
protected String	getFirstActiveTransactionKey()	Returns only the first transaction id in the transaction buffer.
protected Optional<MemoryTransaction>	getOldestTransactionInCache()	Get the oldest transaction in the cache.
protected Map<String,MemoryTransaction>	getTransactionCache()	Returns the TransactionCache implementation.
protected Scn	getTransactionCacheMinimumScn()	Gets the minimum system change number stored in the transaction cache.
protected int	getTransactionEventCount(MemoryTransaction transaction)	Returns the number of events associated with the specified transaction.
protected Iterator<LogMinerEvent>	getTransactionEventIterator(MemoryTransaction transaction)	Get an iterator over the events that are part of the specified transaction.
protected void	handleSchemaChange(LogMinerEventRow row)	Handle processing a LogMinerEventRow for a DDL event.
protected boolean	hasSchemaChangeBeenSeen(LogMinerEventRow row)	Checks whether the LogMinerEvent row for a schema change can be emitted.
protected boolean	isRecentlyProcessed(String transactionId)	Check whether a transaction has been recently processed through either a commit or rollback.
protected void	removeEventWithRowId(LogMinerEventRow row)	Removes a specific transaction event by database row identifier.

Methods inherited from class io.debezium.connector.oracle.logminer.processor.AbstractLogMinerEventProcessor

abandonTransactionOverEventThreshold, abandonTransactions, cleanupAfterTransactionRemovedFromCache, dispatchSchemaChangeEventAndGetTableForNewCapturedTable, getAbandonedTransactionsCache, getConfig, getLastProcessedScn, getLastProcessedScnChangeTime, getLastScnToAbandon, getQueryString, getSchema, getTableMetadataDdl, getTransactionIdPrefix, handleCommit, handleCommitNotFoundInBuffer, handleDataEvent, handleLobWrite, handleMissingScn, handleRollback, handleRollbackNotFoundInBuffer, handleSelectLobLocator, handleStart, handleUnsupportedEvent, isTransactionIdWithNoSequence, isTransactionOverEventThreshold, isTransactionUserExcluded, isTrxIdRawValue, process, processResults, processRow, resetTransactionToStart, warnPotentiallyStuckScn

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOGGER

private static final org.slf4j.Logger LOGGER

dispatcher

private final EventDispatcher<OraclePartition,TableId> dispatcher

partition

private final OraclePartition partition

offsetContext

private final OracleOffsetContext offsetContext

metrics

private final LogMinerStreamingChangeEventSourceMetrics metrics

transactionCache

private final Map<String,MemoryTransaction> transactionCache

Cache of transactions, keyed based on the transaction's unique identifier

recentlyProcessedTransactionsCache

private final Map<String,Scn> recentlyProcessedTransactionsCache

Cache of processed transactions (committed or rolled back), keyed based on the transaction's unique identifier.

schemaChangesCache

private final Set<Scn> schemaChangesCache

Constructor Details

MemoryLogMinerEventProcessor

public MemoryLogMinerEventProcessor(ChangeEventSource.ChangeEventSourceContext context,
 OracleConnectorConfig connectorConfig,
 OracleConnection jdbcConnection,
 EventDispatcher<OraclePartition,TableId> dispatcher,
 OraclePartition partition,
 OracleOffsetContext offsetContext,
 OracleDatabaseSchema schema,
 LogMinerStreamingChangeEventSourceMetrics metrics)

Method Details

getTransactionCache

protected Map<String,MemoryTransaction> getTransactionCache()

Description copied from class: AbstractLogMinerEventProcessor

Returns the TransactionCache implementation.

Specified by:

getTransactionCache in class AbstractLogMinerEventProcessor<MemoryTransaction>

Returns:

the transaction cache, never null

createTransaction

protected MemoryTransaction createTransaction(LogMinerEventRow row)

Description copied from class: AbstractLogMinerEventProcessor

Creates a new transaction based on the supplied START event.

Specified by:

createTransaction in class AbstractLogMinerEventProcessor<MemoryTransaction>

Parameters:

row - the event row, must not be null

Returns:

the implementation-specific Transaction instance

removeEventWithRowId

protected void removeEventWithRowId(LogMinerEventRow row)

Description copied from class: AbstractLogMinerEventProcessor

Removes a specific transaction event by database row identifier.

Specified by:

removeEventWithRowId in class AbstractLogMinerEventProcessor<MemoryTransaction>

Parameters:

row - the event row that contains the row identifier, must not be null

close

public void close()
           throws Exception

Throws:

Exception

isRecentlyProcessed

protected boolean isRecentlyProcessed(String transactionId)

Description copied from class: AbstractLogMinerEventProcessor

Check whether a transaction has been recently processed through either a commit or rollback.

Overrides:

isRecentlyProcessed in class AbstractLogMinerEventProcessor<MemoryTransaction>

Parameters:

transactionId - the unique transaction id

Returns:

true if the transaction has been recently processed, false otherwise

hasSchemaChangeBeenSeen

protected boolean hasSchemaChangeBeenSeen(LogMinerEventRow row)

Description copied from class: AbstractLogMinerEventProcessor

Checks whether the LogMinerEvent row for a schema change can be emitted.

Overrides:

hasSchemaChangeBeenSeen in class AbstractLogMinerEventProcessor<MemoryTransaction>

Parameters:

row - the result set row

Returns:

true if the schema change has been seen, false otherwise.

getAndRemoveTransactionFromCache

protected MemoryTransaction getAndRemoveTransactionFromCache(String transactionId)

Description copied from class: AbstractLogMinerEventProcessor

Gets a transaction instance from the transaction cache while also removing its cache entry.

Specified by:

getAndRemoveTransactionFromCache in class AbstractLogMinerEventProcessor<MemoryTransaction>

Parameters:

transactionId - the transaction's unique identifier, should not be null

Returns:

the transaction instance if found, null if the transaction wasn't found

getTransactionEventIterator

protected Iterator<LogMinerEvent> getTransactionEventIterator(MemoryTransaction transaction)

Description copied from class: AbstractLogMinerEventProcessor

Get an iterator over the events that are part of the specified transaction.

Specified by:

getTransactionEventIterator in class AbstractLogMinerEventProcessor<MemoryTransaction>

Parameters:

transaction - the transaction instance, should never be null

Returns:

an iterator over the transaction's events, never null

finalizeTransactionCommit

protected void finalizeTransactionCommit(String transactionId,
 Scn commitScn)

Description copied from class: AbstractLogMinerEventProcessor

Finalizes the commit of a transaction.

Specified by:

finalizeTransactionCommit in class AbstractLogMinerEventProcessor<MemoryTransaction>

Parameters:

transactionId - the transaction's unique identifier, should not be null

commitScn - the transaction's system change number, should not be null

finalizeTransactionRollback

protected void finalizeTransactionRollback(String transactionId,
 Scn rollbackScn)

Description copied from class: AbstractLogMinerEventProcessor

Finalizes the rollback the specified transaction

Specified by:

finalizeTransactionRollback in class AbstractLogMinerEventProcessor<MemoryTransaction>

Parameters:

transactionId - the unique transaction identifier, never null

rollbackScn - the rollback transaction's system change number, never null

getFirstActiveTransactionKey

protected String getFirstActiveTransactionKey()

Description copied from class: AbstractLogMinerEventProcessor

Returns only the first transaction id in the transaction buffer.

Specified by:

getFirstActiveTransactionKey in class AbstractLogMinerEventProcessor<MemoryTransaction>

Returns:

the first active transaction in the buffer, or null if there is none.

handleSchemaChange

protected void handleSchemaChange(LogMinerEventRow row)
                           throws InterruptedException

Description copied from class: AbstractLogMinerEventProcessor

Handle processing a LogMinerEventRow for a DDL event.

Overrides:

handleSchemaChange in class AbstractLogMinerEventProcessor<MemoryTransaction>

Parameters:

row - the result set row

Throws:

InterruptedException - if the event dispatcher is interrupted sending the event

addToTransaction

protected void addToTransaction(String transactionId,
 LogMinerEventRow row,
 Supplier<LogMinerEvent> eventSupplier)

Description copied from class: AbstractLogMinerEventProcessor

Add a transaction to the transaction map if allowed.

Specified by:

addToTransaction in class AbstractLogMinerEventProcessor<MemoryTransaction>

Parameters:

transactionId - the unqiue transaction id

row - the LogMiner event row

eventSupplier - the supplier of the event to create if the event is allowed to be added

getTransactionEventCount

protected int getTransactionEventCount(MemoryTransaction transaction)

Description copied from class: AbstractLogMinerEventProcessor

Returns the number of events associated with the specified transaction.

Specified by:

getTransactionEventCount in class AbstractLogMinerEventProcessor<MemoryTransaction>

Parameters:

transaction - the transaction, must not be null

Returns:

the number of events in the transaction

createQueryStatement

protected PreparedStatement createQueryStatement()
                                          throws SQLException

Description copied from class: AbstractLogMinerEventProcessor

Create the JDBC query that will be used to fetch the mining result set.

Specified by:

createQueryStatement in class AbstractLogMinerEventProcessor<MemoryTransaction>

Returns:

a prepared query statement, never null

Throws:

SQLException - if a database exception occurred creating the statement

calculateNewStartScn

protected Scn calculateNewStartScn(Scn endScn,
 Scn maxCommittedScn)
                            throws InterruptedException

Description copied from class: AbstractLogMinerEventProcessor

Calculates the new starting system change number based on the current processing range.

Specified by:

calculateNewStartScn in class AbstractLogMinerEventProcessor<MemoryTransaction>

Parameters:

endScn - the end system change number for the previously mined range, never null

maxCommittedScn - the maximum committed system change number, never null

Returns:

the system change number to start then next mining iteration from, never null

Throws:

InterruptedException - if the current thread is interrupted

getTransactionCacheMinimumScn

protected Scn getTransactionCacheMinimumScn()

Description copied from class: AbstractLogMinerEventProcessor

Gets the minimum system change number stored in the transaction cache.

Specified by:

getTransactionCacheMinimumScn in class AbstractLogMinerEventProcessor<MemoryTransaction>

Returns:

the minimum system change number, never null but could be Scn.NULL.

getOldestTransactionInCache

protected Optional<MemoryTransaction> getOldestTransactionInCache()

Description copied from class: AbstractLogMinerEventProcessor

Get the oldest transaction in the cache.

Specified by:

getOldestTransactionInCache in class AbstractLogMinerEventProcessor<MemoryTransaction>

Returns:

the oldest transaction in the cache or maybe null if cache is empty