Package io.debezium.connector.oracle.logminer.processor.memory

Class MemoryLogMinerEventProcessor

java.lang.Object

io.debezium.connector.oracle.logminer.processor.AbstractLogMinerEventProcessor

io.debezium.connector.oracle.logminer.processor.memory.MemoryLogMinerEventProcessor

All Implemented Interfaces:

LogMinerEventProcessor, AutoCloseable

public class MemoryLogMinerEventProcessor
extends AbstractLogMinerEventProcessor

A LogMinerEventProcessor that uses the JVM heap to store events as they're being processed and emitted from Oracle LogMiner.

Author:

Chris Cranford

Nested Class Summary

Nested classes/interfaces inherited from class io.debezium.connector.oracle.logminer.processor.AbstractLogMinerEventProcessor

AbstractLogMinerEventProcessor.Counters

Field Summary

Fields

Modifier and Type	Field	Description
private Set<String>	abandonedTransactionsCache
private ChangeEventSource.ChangeEventSourceContext	context
private Scn	currentOffsetCommitScn
private Scn	currentOffsetScn
private EventDispatcher<TableId>	dispatcher
private OracleConnection	jdbcConnection
private Scn	lastCommittedScn
private static org.slf4j.Logger	LOGGER
private Scn	maxCommittedScn
private OracleStreamingChangeEventSourceMetrics	metrics
private OracleOffsetContext	offsetContext
private OraclePartition	partition
private Map<String,Scn>	recentlyCommittedTransactionsCache
private Set<String>	rollbackTransactionsCache
private Set<Scn>	schemaChangesCache
private MemoryTransactionCache	transactionCache

Fields inherited from class io.debezium.connector.oracle.logminer.processor.AbstractLogMinerEventProcessor

counters

Constructor Summary

Constructors

Constructor	Description
MemoryLogMinerEventProcessor(ChangeEventSource.ChangeEventSourceContext context, OracleConnectorConfig connectorConfig, OracleConnection jdbcConnection, EventDispatcher<TableId> dispatcher, OraclePartition partition, OracleOffsetContext offsetContext, OracleDatabaseSchema schema, OracleStreamingChangeEventSourceMetrics metrics)

Method Summary

Modifier and Type	Method	Description
void	abandonTransactions(Duration retention)	A callback for the event processor to abandon long running transactions.
private Scn	calculateNewStartScn(Scn endScn)
void	close()
private PreparedStatement	createQueryStatement()
protected Optional<Scn>	getLastScnToAbandon(OracleConnection connection, Scn offsetScn, Duration retention)	Calculates the SCN as a watermark to abandon for long running transactions.
protected TransactionCache<?>	getTransactionCache()	Returns the TransactionCache implementation.
protected void	handleCommit(LogMinerEventRow row)	Handle processing a LogMinerEventRow for a COMMIT event.
protected void	handleRollback(LogMinerEventRow row)	Handle processing a LogMinerEventRow for a ROLLBACK event.
protected void	handleSchemaChange(LogMinerEventRow row)	Handle processing a LogMinerEventRow for a DDL event.
protected boolean	hasSchemaChangeBeenSeen(LogMinerEventRow row)	Checks whether the LogMinerEvent row for a schema change can be emitted.
protected boolean	isRecentlyCommitted(String transactionId)	Check whether a transaction has been recently committed.
protected boolean	isTransactionIdAllowed(String transactionId)	Return whether a give transaction can be added to the processor's buffer.
Scn	process(Scn startScn, Scn endScn)	Process Oracle LogMiner events for a given system change number range.

Methods inherited from class io.debezium.connector.oracle.logminer.processor.AbstractLogMinerEventProcessor

addToTransaction, getConfig, getLastProcessedScn, getReconciliation, getSchema, handleDataEvent, handleLobWrite, handleMissingScn, handleSelectLobLocator, handleStart, isTrxIdRawValue, processResults, processRow, warnPotentiallyStuckScn

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOGGER

private static final org.slf4j.Logger LOGGER

context

private final ChangeEventSource.ChangeEventSourceContext context

jdbcConnection

private final OracleConnection jdbcConnection

dispatcher

private final EventDispatcher<TableId> dispatcher

partition

private final OraclePartition partition

offsetContext

private final OracleOffsetContext offsetContext

metrics

private final OracleStreamingChangeEventSourceMetrics metrics

transactionCache

private final MemoryTransactionCache transactionCache

recentlyCommittedTransactionsCache

private final Map<String,Scn> recentlyCommittedTransactionsCache

schemaChangesCache

private final Set<Scn> schemaChangesCache

abandonedTransactionsCache

private final Set<String> abandonedTransactionsCache

rollbackTransactionsCache

private final Set<String> rollbackTransactionsCache

currentOffsetScn

private Scn currentOffsetScn

currentOffsetCommitScn

private Scn currentOffsetCommitScn

lastCommittedScn

private Scn lastCommittedScn

maxCommittedScn

private Scn maxCommittedScn

Constructor Detail

MemoryLogMinerEventProcessor

public MemoryLogMinerEventProcessor(ChangeEventSource.ChangeEventSourceContext context,
                                    OracleConnectorConfig connectorConfig,
                                    OracleConnection jdbcConnection,
                                    EventDispatcher<TableId> dispatcher,
                                    OraclePartition partition,
                                    OracleOffsetContext offsetContext,
                                    OracleDatabaseSchema schema,
                                    OracleStreamingChangeEventSourceMetrics metrics)

Method Detail

getTransactionCache

protected TransactionCache<?> getTransactionCache()

Description copied from class: AbstractLogMinerEventProcessor

Returns the TransactionCache implementation.

Specified by:

getTransactionCache in class AbstractLogMinerEventProcessor

Returns:

the transaction cache, never null

close

public void close()
           throws Exception

Throws:

Exception

process

public Scn process(Scn startScn,
                   Scn endScn)
            throws SQLException,
                   InterruptedException

Description copied from interface: LogMinerEventProcessor

Process Oracle LogMiner events for a given system change number range.

Parameters:

startScn - the starting system change number, must not be null

endScn - the ending system change number, must not be null

Returns:

the next iteration's starting system change number, never null

Throws:

SQLException

InterruptedException

abandonTransactions

public void abandonTransactions(Duration retention)

Description copied from interface: LogMinerEventProcessor

A callback for the event processor to abandon long running transactions.

Parameters:

retention - the maximum duration in which long running transactions are allowed.

isRecentlyCommitted

protected boolean isRecentlyCommitted(String transactionId)

Description copied from class: AbstractLogMinerEventProcessor

Check whether a transaction has been recently committed. Any implementation that does not support recently-committed tracking should return false.

Overrides:

isRecentlyCommitted in class AbstractLogMinerEventProcessor

Parameters:

transactionId - the unique transaction id

Returns:

true if the transaction has been recently committed, false otherwise

isTransactionIdAllowed

protected boolean isTransactionIdAllowed(String transactionId)

Description copied from class: AbstractLogMinerEventProcessor

Return whether a give transaction can be added to the processor's buffer. The default implementation is to allow all transaction ids.

Overrides:

isTransactionIdAllowed in class AbstractLogMinerEventProcessor

Parameters:

transactionId - the unique transaction id

Returns:

whether a transaction id can be added to the processor's buffer

hasSchemaChangeBeenSeen

protected boolean hasSchemaChangeBeenSeen(LogMinerEventRow row)

Description copied from class: AbstractLogMinerEventProcessor

Checks whether the LogMinerEvent row for a schema change can be emitted.

Overrides:

hasSchemaChangeBeenSeen in class AbstractLogMinerEventProcessor

Parameters:

row - the result set row

Returns:

true if the schema change has been seen, false otherwise.

handleCommit

protected void handleCommit(LogMinerEventRow row)
                     throws InterruptedException

Description copied from class: AbstractLogMinerEventProcessor

Handle processing a LogMinerEventRow for a COMMIT event.

Specified by:

handleCommit in class AbstractLogMinerEventProcessor

Parameters:

row - the result set row

Throws:

InterruptedException - if the event dispatcher was interrupted sending events

handleRollback

protected void handleRollback(LogMinerEventRow row)

Description copied from class: AbstractLogMinerEventProcessor

Handle processing a LogMinerEventRow for a ROLLBACK event. The default implementation is a no-op.

Overrides:

handleRollback in class AbstractLogMinerEventProcessor

Parameters:

row - the result set row

handleSchemaChange

protected void handleSchemaChange(LogMinerEventRow row)
                           throws InterruptedException

Description copied from class: AbstractLogMinerEventProcessor

Handle processing a LogMinerEventRow for a DDL event.

Overrides:

handleSchemaChange in class AbstractLogMinerEventProcessor

Parameters:

row - the result set row

Throws:

InterruptedException - if the event dispatcher is interrupted sending the event

createQueryStatement

private PreparedStatement createQueryStatement()
                                        throws SQLException

Throws:

SQLException

calculateNewStartScn

private Scn calculateNewStartScn(Scn endScn)
                          throws InterruptedException

Throws:

InterruptedException

getLastScnToAbandon

protected Optional<Scn> getLastScnToAbandon(OracleConnection connection,
                                            Scn offsetScn,
                                            Duration retention)

Calculates the SCN as a watermark to abandon for long running transactions. The criteria is do not let the offset SCN expire from archives older the specified retention hours.

Parameters:

connection - database connection, should not be null

offsetScn - offset system change number, should not be null

retention - duration to tolerate long running transactions before being abandoned, must not be null

Returns:

an optional system change number as the watermark for transaction buffer abandonment