Package io.debezium.connector.oracle.logminer.processor

Class AbstractLogMinerEventProcessor

java.lang.Object

io.debezium.connector.oracle.logminer.processor.AbstractLogMinerEventProcessor

All Implemented Interfaces:

LogMinerEventProcessor, AutoCloseable

Direct Known Subclasses:

InfinispanLogMinerEventProcessor, MemoryLogMinerEventProcessor

public abstract class AbstractLogMinerEventProcessor
extends Object
implements LogMinerEventProcessor

An abstract implementation of LogMinerEventProcessor that all processors should extend.

Author:

Chris Cranford

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
protected class	AbstractLogMinerEventProcessor.Counters	Wrapper for all counter variables

Field Summary

Fields

Modifier and Type	Field	Description
private OracleConnectorConfig	connectorConfig
private ChangeEventSource.ChangeEventSourceContext	context
protected AbstractLogMinerEventProcessor.Counters	counters
private EventDispatcher<TableId>	dispatcher
private LogMinerDmlParser	dmlParser
private static org.slf4j.Logger	LOGGER
private OracleStreamingChangeEventSourceMetrics	metrics
private OracleOffsetContext	offsetContext
private OraclePartition	partition
private TransactionReconciliation	reconciliation
private OracleDatabaseSchema	schema
private SelectLobParser	selectLobParser

Constructor Summary

Constructors

Constructor	Description
AbstractLogMinerEventProcessor(ChangeEventSource.ChangeEventSourceContext context, OracleConnectorConfig connectorConfig, OracleDatabaseSchema schema, OraclePartition partition, OracleOffsetContext offsetContext, EventDispatcher<TableId> dispatcher, OracleStreamingChangeEventSourceMetrics metrics)

Method Summary

Modifier and Type	Method	Description
protected void	addToTransaction(String transactionId, LogMinerEventRow row, Supplier<LogMinerEvent> eventSupplier)	Add a transaction to the transaction map if allowed.
private Table	dispatchSchemaChangeEventAndGetTableForNewCapturedTable(TableId tableId, OracleOffsetContext offsetContext, EventDispatcher<TableId> dispatcher)	Dispatch a schema change event for a new table and get the newly created relational table model.
protected OracleConnectorConfig	getConfig()
protected TransactionReconciliation	getReconciliation()
protected OracleDatabaseSchema	getSchema()
private String	getTableMetadataDdl(TableId tableId)	Get the specified table's create DDL statement.
protected abstract TransactionCache<?>	getTransactionCache()	Returns the TransactionCache implementation.
protected abstract void	handleCommit(LogMinerEventRow row)	Handle processing a LogMinerEventRow for a COMMIT event.
protected void	handleDataEvent(LogMinerEventRow row)	Handle processing a LogMinerEventRow for a INSERT, UPDATE, or DELETE event.
private void	handleLobErase(LogMinerEventRow row)	Handle processing a LogMinerEventRow for a LOB_ERASE event.
protected void	handleLobWrite(LogMinerEventRow row)	Handle processing a LogMinerEventRow for a LOB_WRITE event.
protected void	handleMissingScn(LogMinerEventRow row)	Handle processing a LogMinerEventRow for a MISSING_SCN event.
protected void	handleRollback(LogMinerEventRow row)	Handle processing a LogMinerEventRow for a ROLLBACK event.
protected void	handleSchemaChange(LogMinerEventRow row)	Handle processing a LogMinerEventRow for a DDL event.
protected void	handleSelectLobLocator(LogMinerEventRow row)	Handle processing a LogMinerEventRow for a SEL_LOB_LOCATOR event.
protected void	handleStart(LogMinerEventRow row)	Handle processing a LogMinerEventRow for a START event.
private boolean	hasNextWithMetricsUpdate(ResultSet resultSet)	Checks whether the result-set has any more data available.
protected boolean	hasSchemaChangeBeenSeen(LogMinerEventRow row)	Checks whether the LogMinerEvent row for a schema change can be emitted.
protected boolean	isRecentlyCommitted(String transactionId)	Check whether a transaction has been recently committed.
protected boolean	isTransactionIdAllowed(String transactionId)	Return whether a give transaction can be added to the processor's buffer.
protected boolean	isTrxIdRawValue()
private LogMinerDmlEntry	parseDmlStatement(String redoSql, Table table, String transactionId)	Parse a DML redo SQL statement.
private String	parseLobWriteSql(String sql)	Parses a LOB_WRITE operation SQL fragment.
protected void	processResults(ResultSet resultSet)	Processes the LogMiner results.
protected void	processRow(LogMinerEventRow row)	Processes a single LogMinerEventRow.
protected void	warnPotentiallyStuckScn(Scn previousOffsetScn, Scn previousOffsetCommitScn)	Checks to see whether the offset's scn is remaining the same across multiple mining sessions while the offset's commit_scn is changing between sessions.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.lang.AutoCloseable

close

Methods inherited from interface io.debezium.connector.oracle.logminer.processor.LogMinerEventProcessor

abandonTransactions, process

Field Detail

LOGGER

private static final org.slf4j.Logger LOGGER

context

private final ChangeEventSource.ChangeEventSourceContext context

connectorConfig

private final OracleConnectorConfig connectorConfig

schema

private final OracleDatabaseSchema schema

partition

private final OraclePartition partition

offsetContext

private final OracleOffsetContext offsetContext

dispatcher

private final EventDispatcher<TableId> dispatcher

metrics

private final OracleStreamingChangeEventSourceMetrics metrics

reconciliation

private final TransactionReconciliation reconciliation

dmlParser

private final LogMinerDmlParser dmlParser

selectLobParser

private final SelectLobParser selectLobParser

counters

protected final AbstractLogMinerEventProcessor.Counters counters

Constructor Detail

AbstractLogMinerEventProcessor

public AbstractLogMinerEventProcessor(ChangeEventSource.ChangeEventSourceContext context,
                                      OracleConnectorConfig connectorConfig,
                                      OracleDatabaseSchema schema,
                                      OraclePartition partition,
                                      OracleOffsetContext offsetContext,
                                      EventDispatcher<TableId> dispatcher,
                                      OracleStreamingChangeEventSourceMetrics metrics)

Method Detail

getConfig

protected OracleConnectorConfig getConfig()

getSchema

protected OracleDatabaseSchema getSchema()

getReconciliation

protected TransactionReconciliation getReconciliation()

isRecentlyCommitted

protected boolean isRecentlyCommitted(String transactionId)

Check whether a transaction has been recently committed. Any implementation that does not support recently-committed tracking should return false.

Parameters:

transactionId - the unique transaction id

Returns:

true if the transaction has been recently committed, false otherwise

hasSchemaChangeBeenSeen

protected boolean hasSchemaChangeBeenSeen(LogMinerEventRow row)

Checks whether the LogMinerEvent row for a schema change can be emitted.

Parameters:

row - the result set row

Returns:

true if the schema change has been seen, false otherwise.

isTransactionIdAllowed

protected boolean isTransactionIdAllowed(String transactionId)

Return whether a give transaction can be added to the processor's buffer. The default implementation is to allow all transaction ids.

Parameters:

transactionId - the unique transaction id

Returns:

whether a transaction id can be added to the processor's buffer

getTransactionCache

protected abstract TransactionCache<?> getTransactionCache()

Returns the TransactionCache implementation.

Returns:

the transaction cache, never null

isTrxIdRawValue

protected boolean isTrxIdRawValue()

processResults

protected void processResults(ResultSet resultSet)
                       throws SQLException,
                              InterruptedException

Processes the LogMiner results.

Parameters:

resultSet - the result set from a LogMiner query

Throws:

SQLException - if a database exception occurred

InterruptedException - if the dispatcher was interrupted sending an event

processRow

protected void processRow(LogMinerEventRow row)
                   throws SQLException,
                          InterruptedException

Processes a single LogMinerEventRow.

Parameters:

row - the event row, must not be null

Throws:

SQLException - if a database exception occurred

InterruptedException - if the dispatcher was interrupted sending an event

handleMissingScn

protected void handleMissingScn(LogMinerEventRow row)

Handle processing a LogMinerEventRow for a MISSING_SCN event.

Parameters:

row - the result set row

handleStart

protected void handleStart(LogMinerEventRow row)

Handle processing a LogMinerEventRow for a START event.

Parameters:

row - the result set row

handleCommit

protected abstract void handleCommit(LogMinerEventRow row)
                              throws InterruptedException

Handle processing a LogMinerEventRow for a COMMIT event.

Parameters:

row - the result set row

Throws:

InterruptedException - if the event dispatcher was interrupted sending events

handleRollback

protected void handleRollback(LogMinerEventRow row)

Handle processing a LogMinerEventRow for a ROLLBACK event. The default implementation is a no-op.

Parameters:

row - the result set row

handleSchemaChange

protected void handleSchemaChange(LogMinerEventRow row)
                           throws InterruptedException

Handle processing a LogMinerEventRow for a DDL event.

Parameters:

row - the result set row

Throws:

InterruptedException - if the event dispatcher is interrupted sending the event

handleSelectLobLocator

protected void handleSelectLobLocator(LogMinerEventRow row)

Handle processing a LogMinerEventRow for a SEL_LOB_LOCATOR event.

Parameters:

row - the result set row

handleLobWrite

protected void handleLobWrite(LogMinerEventRow row)

Handle processing a LogMinerEventRow for a LOB_WRITE event.

Parameters:

row - the result set row

handleLobErase

private void handleLobErase(LogMinerEventRow row)

Handle processing a LogMinerEventRow for a LOB_ERASE event.

Parameters:

row - the result set row

handleDataEvent

protected void handleDataEvent(LogMinerEventRow row)
                        throws SQLException,
                               InterruptedException

Handle processing a LogMinerEventRow for a INSERT, UPDATE, or DELETE event.

Parameters:

row - the result set row

Throws:

SQLException - if a database exception occurs

InterruptedException - if the dispatch of an event is interrupted

warnPotentiallyStuckScn

protected void warnPotentiallyStuckScn(Scn previousOffsetScn,
                                       Scn previousOffsetCommitScn)

Checks to see whether the offset's scn is remaining the same across multiple mining sessions while the offset's commit_scn is changing between sessions.

Parameters:

previousOffsetScn - the previous offset system change number

previousOffsetCommitScn - the previous offset commit system change number

hasNextWithMetricsUpdate

private boolean hasNextWithMetricsUpdate(ResultSet resultSet)
                                  throws SQLException

Checks whether the result-set has any more data available. When a new row is available, the streaming metrics is updated with the fetch timings.

Parameters:

resultSet - the result set to check if any more rows exist

Returns:

true if another row exists, false otherwise

Throws:

SQLException - if there was a database exception

addToTransaction

protected void addToTransaction(String transactionId,
                                LogMinerEventRow row,
                                Supplier<LogMinerEvent> eventSupplier)

Add a transaction to the transaction map if allowed.

Parameters:

transactionId - the unqiue transaction id

row - the LogMiner event row

eventSupplier - the supplier of the event to create if the event is allowed to be added

dispatchSchemaChangeEventAndGetTableForNewCapturedTable

private Table dispatchSchemaChangeEventAndGetTableForNewCapturedTable(TableId tableId,
                                                                      OracleOffsetContext offsetContext,
                                                                      EventDispatcher<TableId> dispatcher)
                                                               throws SQLException,
                                                                      InterruptedException

Dispatch a schema change event for a new table and get the newly created relational table model.

Parameters:

tableId - the unique table identifier, must not be null

offsetContext - the offset context

dispatcher - the event dispatcher

Returns:

the relational table model

Throws:

SQLException - if a database exception occurred

InterruptedException - if the event dispatch was interrupted

getTableMetadataDdl

private String getTableMetadataDdl(TableId tableId)
                            throws SQLException

Get the specified table's create DDL statement.

Parameters:

tableId - the table identifier, must not be null

Returns:

the table's create DDL statement, never null

Throws:

SQLException - if an exception occurred obtaining the DDL statement

parseDmlStatement

private LogMinerDmlEntry parseDmlStatement(String redoSql,
                                           Table table,
                                           String transactionId)

Parse a DML redo SQL statement.

Parameters:

redoSql - the redo SQL statement

table - the table the SQL statement is for

transactionId - the associated transaction id for the SQL statement

Returns:

a parse object for the redo SQL statement

parseLobWriteSql

private String parseLobWriteSql(String sql)

Parses a LOB_WRITE operation SQL fragment.

Parameters:

sql - sql statement

Returns:

the parsed statement

Throws:

io.debezium.DebeziumException - if an unexpected SQL fragment is provided that cannot be parsed