Package io.debezium.connector.oracle.logminer.processor

Class AbstractLogMinerEventProcessor<T extends AbstractTransaction>

java.lang.Object
io.debezium.connector.oracle.logminer.processor.AbstractLogMinerEventProcessor<T>
All Implemented Interfaces:
LogMinerEventProcessor, AutoCloseable
Direct Known Subclasses:
AbstractInfinispanLogMinerEventProcessor, MemoryLogMinerEventProcessor
public abstract class AbstractLogMinerEventProcessor<T extends AbstractTransaction> extends Object implements LogMinerEventProcessor
An abstract implementation of LogMinerEventProcessor that all processors should extend.
Author:
Chris Cranford

  • Field Details

  • Constructor Details

  • Method Details

    • getConfig

      protected OracleConnectorConfig getConfig()

    • getSchema

      protected OracleDatabaseSchema getSchema()

    • isRecentlyProcessed

      protected boolean isRecentlyProcessed(String transactionId)
      Check whether a transaction has been recently processed through either a commit or rollback.
      Parameters:
      transactionId - the unique transaction id
      Returns:
      true if the transaction has been recently processed, false otherwise

    • hasSchemaChangeBeenSeen

      protected boolean hasSchemaChangeBeenSeen(LogMinerEventRow row)
      Checks whether the LogMinerEvent row for a schema change can be emitted.
      Parameters:
      row - the result set row
      Returns:
      true if the schema change has been seen, false otherwise.

    • getLastProcessedScn

      protected Scn getLastProcessedScn()
      Return the last processed system change number handled by the processor.
      Returns:
      the last processed system change number, never null.

    • getTransactionCache

      protected abstract Map<String,T> getTransactionCache()
      Returns the TransactionCache implementation.
      Returns:
      the transaction cache, never null

    • createTransaction

      protected abstract T createTransaction(LogMinerEventRow row)
      Creates a new transaction based on the supplied START event.
      Parameters:
      row - the event row, must not be null
      Returns:
      the implementation-specific Transaction instance

    • removeEventWithRowId

      protected abstract void removeEventWithRowId(LogMinerEventRow row)
      Removes a specific transaction event by database row identifier.
      Parameters:
      row - the event row that contains the row identifier, must not be null

    • getTransactionEventCount

      protected abstract int getTransactionEventCount(T transaction)
      Returns the number of events associated with the specified transaction.
      Parameters:
      transaction - the transaction, must not be null
      Returns:
      the number of events in the transaction

    • isTrxIdRawValue

      protected boolean isTrxIdRawValue()

    • process

      public Scn process(Scn startScn, Scn endScn) throws SQLException, InterruptedException
      Description copied from interface: LogMinerEventProcessor
      Process Oracle LogMiner events for a given system change number range.
      Specified by:
      process in interface LogMinerEventProcessor
      Parameters:
      startScn - the starting system change number, must not be null
      endScn - the ending system change number, must not be null
      Returns:
      the next iteration's starting system change number, never null
      Throws:
      SQLException
      InterruptedException

    • createQueryStatement

      protected abstract PreparedStatement createQueryStatement() throws SQLException
      Create the JDBC query that will be used to fetch the mining result set.
      Returns:
      a prepared query statement, never null
      Throws:
      SQLException - if a database exception occurred creating the statement

    • calculateNewStartScn

      protected abstract Scn calculateNewStartScn(Scn endScn, Scn maxCommittedScn) throws InterruptedException
      Calculates the new starting system change number based on the current processing range.
      Parameters:
      endScn - the end system change number for the previously mined range, never null
      maxCommittedScn - the maximum committed system change number, never null
      Returns:
      the system change number to start then next mining iteration from, never null
      Throws:
      InterruptedException - if the current thread is interrupted

    • processResults

      protected void processResults(OraclePartition partition, ResultSet resultSet) throws SQLException, InterruptedException
      Processes the LogMiner results.
      Parameters:
      resultSet - the result set from a LogMiner query
      Throws:
      SQLException - if a database exception occurred
      InterruptedException - if the dispatcher was interrupted sending an event

    • processRow

      protected void processRow(OraclePartition partition, LogMinerEventRow row) throws SQLException, InterruptedException
      Processes a single LogMinerEventRow.
      Parameters:
      row - the event row, must not be null
      Throws:
      SQLException - if a database exception occurred
      InterruptedException - if the dispatcher was interrupted sending an event

    • handleMissingScn

      protected void handleMissingScn(LogMinerEventRow row)
      Handle processing a LogMinerEventRow for a MISSING_SCN event.
      Parameters:
      row - the result set row

    • handleStart

      protected void handleStart(LogMinerEventRow row)
      Handle processing a LogMinerEventRow for a START event.
      Parameters:
      row - the result set row

    • handleCommit

      protected void handleCommit(OraclePartition partition, LogMinerEventRow row) throws InterruptedException
      Handle processing a LogMinerEventRow for a COMMIT event.
      Parameters:
      row - the result set row
      Throws:
      InterruptedException - if the event dispatcher was interrupted sending events

    • getAndRemoveTransactionFromCache

      protected abstract T getAndRemoveTransactionFromCache(String transactionId)
      Gets a transaction instance from the transaction cache while also removing its cache entry.
      Parameters:
      transactionId - the transaction's unique identifier, should not be null
      Returns:
      the transaction instance if found, null if the transaction wasn't found

    • removeTransactionAndEventsFromCache

      protected abstract void removeTransactionAndEventsFromCache(T transaction)
      Removes the transaction and all its associated event entries from the connector's caches.
      Parameters:
      transaction - the transaction instance, should never be null

    • getTransactionEventIterator

      protected abstract Iterator<LogMinerEvent> getTransactionEventIterator(T transaction)
      Get an iterator over the events that are part of the specified transaction.
      Parameters:
      transaction - the transaction instance, should never be null
      Returns:
      an iterator over the transaction's events, never null

    • finalizeTransactionCommit

      protected abstract void finalizeTransactionCommit(String transactionId, Scn commitScn)
      Finalizes the commit of a transaction.
      Parameters:
      transactionId - the transaction's unique identifier, should not be null
      commitScn - the transaction's system change number, should not be null

    • isTransactionUserExcluded

      protected boolean isTransactionUserExcluded(T transaction)
      Check whether the supplied username associated with the specified transaction is excluded.
      Parameters:
      transaction - the transaction, never null
      Returns:
      true if the transaction should be skipped; false if transaction should be emitted

    • handleRollback

      protected void handleRollback(LogMinerEventRow row)
      Handle processing a LogMinerEventRow for a ROLLBACK event.
      Parameters:
      row - the result set row

    • finalizeTransactionRollback

      protected abstract void finalizeTransactionRollback(String transactionId, Scn rollbackScn)
      Finalizes the rollback the specified transaction
      Parameters:
      transactionId - the unique transaction identifier, never null
      rollbackScn - the rollback transaction's system change number, never null

    • handleSchemaChange

      protected void handleSchemaChange(LogMinerEventRow row) throws InterruptedException
      Handle processing a LogMinerEventRow for a DDL event.
      Parameters:
      row - the result set row
      Throws:
      InterruptedException - if the event dispatcher is interrupted sending the event

    • processTruncateEvent

      private void processTruncateEvent(LogMinerEventRow row)

    • handleSelectLobLocator

      protected void handleSelectLobLocator(LogMinerEventRow row)
      Handle processing a LogMinerEventRow for a SEL_LOB_LOCATOR event.
      Parameters:
      row - the result set row

    • handleLobWrite

      protected void handleLobWrite(LogMinerEventRow row)
      Handle processing a LogMinerEventRow for a LOB_WRITE event.
      Parameters:
      row - the result set row

    • handleLobErase

      private void handleLobErase(LogMinerEventRow row)
      Handle processing a LogMinerEventRow for a LOB_ERASE event.
      Parameters:
      row - the result set row

    • handleDataEvent

      protected void handleDataEvent(LogMinerEventRow row) throws SQLException, InterruptedException
      Handle processing a LogMinerEventRow for a INSERT, UPDATE, or DELETE event.
      Parameters:
      row - the result set row
      Throws:
      SQLException - if a database exception occurs
      InterruptedException - if the dispatch of an event is interrupted

    • handleUnsupportedEvent

      protected void handleUnsupportedEvent(LogMinerEventRow row)

    • warnPotentiallyStuckScn

      protected void warnPotentiallyStuckScn(Scn previousOffsetScn, Map<Integer,Scn> previousOffsetCommitScns)
      Checks to see whether the offset's scn is remaining the same across multiple mining sessions while the offset's commit_scn is changing between sessions.
      Parameters:
      previousOffsetScn - the previous offset system change number
      previousOffsetCommitScns - the previous offset commit system change number

    • getTableForDataEvent

      private Table getTableForDataEvent(LogMinerEventRow row) throws SQLException, InterruptedException
      Throws:
      SQLException
      InterruptedException

    • hasNextWithMetricsUpdate

      private boolean hasNextWithMetricsUpdate(ResultSet resultSet) throws SQLException
      Checks whether the result-set has any more data available. When a new row is available, the streaming metrics is updated with the fetch timings.
      Parameters:
      resultSet - the result set to check if any more rows exist
      Returns:
      true if another row exists, false otherwise
      Throws:
      SQLException - if there was a database exception

    • addToTransaction

      protected abstract void addToTransaction(String transactionId, LogMinerEventRow row, Supplier<LogMinerEvent> eventSupplier)
      Add a transaction to the transaction map if allowed.
      Parameters:
      transactionId - the unqiue transaction id
      row - the LogMiner event row
      eventSupplier - the supplier of the event to create if the event is allowed to be added

    • dispatchSchemaChangeEventAndGetTableForNewCapturedTable

      private Table dispatchSchemaChangeEventAndGetTableForNewCapturedTable(TableId tableId, OracleOffsetContext offsetContext, EventDispatcher<OraclePartition,TableId> dispatcher) throws SQLException, InterruptedException
      Dispatch a schema change event for a new table and get the newly created relational table model.
      Parameters:
      tableId - the unique table identifier, must not be null
      offsetContext - the offset context
      dispatcher - the event dispatcher
      Returns:
      the relational table model
      Throws:
      SQLException - if a database exception occurred
      InterruptedException - if the event dispatch was interrupted

    • getTableMetadataDdl

      private String getTableMetadataDdl(TableId tableId) throws SQLException
      Get the specified table's create DDL statement.
      Parameters:
      tableId - the table identifier, must not be null
      Returns:
      the table's create DDL statement, never null
      Throws:
      SQLException - if an exception occurred obtaining the DDL statement

    • parseDmlStatement

      private LogMinerDmlEntry parseDmlStatement(String redoSql, Table table)
      Parse a DML redo SQL statement.
      Parameters:
      redoSql - the redo SQL statement
      table - the table the SQL statement is for
      Returns:
      a parse object for the redo SQL statement

    • parseLobWriteSql

      private AbstractLogMinerEventProcessor<T>.ParsedLobWriteSql parseLobWriteSql(String sql)
      Parses a LOB_WRITE operation SQL fragment.
      Parameters:
      sql - sql statement
      Returns:
      the parsed statement
      Throws:
      io.debezium.DebeziumException - if an unexpected SQL fragment is provided that cannot be parsed

    • getTransactionCacheMinimumScn

      protected abstract Scn getTransactionCacheMinimumScn()
      Gets the minimum system change number stored in the transaction cache.
      Returns:
      the minimum system change number, never null but could be Scn.NULL.

    • isTransactionIdWithNoSequence

      protected boolean isTransactionIdWithNoSequence(String transactionId)
      Returns whether the transaction id has no sequence number component. Oracle transaction identifiers are a composite of:
      1. Undo segment number
      2. Slot numbber of the transaction that generated the change
      3. Sequence number of the transaction that generated the change
      When Oracle LogMiner mines records, it is possible that when an undo operation is detected, often the product of a constraint violation, the LogMiner row will have the same explicit XID (transaction id) as the source operation that we should undo; however, if the record to be undone was mined in a prior iteration, Oracle LogMiner won't be able to make a link back to the full transaction's sequence number, therefore the XID value for the undo row will contain only the undo segment number and slot number, setting the sequence number to 4294967295 (aka -1 or 0xFFFFFFFF). This method explicitly checks if the provided transaction id has the no sequence sentinel value and if so, returns true; otherwise returns false.
      Parameters:
      transactionId - the transaction identifier to check, should not be null
      Returns:
      true if the transaction has no sequence reference, false if it does

    • getTransactionIdPrefix

      protected String getTransactionIdPrefix(String transactionId)

    • isTransactionOverEventThreshold

      protected boolean isTransactionOverEventThreshold(T transaction)

    • abandonTransactionOverEventThreshold

      protected void abandonTransactionOverEventThreshold(T transaction)