Package io.debezium.connector.oracle.logminer.processor

Class AbstractLogMinerEventProcessor<T extends AbstractTransaction>

java.lang.Object
io.debezium.connector.oracle.logminer.processor.AbstractLogMinerEventProcessor<T>
All Implemented Interfaces:
LogMinerEventProcessor, AutoCloseable
Direct Known Subclasses:
AbstractInfinispanLogMinerEventProcessor, MemoryLogMinerEventProcessor
public abstract class AbstractLogMinerEventProcessor<T extends AbstractTransaction> extends Object implements LogMinerEventProcessor
An abstract implementation of LogMinerEventProcessor that all processors should extend.
Author:
Chris Cranford

  • Field Details

  • Constructor Details

  • Method Details

    • getConfig

      protected OracleConnectorConfig getConfig()

    • getSchema

      protected OracleDatabaseSchema getSchema()

    • isRecentlyProcessed

      protected boolean isRecentlyProcessed(String transactionId)
      Check whether a transaction has been recently processed through either a commit or rollback.
      Parameters:
      transactionId - the unique transaction id
      Returns:
      true if the transaction has been recently processed, false otherwise

    • hasSchemaChangeBeenSeen

      protected boolean hasSchemaChangeBeenSeen(LogMinerEventRow row)
      Checks whether the LogMinerEvent row for a schema change can be emitted.
      Parameters:
      row - the result set row
      Returns:
      true if the schema change has been seen, false otherwise.

    • getLastProcessedScn

      protected Scn getLastProcessedScn()
      Return the last processed system change number handled by the processor.
      Returns:
      the last processed system change number, never null.

    • getTransactionCache

      protected abstract Map<String,T> getTransactionCache()
      Returns the TransactionCache implementation.
      Returns:
      the transaction cache, never null

    • createTransaction

      protected abstract T createTransaction(LogMinerEventRow row)
      Creates a new transaction based on the supplied START event.
      Parameters:
      row - the event row, must not be null
      Returns:
      the implementation-specific Transaction instance

    • removeEventWithRowId

      protected abstract void removeEventWithRowId(LogMinerEventRow row)
      Removes a specific transaction event by database row identifier.
      Parameters:
      row - the event row that contains the row identifier, must not be null

    • getTransactionEventCount

      protected abstract int getTransactionEventCount(T transaction)
      Returns the number of events associated with the specified transaction.
      Parameters:
      transaction - the transaction, must not be null
      Returns:
      the number of events in the transaction

    • isTrxIdRawValue

      protected boolean isTrxIdRawValue()

    • process

      public Scn process(OraclePartition partition, Scn startScn, Scn endScn) throws SQLException, InterruptedException
      Description copied from interface: LogMinerEventProcessor
      Process Oracle LogMiner events for a given system change number range.
      Specified by:
      process in interface LogMinerEventProcessor
      startScn - the starting system change number, must not be null
      endScn - the ending system change number, must not be null
      Returns:
      the next iteration's starting system change number, never null
      Throws:
      SQLException
      InterruptedException

    • createQueryStatement

      protected abstract PreparedStatement createQueryStatement() throws SQLException
      Create the JDBC query that will be used to fetch the mining result set.
      Returns:
      a prepared query statement, never null
      Throws:
      SQLException - if a database exception occurred creating the statement

    • calculateNewStartScn

      protected abstract Scn calculateNewStartScn(Scn endScn, Scn maxCommittedScn) throws InterruptedException
      Calculates the new starting system change number based on the current processing range.
      Parameters:
      endScn - the end system change number for the previously mined range, never null
      maxCommittedScn - the maximum committed system change number, never null
      Returns:
      the system change number to start then next mining iteration from, never null
      Throws:
      InterruptedException - if the current thread is interrupted

    • processResults

      protected void processResults(OraclePartition partition, ResultSet resultSet) throws SQLException, InterruptedException
      Processes the LogMiner results.
      Parameters:
      resultSet - the result set from a LogMiner query
      Throws:
      SQLException - if a database exception occurred
      InterruptedException - if the dispatcher was interrupted sending an event

    • processRow

      protected void processRow(OraclePartition partition, LogMinerEventRow row) throws SQLException, InterruptedException
      Processes a single LogMinerEventRow.
      Parameters:
      row - the event row, must not be null
      Throws:
      SQLException - if a database exception occurred
      InterruptedException - if the dispatcher was interrupted sending an event

    • handleMissingScn

      protected void handleMissingScn(LogMinerEventRow row)
      Handle processing a LogMinerEventRow for a MISSING_SCN event.
      Parameters:
      row - the result set row

    • handleStart

      protected void handleStart(LogMinerEventRow row)
      Handle processing a LogMinerEventRow for a START event.
      Parameters:
      row - the result set row

    • handleCommit

      protected void handleCommit(OraclePartition partition, LogMinerEventRow row) throws InterruptedException
      Handle processing a LogMinerEventRow for a COMMIT event.
      Parameters:
      row - the result set row
      Throws:
      InterruptedException - if the event dispatcher was interrupted sending events

    • isTransactionAlreadyProcessed

      protected boolean isTransactionAlreadyProcessed(Scn commitScn, Scn offsetCommitScn)
      Checks whether the transaction commit system change number has already been processed.
      Parameters:
      commitScn - the transaction's commit system change number, should not be null
      offsetCommitScn - the current offsets commit system change number, should not be null
      Returns:
      true if the transaction has been seen based on the offsets, false otherwise

    • getAndRemoveTransactionFromCache

      protected abstract T getAndRemoveTransactionFromCache(String transactionId)
      Gets a transaction instance from the transaction cache while also removing its cache entry.
      Parameters:
      transactionId - the transaction's unique identifier, should not be null
      Returns:
      the transaction instance if found, null if the transaction wasn't found

    • removeTransactionAndEventsFromCache

      protected abstract void removeTransactionAndEventsFromCache(T transaction)
      Removes the transaction and all its associated event entries from the connector's caches.
      Parameters:
      transaction - the transaction instance, should never be null

    • getTransactionEventIterator

      protected abstract Iterator<LogMinerEvent> getTransactionEventIterator(T transaction)
      Get an iterator over the events that are part of the specified transaction.
      Parameters:
      transaction - the transaction instance, should never be null
      Returns:
      an iterator over the transaction's events, never null

    • finalizeTransactionCommit

      protected abstract void finalizeTransactionCommit(String transactionId, Scn commitScn)
      Finalizes the commit of a transaction.
      Parameters:
      transactionId - the transaction's unique identifier, should not be null
      commitScn - the transaction's system change number, should not be null

    • isTransactionUserExcluded

      protected boolean isTransactionUserExcluded(T transaction)
      Check whether the supplied username associated with the specified transaction is excluded.
      Parameters:
      transaction - the transaction, never null
      Returns:
      true if the transaction should be skipped; false if transaction should be emitted

    • handleRollback

      protected void handleRollback(LogMinerEventRow row)
      Handle processing a LogMinerEventRow for a ROLLBACK event.
      Parameters:
      row - the result set row

    • finalizeTransactionRollback

      protected abstract void finalizeTransactionRollback(String transactionId, Scn rollbackScn)
      Finalizes the rollback the specified transaction
      Parameters:
      transactionId - the unique transaction identifier, never null
      rollbackScn - the rollback transaction's system change number, never null

    • handleSchemaChange

      protected void handleSchemaChange(LogMinerEventRow row) throws InterruptedException
      Handle processing a LogMinerEventRow for a DDL event.
      Parameters:
      row - the result set row
      Throws:
      InterruptedException - if the event dispatcher is interrupted sending the event

    • processTruncateEvent

      private void processTruncateEvent(LogMinerEventRow row)

    • handleSelectLobLocator

      protected void handleSelectLobLocator(LogMinerEventRow row)
      Handle processing a LogMinerEventRow for a SEL_LOB_LOCATOR event.
      Parameters:
      row - the result set row

    • handleLobWrite

      protected void handleLobWrite(LogMinerEventRow row)
      Handle processing a LogMinerEventRow for a LOB_WRITE event.
      Parameters:
      row - the result set row

    • handleLobErase

      private void handleLobErase(LogMinerEventRow row)
      Handle processing a LogMinerEventRow for a LOB_ERASE event.
      Parameters:
      row - the result set row

    • handleDataEvent

      protected void handleDataEvent(LogMinerEventRow row) throws SQLException, InterruptedException
      Handle processing a LogMinerEventRow for a INSERT, UPDATE, or DELETE event.
      Parameters:
      row - the result set row
      Throws:
      SQLException - if a database exception occurs
      InterruptedException - if the dispatch of an event is interrupted

    • warnPotentiallyStuckScn

      protected void warnPotentiallyStuckScn(Scn previousOffsetScn, Scn previousOffsetCommitScn)
      Checks to see whether the offset's scn is remaining the same across multiple mining sessions while the offset's commit_scn is changing between sessions.
      Parameters:
      previousOffsetScn - the previous offset system change number
      previousOffsetCommitScn - the previous offset commit system change number

    • getTableForDataEvent

      private Table getTableForDataEvent(LogMinerEventRow row) throws SQLException, InterruptedException
      Throws:
      SQLException
      InterruptedException

    • hasNextWithMetricsUpdate

      private boolean hasNextWithMetricsUpdate(ResultSet resultSet) throws SQLException
      Checks whether the result-set has any more data available. When a new row is available, the streaming metrics is updated with the fetch timings.
      Parameters:
      resultSet - the result set to check if any more rows exist
      Returns:
      true if another row exists, false otherwise
      Throws:
      SQLException - if there was a database exception

    • addToTransaction

      protected abstract void addToTransaction(String transactionId, LogMinerEventRow row, Supplier<LogMinerEvent> eventSupplier)
      Add a transaction to the transaction map if allowed.
      Parameters:
      transactionId - the unqiue transaction id
      row - the LogMiner event row
      eventSupplier - the supplier of the event to create if the event is allowed to be added

    • dispatchSchemaChangeEventAndGetTableForNewCapturedTable

      private Table dispatchSchemaChangeEventAndGetTableForNewCapturedTable(TableId tableId, OracleOffsetContext offsetContext, EventDispatcher<OraclePartition,TableId> dispatcher) throws SQLException, InterruptedException
      Dispatch a schema change event for a new table and get the newly created relational table model.
      Parameters:
      tableId - the unique table identifier, must not be null
      offsetContext - the offset context
      dispatcher - the event dispatcher
      Returns:
      the relational table model
      Throws:
      SQLException - if a database exception occurred
      InterruptedException - if the event dispatch was interrupted

    • getTableMetadataDdl

      private String getTableMetadataDdl(TableId tableId) throws SQLException
      Get the specified table's create DDL statement.
      Parameters:
      tableId - the table identifier, must not be null
      Returns:
      the table's create DDL statement, never null
      Throws:
      SQLException - if an exception occurred obtaining the DDL statement

    • parseDmlStatement

      private LogMinerDmlEntry parseDmlStatement(String redoSql, Table table, String transactionId)
      Parse a DML redo SQL statement.
      Parameters:
      redoSql - the redo SQL statement
      table - the table the SQL statement is for
      transactionId - the associated transaction id for the SQL statement
      Returns:
      a parse object for the redo SQL statement

    • parseLobWriteSql

      private AbstractLogMinerEventProcessor<T>.ParsedLobWriteSql parseLobWriteSql(String sql)
      Parses a LOB_WRITE operation SQL fragment.
      Parameters:
      sql - sql statement
      Returns:
      the parsed statement
      Throws:
      io.debezium.DebeziumException - if an unexpected SQL fragment is provided that cannot be parsed

    • getTransactionCacheMinimumScn

      protected abstract Scn getTransactionCacheMinimumScn()
      Gets the minimum system change number stored in the transaction cache.
      Returns:
      the minimum system change number, never null but could be Scn.NULL.