Package io.debezium.connector.oracle.logminer

Class TransactionalBuffer

  • All Implemented Interfaces:
    AutoCloseable
    @NotThreadSafe
public final class TransactionalBuffer
extends Object
implements AutoCloseable
    Buffer that stores transactions and related callbacks that will be executed when a transaction commits or discarded when a transaction has been rolled back.
    Author:
    Andrey Pustovetov

    • Method Detail

      • getRolledBackTransactionIds

        Set<String> getRolledBackTransactionIds()
        Returns:
        rolled back transactions

      • registerDdlOperation

        void registerDdlOperation​(Scn scn)
        Registers a DDL operation with the buffer.
        Parameters:
        scn - the system change number

      • isDdlOperationRegistered

        boolean isDdlOperationRegistered​(Scn scn)
        Returns whether the ddl operation has been registered.
        Parameters:
        scn - the system change number
        Returns:
        true if the ddl operation has been seen and processed, false otherwise.

      • registerDmlOperation

        void registerDmlOperation​(int operation,
                          String transactionId,
                          Scn scn,
                          TableId tableId,
                          Supplier<LogMinerDmlEntry> entrySupplier,
                          Instant changeTime,
                          String rowId,
                          Object rsId)
        Register a DML operation with the transaction buffer.
        Parameters:
        operation - operation type
        transactionId - unique transaction identifier
        scn - system change number
        tableId - table identifier
        entrySupplier - parser entry provider
        changeTime - time the DML operation occurred
        rowId - unique row identifier
        rsId - rollback sequence identifier

      • registerSelectLobOperation

        void registerSelectLobOperation​(int operation,
                                String transactionId,
                                Scn scn,
                                TableId tableId,
                                Instant changeTime,
                                String rowId,
                                Object rsId,
                                String segOwner,
                                String tableName,
                                String redoSql,
                                Table table,
                                SelectLobParser selectLobParser)
        Register a SEL_LOB_LOCATOR operation with the transaction buffer.
        Parameters:
        operation - operation type
        transactionId - unique transaction identifier
        scn - system change number
        tableId - table identifier
        changeTime - time the operation occurred
        rowId - unique row identifier
        rsId - rollback sequence identifier
        segOwner - table owner
        tableName - table name
        redoSql - the redo sql statement
        table - the relational table
        selectLobParser - the select LOB parser

      • registerLobWriteOperation

        void registerLobWriteOperation​(int operation,
                               String transactionId,
                               Scn scn,
                               TableId tableId,
                               String data,
                               Instant changeTime,
                               String rowId,
                               Object rsId)
        Register a LOB_WRITE operation with the transaction buffer.
        Parameters:
        operation - operation type
        transactionId - unique transaction identifier
        scn - system change number
        tableId - table identifier
        data - data written by the LOB operation
        changeTime - time the operation occurred
        rowId - unique row identifier
        rsId - rollback sequence identifier

      • registerLobEraseOperation

        void registerLobEraseOperation​(int operation,
                               String transactionId,
                               Scn scn,
                               TableId tableId,
                               Instant changeTime,
                               String rowId,
                               Object rsId)
        Register a LOB_ERASE operation with the transction buffer.
        Parameters:
        operation - operation type
        transactionId - unique transaction identifier
        scn - system change number
        tableId - table identifier
        changeTime - time the operation occurred
        rowId - unique row identifier
        rsId - rollback sequence identifier

      • undoDmlOperation

        void undoDmlOperation​(String transactionId,
                      String undoRowId,
                      TableId tableId)
        Undo a staged DML operation in the transaction buffer.
        Parameters:
        transactionId - unique transaction identifier
        undoRowId - unique row identifier to be undone
        tableId - table identifier

      • registerTransaction

        void registerTransaction​(String transactionId,
                         Scn scn)
        Register a new transaction with the transaction buffer.
        Parameters:
        transactionId - unique transaction identifier
        scn - starting SCN of the transaction

      • commit

        boolean commit​(String transactionId,
               Scn scn,
               OracleOffsetContext offsetContext,
               Timestamp timestamp,
               ChangeEventSource.ChangeEventSourceContext context,
               String debugMessage,
               EventDispatcher<TableId> dispatcher)
        Commits a transaction by looking up the transaction in the buffer and if exists, all registered callbacks will be executed in chronological order, emitting events for each followed by a transaction commit event.
        Parameters:
        transactionId - transaction identifier
        scn - SCN of the commit.
        offsetContext - Oracle offset
        timestamp - commit timestamp
        context - context to check that source is running
        debugMessage - message
        dispatcher - event dispatcher
        Returns:
        true if committed transaction is in the buffer, was not processed yet and processed now

      • updateOffsetContext

        Scn updateOffsetContext​(OracleOffsetContext offsetContext,
                        EventDispatcher<TableId> dispatcher)
                 throws InterruptedException
        Update the offset context based on the current state of the transaction buffer.
        Parameters:
        offsetContext - offset context, should not be null
        dispatcher - event dispatcher, should not be null
        Returns:
        offset context SCN, never null
        Throws:
        InterruptedException - thrown if dispatch of heartbeat event fails

      • getMinimumScn

        Scn getMinimumScn()

      • rollback

        boolean rollback​(String transactionId,
                 String debugMessage)
        Clears registered callbacks for given transaction identifier.
        Parameters:
        transactionId - transaction id
        debugMessage - message
        Returns:
        true if the rollback is for a transaction in the buffer

      • abandonLongTransactions

        void abandonLongTransactions​(Scn thresholdScn,
                             OracleOffsetContext offsetContext)
        If for some reason the connector got restarted, the offset will point to the beginning of the oldest captured transaction. If that transaction was lasted for a long time, let say > 4 hours, the offset might be not accessible after restart, Hence we have to address these cases manually.

        In case of an abandonment, all DMLs/Commits/Rollbacks for this transaction will be ignored

        Parameters:
        thresholdScn - the smallest SVN of any transaction to keep in the buffer. All others will be removed.
        offsetContext - the offset context

      • isTransactionRegistered

        boolean isTransactionRegistered​(String txId)

      • calculateSmallestScn

        private Scn calculateSmallestScn()

      • isEmpty

        boolean isEmpty()
        Returns true if buffer is empty, otherwise false.
        Returns:
        true if buffer is empty, otherwise false

      • registerEvent

        private boolean registerEvent​(String transactionId,
                              Scn scn,
                              Instant changeTime,
                              Supplier<TransactionalBuffer.LogMinerEvent> supplier)
        Helper method to register a given TransactionalBuffer.LogMinerEvent implementation with the buffer. If the event is registered, the underlying metrics active transactions and lag will be re-calculated.
        Parameters:
        transactionId - transaction id that contained the given event
        scn - system change number for the event
        changeTime - the time the event occurred
        supplier - supplier function to generate the event if validity checks pass
        Returns:
        true if the event was registered, false otherwise

      • isRecentlyCommitted

        private boolean isRecentlyCommitted​(String transactionId)
        Returns whether the specified transaction has recently been committed.
        Parameters:
        transactionId - the transaction identifier
        Returns:
        true if the transaction has been recently committed (seen by the connector), otherwise false.

      • parseLobWriteSql

        private String parseLobWriteSql​(String sql)
        Parses a LOB_WRITE operation SQL fragment.
        Parameters:
        sql - sql statement
        Returns:
        the parsed statement
        Throws:
        io.debezium.DebeziumException - if an unexpected SQL fragment is provided that cannot be parsed

      • reconcileTransaction

        private void reconcileTransaction​(TransactionalBuffer.Transaction transaction)
        Reconcile the specified transaction by merging multiple events that should be emitted as a single logical event, such as changes made to LOB column types that involve multiple events.
        Parameters:
        transaction - transaction to be reconciled, never null

      • shouldMergeSelectLobLocatorEvent

        private boolean shouldMergeSelectLobLocatorEvent​(TransactionalBuffer.Transaction transaction,
                                                 int index,
                                                 TransactionalBuffer.SelectLobLocatorEvent event,
                                                 TransactionalBuffer.LogMinerEvent prevEvent)
        Attempts to merge the provided SEL_LOB_LOCATOR event with the previous event in the transaction.
        Parameters:
        transaction - transaction being processed, never null
        index - event index being processed
        event - event being processed, never null
        prevEvent - previous event in the transaction, can be null
        Returns:
        true if the event is merged, false if the event was not merged.

      • shouldMergeDmlEvent

        private boolean shouldMergeDmlEvent​(TransactionalBuffer.Transaction transaction,
                                    int index,
                                    TransactionalBuffer.DmlEvent event,
                                    TransactionalBuffer.LogMinerEvent prevEvent)
        Attempts to merge the provided DML event with the previous event in the transaction.
        Parameters:
        transaction - transaction being processed, never null
        index - event index being processed
        event - event being processed, never null
        prevEvent - previous event in the transaction, can be null
        Returns:
        true if the event is merged, false if the event was not merged

      • readAndCombineLobWriteEvents

        private List<String> readAndCombineLobWriteEvents​(TransactionalBuffer.Transaction transaction,
                                                  int index,
                                                  boolean binaryData)
        Reads the transaction event queue and combines all LOB_WRITE events starting at the provided index. for a SEL_LOB_LOCATOR event which is for binary data (BLOB) data types.
        Parameters:
        transaction - transaction being processed, never null
        index - index to the first LOB_WRITE operation
        Returns:
        list of string-based values for each LOB_WRITE operation

      • readAndConsumeLobEraseEvents

        private int readAndConsumeLobEraseEvents​(TransactionalBuffer.Transaction transaction,
                                         int index)
        Read and remove all LobErase events detected in the transaction event queue.
        Parameters:
        transaction - transaction being processed, never null
        index - index to the first LOB_ERASE operation
        Returns:
        number of LOB_ERASE events consumed and removed from the event queue

      • isForSameTableOrScn

        private boolean isForSameTableOrScn​(TransactionalBuffer.LogMinerEvent event,
                                    TransactionalBuffer.LogMinerEvent prevEvent)
        Checks whether the two events are for the same table or participate in the same system change.
        Parameters:
        event - current event being processed, never null
        prevEvent - previous/parent event that has been processed, may be null
        Returns:
        true if the two events are for the same table or system change number, false otherwise

      • isSameTableRow

        private boolean isSameTableRow​(TransactionalBuffer.LogMinerEvent event,
                               TransactionalBuffer.LogMinerEvent prevEvent)
        Checks whether the two events are for the same table row.
        Parameters:
        event - current event being processed, never null
        prevEvent - previous/parent event that has been processed, never null
        Returns:
        true if the two events are for the same table row, false otherwise