Package io.debezium.connector.oracle.logminer.processor

Class AbstractLogMinerEventProcessor<T extends AbstractTransaction>

    • Method Detail

      • isRecentlyProcessed

        protected boolean isRecentlyProcessed​(String transactionId)
        Check whether a transaction has been recently processed through either a commit or rollback.
        Parameters:
        transactionId - the unique transaction id
        Returns:
        true if the transaction has been recently processed, false otherwise

      • hasSchemaChangeBeenSeen

        protected boolean hasSchemaChangeBeenSeen​(LogMinerEventRow row)
        Checks whether the LogMinerEvent row for a schema change can be emitted.
        Parameters:
        row - the result set row
        Returns:
        true if the schema change has been seen, false otherwise.

      • getLastProcessedScn

        protected Scn getLastProcessedScn()
        Return the last processed system change number handled by the processor.
        Returns:
        the last processed system change number, never null.

      • getTransactionCache

        protected abstract Map<String,​T> getTransactionCache()
        Returns the TransactionCache implementation.
        Returns:
        the transaction cache, never null

      • createTransaction

        protected abstract T createTransaction​(LogMinerEventRow row)
        Creates a new transaction based on the supplied START event.
        Parameters:
        row - the event row, must not be null
        Returns:
        the implementation-specific Transaction instance

      • removeEventWithRowId

        protected abstract void removeEventWithRowId​(LogMinerEventRow row)
        Removes a specific transaction event by database row identifier.
        Parameters:
        row - the event row that contains the row identifier, must not be null

      • getTransactionEventCount

        protected abstract int getTransactionEventCount​(T transaction)
        Returns the number of events associated with the specified transaction.
        Parameters:
        transaction - the transaction, must not be null
        Returns:
        the number of events in the transaction

      • isTrxIdRawValue

        protected boolean isTrxIdRawValue()

      • createQueryStatement

        protected abstract PreparedStatement createQueryStatement()
                                                   throws SQLException
        Create the JDBC query that will be used to fetch the mining result set.
        Returns:
        a prepared query statement, never null
        Throws:
        SQLException - if a database exception occurred creating the statement

      • calculateNewStartScn

        protected abstract Scn calculateNewStartScn​(Scn endScn,
                                            Scn maxCommittedScn)
                                     throws InterruptedException
        Calculates the new starting system change number based on the current processing range.
        Parameters:
        endScn - the end system change number for the previously mined range, never null
        maxCommittedScn - the maximum committed system change number, never null
        Returns:
        the system change number to start then next mining iteration from, never null
        Throws:
        InterruptedException - if the current thread is interrupted

      • handleMissingScn

        protected void handleMissingScn​(LogMinerEventRow row)
        Handle processing a LogMinerEventRow for a MISSING_SCN event.
        Parameters:
        row - the result set row

      • handleStart

        protected void handleStart​(LogMinerEventRow row)
        Handle processing a LogMinerEventRow for a START event.
        Parameters:
        row - the result set row

      • isTransactionAlreadyProcessed

        protected boolean isTransactionAlreadyProcessed​(Scn commitScn,
                                                Scn offsetCommitScn)
        Checks whether the transaction commit system change number has already been processed.
        Parameters:
        commitScn - the transaction's commit system change number, should not be null
        offsetCommitScn - the current offsets commit system change number, should not be null
        Returns:
        true if the transaction has been seen based on the offsets, false otherwise

      • getAndRemoveTransactionFromCache

        protected abstract T getAndRemoveTransactionFromCache​(String transactionId)
        Gets a transaction instance from the transaction cache while also removing its cache entry.
        Parameters:
        transactionId - the transaction's unique identifier, should not be null
        Returns:
        the transaction instance if found, null if the transaction wasn't found

      • removeTransactionAndEventsFromCache

        protected abstract void removeTransactionAndEventsFromCache​(T transaction)
        Removes the transaction and all its associated event entries from the connector's caches.
        Parameters:
        transaction - the transaction instance, should never be null

      • getTransactionEventIterator

        protected abstract Iterator<LogMinerEvent> getTransactionEventIterator​(T transaction)
        Get an iterator over the events that are part of the specified transaction.
        Parameters:
        transaction - the transaction instance, should never be null
        Returns:
        an iterator over the transaction's events, never null

      • finalizeTransactionCommit

        protected abstract void finalizeTransactionCommit​(String transactionId,
                                                  Scn commitScn)
        Finalizes the commit of a transaction.
        Parameters:
        transactionId - the transaction's unique identifier, should not be null
        commitScn - the transaction's system change number, should not be null

      • isTransactionUserExcluded

        protected boolean isTransactionUserExcluded​(T transaction)
        Check whether the supplied username associated with the specified transaction is excluded.
        Parameters:
        transaction - the transaction, never null
        Returns:
        true if the transaction should be skipped; false if transaction should be emitted

      • handleRollback

        protected void handleRollback​(LogMinerEventRow row)
        Handle processing a LogMinerEventRow for a ROLLBACK event.
        Parameters:
        row - the result set row

      • finalizeTransactionRollback

        protected abstract void finalizeTransactionRollback​(String transactionId,
                                                    Scn rollbackScn)
        Finalizes the rollback the specified transaction
        Parameters:
        transactionId - the unique transaction identifier, never null
        rollbackScn - the rollback transaction's system change number, never null

      • handleSchemaChange

        protected void handleSchemaChange​(LogMinerEventRow row)
                           throws InterruptedException
        Handle processing a LogMinerEventRow for a DDL event.
        Parameters:
        row - the result set row
        Throws:
        InterruptedException - if the event dispatcher is interrupted sending the event

      • processTruncateEvent

        private void processTruncateEvent​(LogMinerEventRow row)

      • handleSelectLobLocator

        protected void handleSelectLobLocator​(LogMinerEventRow row)
        Handle processing a LogMinerEventRow for a SEL_LOB_LOCATOR event.
        Parameters:
        row - the result set row

      • handleLobWrite

        protected void handleLobWrite​(LogMinerEventRow row)
        Handle processing a LogMinerEventRow for a LOB_WRITE event.
        Parameters:
        row - the result set row

      • handleLobErase

        private void handleLobErase​(LogMinerEventRow row)
        Handle processing a LogMinerEventRow for a LOB_ERASE event.
        Parameters:
        row - the result set row

      • warnPotentiallyStuckScn

        protected void warnPotentiallyStuckScn​(Scn previousOffsetScn,
                                       Scn previousOffsetCommitScn)
        Checks to see whether the offset's scn is remaining the same across multiple mining sessions while the offset's commit_scn is changing between sessions.
        Parameters:
        previousOffsetScn - the previous offset system change number
        previousOffsetCommitScn - the previous offset commit system change number

      • hasNextWithMetricsUpdate

        private boolean hasNextWithMetricsUpdate​(ResultSet resultSet)
                                  throws SQLException
        Checks whether the result-set has any more data available. When a new row is available, the streaming metrics is updated with the fetch timings.
        Parameters:
        resultSet - the result set to check if any more rows exist
        Returns:
        true if another row exists, false otherwise
        Throws:
        SQLException - if there was a database exception

      • addToTransaction

        protected abstract void addToTransaction​(String transactionId,
                                         LogMinerEventRow row,
                                         Supplier<LogMinerEvent> eventSupplier)
        Add a transaction to the transaction map if allowed.
        Parameters:
        transactionId - the unqiue transaction id
        row - the LogMiner event row
        eventSupplier - the supplier of the event to create if the event is allowed to be added

      • dispatchSchemaChangeEventAndGetTableForNewCapturedTable

        private Table dispatchSchemaChangeEventAndGetTableForNewCapturedTable​(TableId tableId,
                                                                      OracleOffsetContext offsetContext,
                                                                      EventDispatcher<OraclePartition,​TableId> dispatcher)
                                                               throws SQLException,
                                                                      InterruptedException
        Dispatch a schema change event for a new table and get the newly created relational table model.
        Parameters:
        tableId - the unique table identifier, must not be null
        offsetContext - the offset context
        dispatcher - the event dispatcher
        Returns:
        the relational table model
        Throws:
        SQLException - if a database exception occurred
        InterruptedException - if the event dispatch was interrupted

      • getTableMetadataDdl

        private String getTableMetadataDdl​(TableId tableId)
                            throws SQLException
        Get the specified table's create DDL statement.
        Parameters:
        tableId - the table identifier, must not be null
        Returns:
        the table's create DDL statement, never null
        Throws:
        SQLException - if an exception occurred obtaining the DDL statement

      • parseDmlStatement

        private LogMinerDmlEntry parseDmlStatement​(String redoSql,
                                           Table table,
                                           String transactionId)
        Parse a DML redo SQL statement.
        Parameters:
        redoSql - the redo SQL statement
        table - the table the SQL statement is for
        transactionId - the associated transaction id for the SQL statement
        Returns:
        a parse object for the redo SQL statement

      • parseLobWriteSql

        private AbstractLogMinerEventProcessor.ParsedLobWriteSql parseLobWriteSql​(String sql)
        Parses a LOB_WRITE operation SQL fragment.
        Parameters:
        sql - sql statement
        Returns:
        the parsed statement
        Throws:
        io.debezium.DebeziumException - if an unexpected SQL fragment is provided that cannot be parsed

      • getTransactionCacheMinimumScn

        protected abstract Scn getTransactionCacheMinimumScn()
        Gets the minimum system change number stored in the transaction cache.
        Returns:
        the minimum system change number, never null but could be Scn.NULL.