Package io.debezium.connector.oracle.logminer

Class LogMinerStreamingChangeEventSource

    • Method Detail

      • getFirstScnInLogs

        private Scn getFirstScnInLogs​(OracleConnection connection)
                       throws SQLException
        Gets the first system change number in both archive and redo logs.
        Parameters:
        connection - database connection, should not be null
        Returns:
        the oldest system change number
        Throws:
        SQLException - if a database exception occurred
        io.debezium.DebeziumException - if the oldest system change number cannot be found due to no logs available

      • buildDataDictionary

        private void buildDataDictionary​(OracleConnection connection)
                          throws SQLException
        Requests Oracle to build the data dictionary. During the build step, Oracle will perform an additional series of redo log switches. Additionally, this call may introduce a delay in delivering incremental changes since the dictionary will need to have statistics gathered, analyzed, and prepared by LogMiner before any redo entries can be mined. This should only be used in conjunction with the mining strategy OracleConnectorConfig.LogMiningStrategy.CATALOG_IN_REDO.
        Parameters:
        connection - database connection
        Throws:
        SQLException - if a database exception occurred

      • hasLogSwitchOccurred

        private boolean hasLogSwitchOccurred()
                              throws SQLException
        Checks whether a database log switch has occurred and updates metrics if so.
        Returns:
        true if a log switch was detected, otherwise false
        Throws:
        SQLException - if a database exception occurred

      • getCurrentRedoLogFiles

        private Set<String> getCurrentRedoLogFiles​(OracleConnection connection)
                                    throws SQLException
        Get a list of all the CURRENT redo log file names. For Oracle RAC clusters, multiple filenames will be returned, one for each node that participates in the cluster.
        Parameters:
        connection - database connection, should not be null
        Returns:
        unique set of all current redo log file names, with full paths, never null
        Throws:
        SQLException - if a database exception occurred

      • getCurrentRedoLogSequences

        private List<BigInteger> getCurrentRedoLogSequences()
                                             throws SQLException
        Get the current redo log sequence(s). In an Oracle RAC environment, there are multiple current redo logs and therefore this method returns multiple values, each relating to a single RAC node in the Oracle cluster.
        Returns:
        list of sequence numbers
        Throws:
        SQLException - if a database exception occurred

      • setNlsSessionParameters

        private void setNlsSessionParameters​(OracleConnection connection)
                              throws SQLException
        Sets the NLS parameters for the mining session.
        Parameters:
        connection - database connection, should not be null
        Throws:
        SQLException - if a database exception occurred

      • getDatabaseSystemTime

        private OffsetDateTime getDatabaseSystemTime​(OracleConnection connection)
                                      throws SQLException
        Get the database system time in the database system's time zone.
        Parameters:
        connection - database connection, should not be null
        Returns:
        the database system time
        Throws:
        SQLException - if a database exception occurred

      • startMiningSession

        public void startMiningSession​(OracleConnection connection,
                               Scn startScn,
                               Scn endScn)
                        throws SQLException
        Starts a new Oracle LogMiner session. When this is called, LogMiner prepares all the necessary state for an upcoming LogMiner view query. If the mining statement defines using DDL tracking, the data dictionary will be mined as a part of this call to prepare DDL tracking state for the upcoming LogMiner view query.
        Parameters:
        connection - database connection, should not be null
        startScn - mining session's starting system change number (inclusive), should not be null
        endScn - mining session's ending system change number (inclusive), can be null
        Throws:
        SQLException - if mining session failed to start

      • endMiningSession

        public void endMiningSession​(OracleConnection connection,
                             OracleOffsetContext offsetContext)
                      throws SQLException
        End the current Oracle LogMiner session, if one is in progress. If the current session does not have an active mining session, a log message is recorded and the method is a no-op.
        Parameters:
        connection - database connection, should not be null
        offsetContext - connector offset context, should not be null
        Throws:
        SQLException - if the current mining session cannot be ended gracefully

      • calculateEndScn

        private Scn calculateEndScn​(OracleConnection connection,
                            Scn startScn,
                            Scn prevEndScn)
                     throws SQLException
        Calculates the mining session's end system change number. This calculation is based upon a sliding window algorithm to where if the connector is falling behind, the mining session's end point will be calculated based on the batch size and either be increased up to the maximum batch size or reduced to as low as the minimum batch size. Additionally, this method calculates and maintains a sliding algorithm for the sleep time between the mining sessions, increasing the pause up to the maximum sleep time if the connector is not behind or is mining too quick and reducing the pause down to the mimum sleep time if the connector has fallen behind and needs to catch-up faster.
        Parameters:
        connection - database connection, should not be null
        startScn - upcoming mining session's starting change number, should not be null
        prevEndScn - last mining session's ending system change number, can be null
        Returns:
        the ending system change number to be used for the upcoming mining session, never null
        Throws:
        SQLException - if the current max system change number cannot be obtained from the database

      • checkSupplementalLogging

        private void checkSupplementalLogging​(OracleConnection connection,
                                      String pdbName,
                                      OracleDatabaseSchema schema)
                               throws SQLException
        Validates the supplemental logging configuration for the source database and its captured tables.
        Parameters:
        connection - database connection, should not be null
        pdbName - pluggable database name, can be null when not using pluggable databases
        schema - connector's database schema, should not be null
        Throws:
        SQLException - if a database exception occurred

      • isDatabaseAllSupplementalLoggingEnabled

        private boolean isDatabaseAllSupplementalLoggingEnabled​(OracleConnection connection)
                                                 throws SQLException
        Returns whether the database is configured with ALL supplemental logging.
        Parameters:
        connection - database connection, must not be null
        Returns:
        true if all supplemental logging is enabled, false otherwise
        Throws:
        SQLException - if a database exception occurred

      • isDatabaseMinSupplementalLoggingEnabled

        private boolean isDatabaseMinSupplementalLoggingEnabled​(OracleConnection connection)
                                                 throws SQLException
        Returns whether the database is configured with MIN supplemental logging.
        Parameters:
        connection - database connection, must not be null
        Returns:
        true if min supplemental logging is enabled, false otherwise
        Throws:
        SQLException - if a database exception occurred

      • isTableAllColumnsSupplementalLoggingEnabled

        private boolean isTableAllColumnsSupplementalLoggingEnabled​(OracleConnection connection,
                                                            TableId tableId)
                                                     throws SQLException
        Return whether the table is configured with ALL COLUMN supplemental logging.
        Parameters:
        connection - database connection, must not be null
        tableId - table identifier, must not be null
        Returns:
        true if all column supplemental logging is enabled, false otherwise
        Throws:
        SQLException - if a database exception occurred

      • resolveFlushStrategy

        private LogWriterFlushStrategy resolveFlushStrategy()
        Resolves the Oracle LGWR buffer flushing strategy.
        Returns:
        the strategy to be used to flush Oracle's LGWR process, never null.

      • waitForStartScnInArchiveLogs

        private boolean waitForStartScnInArchiveLogs​(ChangeEventSource.ChangeEventSourceContext context,
                                             Scn startScn)
                                      throws SQLException,
                                             InterruptedException
        Waits for the starting system change number to exist in the archive logs before returning.
        Parameters:
        context - the change event source context
        startScn - the starting system change number
        Returns:
        true if the code should continue, false if the code should end.
        Throws:
        SQLException - if a database exception occurred
        InterruptedException - if the pause between checks is interrupted

      • isStartScnInArchiveLogs

        private boolean isStartScnInArchiveLogs​(Scn startScn)
                                 throws SQLException
        Returns whether the starting system change number is in the archive logs.
        Parameters:
        startScn - the starting system change number
        Returns:
        true if the starting system change number is in the archive logs; false otherwise.
        Throws:
        SQLException - if a database exception occurred