io.debezium.connector.oracle.logminer

Class LogMinerHelper

java.lang.Object

io.debezium.connector.oracle.logminer.LogMinerHelper

public class LogMinerHelper
extends Object

This class contains methods to configure and manage Log Miner utility

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
private static class	LogMinerHelper.DATATYPE

Field Summary

Fields

Modifier and Type	Field and Description
private static org.slf4j.Logger	LOGGER
private static String	UNKNOWN

Constructor Summary

Constructors

Constructor and Description
LogMinerHelper()

Method Summary

Modifier and Type	Method and Description
(package private) static void	buildDataDictionary(Connection connection) This builds data dictionary objects in redo log files.
(package private) static void	checkSupplementalLogging(OracleConnection connection, String pdbName) This method checks if supplemental logging was set on the database level.
(package private) static void	createAuditTable(Connection connection)
(package private) static void	endMining(Connection connection) This call completes log miner session.
private static void	executeCallableStatement(Connection connection, String statement)
(package private) static Set<String>	getCurrentRedoLogFiles(Connection connection, LogMinerMetrics metrics) This method query the database to get CURRENT online redo log file(s).
static long	getCurrentScn(Connection connection) This method returns current SCN from the database
(package private) static long	getEndScn(Connection connection, long startScn, LogMinerMetrics metrics) This method returns next SCN for mining and also updates MBean metrics We use a configurable limit, because the larger mining range, the slower query from Log Miner content view.
(package private) static long	getFirstOnlineLogScn(Connection connection) This method fetches the oldest SCN from online redo log files
(package private) static Optional<Long>	getLastScnFromTheOldestOnlineRedo(Connection connection, Long offsetScn) This method returns SCN as a watermark to abandon long lasting transactions.
private static Map<String,Long>	getLogFilesForOffsetScn(Connection connection, Long offsetScn) This method returns all online log files, starting from one which contains offset SCN and ending with one containing largest SCN 18446744073709551615 on Ora 19c is the max value of the nextScn in the current redo todo replace all Long with BigInteger for SCN
private static Map<String,String>	getMap(Connection connection, String query, String nullReplacement)
private static int	getRedoLogGroupSize(Connection connection) get size of online REDO groups
private static Map<String,String>	getRedoLogStatus(Connection connection) This fetches online redo log statuses
private static Object	getSingleResult(Connection connection, String query, LogMinerHelper.DATATYPE type)
private static int	getSwitchCount(Connection connection) This fetches REDO LOG switch count for the last day
(package private) static Duration	getTimeDifference(Connection connection) Calculate time difference between database and connector timers.
(package private) static void	logError(TransactionalBufferMetrics metrics, String format, Object... args)
(package private) static void	logWarn(TransactionalBufferMetrics metrics, String format, Object... args)
(package private) static void	setNlsSessionParameters(io.debezium.jdbc.JdbcConnection connection) Sets NLS parameters for mining session.
(package private) static void	setRedoLogFilesForMining(Connection connection, Long lastProcessedScn) This method substitutes CONTINUOUS_MINE functionality for online files only
(package private) static void	startOnlineMining(Connection connection, Long startScn, Long endScn, OracleConnectorConfig.LogMiningStrategy strategy, boolean isContinuousMining) This method builds mining view to query changes from.
private static void	updateRedoLogMetrics(Connection connection, LogMinerMetrics metrics, Set<String> fileNames) This is to update MBean metrics associated with REDO LOG groups

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

UNKNOWN

private static final String UNKNOWN

See Also:

Constant Field Values

LOGGER

private static final org.slf4j.Logger LOGGER

Constructor Detail

LogMinerHelper

public LogMinerHelper()

Method Detail

buildDataDictionary

static void buildDataDictionary(Connection connection)
                         throws SQLException

This builds data dictionary objects in redo log files. During this build, Oracle does an additional REDO LOG switch. This call may take time, which leads to delay in delivering incremental changes. With this option the lag between source database and dispatching event fluctuates.

Parameters:

connection - connection to the database as log miner user (connection to the container)

Throws:

SQLException - any exception

getCurrentScn

public static long getCurrentScn(Connection connection)
                          throws SQLException

This method returns current SCN from the database

Parameters:

connection - container level database connection

Returns:

current SCN

Throws:

SQLException - if anything unexpected happens

createAuditTable

static void createAuditTable(Connection connection)
                      throws SQLException

Throws:

SQLException

getEndScn

static long getEndScn(Connection connection,
                      long startScn,
                      LogMinerMetrics metrics)
               throws SQLException

This method returns next SCN for mining and also updates MBean metrics We use a configurable limit, because the larger mining range, the slower query from Log Miner content view. In addition capturing unlimited number of changes can blow up Java heap. Gradual querying helps to catch up faster after long delays in mining.

Parameters:

connection - container level database connection

metrics - MBean accessible metrics

startScn - start SCN

Returns:

next SCN to mine to

Throws:

SQLException - if anything unexpected happens

getTimeDifference

static Duration getTimeDifference(Connection connection)
                           throws SQLException

Calculate time difference between database and connector timers. It could be negative if DB time is ahead.

Parameters:

connection - connection

Returns:

the time difference as a Duration

Throws:

SQLException

startOnlineMining

static void startOnlineMining(Connection connection,
                              Long startScn,
                              Long endScn,
                              OracleConnectorConfig.LogMiningStrategy strategy,
                              boolean isContinuousMining)
                       throws SQLException

This method builds mining view to query changes from. This view is built for online redo log files. It starts log mining session. It uses data dictionary objects, incorporated in previous steps. It tracks DDL changes and mines committed data only.

Parameters:

connection - container level database connection

startScn - the SCN to mine from

endScn - the SCN to mine to

strategy - this is about dictionary location

isContinuousMining - works < 19 version only

Throws:

SQLException - if anything unexpected happens

getCurrentRedoLogFiles

static Set<String> getCurrentRedoLogFiles(Connection connection,
                                          LogMinerMetrics metrics)
                                   throws SQLException

This method query the database to get CURRENT online redo log file(s). Multiple is applicable for RAC systems.

Parameters:

connection - connection to reuse

metrics - MBean accessible metrics

Returns:

full redo log file name(s), including path

Throws:

SQLException - if anything unexpected happens

getFirstOnlineLogScn

static long getFirstOnlineLogScn(Connection connection)
                          throws SQLException

This method fetches the oldest SCN from online redo log files

Parameters:

connection - container level database connection

Returns:

oldest SCN from online redo log

Throws:

SQLException - if anything unexpected happens

setNlsSessionParameters

static void setNlsSessionParameters(io.debezium.jdbc.JdbcConnection connection)
                             throws SQLException

Sets NLS parameters for mining session.

Parameters:

connection - session level database connection

Throws:

SQLException - if anything unexpected happens

updateRedoLogMetrics

private static void updateRedoLogMetrics(Connection connection,
                                         LogMinerMetrics metrics,
                                         Set<String> fileNames)

This is to update MBean metrics associated with REDO LOG groups

Parameters:

connection - connection

fileNames - name of current REDO LOG files

metrics - current metrics

getRedoLogStatus

private static Map<String,String> getRedoLogStatus(Connection connection)
                                            throws SQLException

This fetches online redo log statuses

Parameters:

connection - privileged connection

Returns:

REDO LOG statuses Map, where key is the REDO name and value is the status

Throws:

SQLException - if anything unexpected happens

getSwitchCount

private static int getSwitchCount(Connection connection)

This fetches REDO LOG switch count for the last day

Parameters:

connection - privileged connection

Returns:

counter

checkSupplementalLogging

static void checkSupplementalLogging(OracleConnection connection,
                                     String pdbName)
                              throws SQLException

This method checks if supplemental logging was set on the database level. This is critical check, cannot work if not.

Parameters:

connection - oracle connection on logminer level

pdbName - pdb name

Throws:

SQLException - if anything unexpected happens

endMining

static void endMining(Connection connection)

This call completes log miner session. Complete gracefully.

Parameters:

connection - container level database connection

setRedoLogFilesForMining

static void setRedoLogFilesForMining(Connection connection,
                                     Long lastProcessedScn)
                              throws SQLException

This method substitutes CONTINUOUS_MINE functionality for online files only

Parameters:

connection - connection

lastProcessedScn - current offset

Throws:

SQLException - if anything unexpected happens

getLastScnFromTheOldestOnlineRedo

static Optional<Long> getLastScnFromTheOldestOnlineRedo(Connection connection,
                                                        Long offsetScn)
                                                 throws SQLException

This method returns SCN as a watermark to abandon long lasting transactions.

Parameters:

connection - connection

offsetScn - current offset

Returns:

Optional last SCN in a redo log

Throws:

SQLException - if anything unexpected happens

logWarn

static void logWarn(TransactionalBufferMetrics metrics,
                    String format,
                    Object... args)

logError

static void logError(TransactionalBufferMetrics metrics,
                     String format,
                     Object... args)

getRedoLogGroupSize

private static int getRedoLogGroupSize(Connection connection)
                                throws SQLException

get size of online REDO groups

Parameters:

connection - connection

Returns:

size

Throws:

SQLException

getLogFilesForOffsetScn

private static Map<String,Long> getLogFilesForOffsetScn(Connection connection,
                                                        Long offsetScn)
                                                 throws SQLException

This method returns all online log files, starting from one which contains offset SCN and ending with one containing largest SCN 18446744073709551615 on Ora 19c is the max value of the nextScn in the current redo todo replace all Long with BigInteger for SCN

Throws:

SQLException

executeCallableStatement

private static void executeCallableStatement(Connection connection,
                                             String statement)
                                      throws SQLException

Throws:

SQLException

getMap

private static Map<String,String> getMap(Connection connection,
                                         String query,
                                         String nullReplacement)
                                  throws SQLException

Throws:

SQLException

getSingleResult

private static Object getSingleResult(Connection connection,
                                      String query,
                                      LogMinerHelper.DATATYPE type)
                               throws SQLException

Throws:

SQLException