io.debezium.connector.sqlserver

Class SqlServerConnection

java.lang.Object

io.debezium.jdbc.JdbcConnection

io.debezium.connector.sqlserver.SqlServerConnection

All Implemented Interfaces:

AutoCloseable

public class SqlServerConnection
extends JdbcConnection

JdbcConnection extension to be used with Microsoft SQL Server

Author:

Horia Chiorean (hchiorea@redhat.com), Jiri Pechanec

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	SqlServerConnection.CdcEnabledTable

Nested classes/interfaces inherited from class io.debezium.jdbc.JdbcConnection

JdbcConnection.BlockingMultiResultSetConsumer, JdbcConnection.BlockingResultSetConsumer, JdbcConnection.CallPreparer, JdbcConnection.ConnectionFactory, JdbcConnection.MultiResultSetConsumer, JdbcConnection.Operations, JdbcConnection.ParameterResultSetConsumer, JdbcConnection.ResultSetConsumer, JdbcConnection.ResultSetExtractor<T>, JdbcConnection.ResultSetMapper<T>, JdbcConnection.StatementFactory, JdbcConnection.StatementPreparer

Field Summary

Fields

Modifier and Type	Field and Description
private static Pattern	BRACKET_PATTERN
private static int	CHANGE_TABLE_DATA_COLUMN_OFFSET
private Clock	clock
private SqlServerDefaultValueConverter	defaultValueConverter
private static JdbcConnection.ConnectionFactory	FACTORY
private static String	GET_ALL_CHANGES_FOR_TABLE
private static String	GET_DATABASE_NAME
private static String	GET_LIST_OF_CDC_ENABLED_TABLES
private static String	GET_LIST_OF_KEY_COLUMNS
private static String	GET_LIST_OF_NEW_CDC_ENABLED_TABLES
private static String	GET_MAX_LSN
private static String	GET_MAX_LSN_SKIP_LOW_ACTIVTY
private static String	GET_MIN_LSN
private static String	INCREMENT_LSN
static String	INSTANCE_NAME
private static String	LOCK_TABLE
private static org.slf4j.Logger	LOGGER
private BoundedConcurrentHashMap<Lsn,Instant>	lsnToInstantCache
private String	lsnToTimestamp
private int	queryFetchSize
private String	realDatabaseName actual name of the database, which could differ in casing from the database name given in the connector config.
static String	SERVER_TIMEZONE_PROP_NAME
private SourceTimestampMode	sourceTimestampMode
private static String	SQL_SERVER_VERSION
private static String	STATEMENTS_PLACEHOLDER
private ZoneId	transactionTimezone
private static String	URL_PATTERN

Constructor Summary

Constructors

Constructor and Description
SqlServerConnection(Configuration config, Clock clock, SourceTimestampMode sourceTimestampMode, SqlServerValueConverters valueConverters) Creates a new connection using the supplied configuration.
SqlServerConnection(Configuration config, Clock clock, SourceTimestampMode sourceTimestampMode, SqlServerValueConverters valueConverters, Supplier<ClassLoader> classLoaderSupplier) Creates a new connection using the supplied configuration.

Method Summary

Modifier and Type	Method and Description
private String	cdcNameForTable(TableId tableId)
void	getChangesForTable(TableId tableId, Lsn fromLsn, Lsn toLsn, JdbcConnection.ResultSetConsumer consumer) Provides all changes recorded by the SQL Server CDC capture process for a given table.
void	getChangesForTables(SqlServerChangeTable[] changeTables, Lsn intervalFromLsn, Lsn intervalToLsn, JdbcConnection.BlockingMultiResultSetConsumer consumer) Provides all changes recorder by the SQL Server CDC capture process for a set of tables.
protected Optional<Object>	getDefaultValue(Column column, String defaultValue)
private Lsn	getFromLsn(SqlServerChangeTable changeTable, Lsn intervalFromLsn)
private static String	getLsnToTimestamp(boolean supportsAtTimeZone) Returns the query for obtaining the LSN-to-TIMESTAMP query.
Lsn	getMaxLsn()
MaxLsnResult	getMaxLsnResult(boolean skipLowActivityLsnsEnabled)
Lsn	getMinLsn(String changeTableName)
String	getNameOfChangeTable(String captureName)
String	getRealDatabaseName()
private Optional<Integer>	getSqlServerVersion()
Table	getTableSchemaFromChangeTable(SqlServerChangeTable changeTable)
Table	getTableSchemaFromTable(SqlServerChangeTable changeTable)
Lsn	incrementLsn(Lsn lsn) Obtain the next available position in the database log.
Set<SqlServerChangeTable>	listOfChangeTables()
Set<SqlServerChangeTable>	listOfNewChangeTables(Lsn fromLsn, Lsn toLsn)
void	lockTable(TableId tableId) Creates an exclusive lock for a given table.
private Instant	normalize(Timestamp timestamp)
private String	retrieveRealDatabaseName()
private ZoneId	retrieveTransactionTimezone(boolean supportsAtTimeZone)
private boolean	supportsAtTimeZone() SELECT ...
Instant	timestampOfLsn(Lsn lsn) Map a commit LSN to a point in time when the commit happened.

Methods inherited from class io.debezium.jdbc.JdbcConnection

call, close, commit, config, connect, connection, connection, connectionString, database, execute, execute, executeWithoutCommitting, isConnected, isNullable, parseSqlStatementString, patternBasedFactory, patternBasedFactory, prepareQuery, prepareQuery, prepareQuery, prepareQuery, prepareQueryAndMap, prepareQueryWithBlockingConsumer, prepareUpdate, print, print, query, query, queryAndMap, queryAndMap, querySingleValue, queryWithBlockingConsumer, readAllCatalogNames, readAllSchemaNames, readAllTableNames, readPrimaryKeyNames, readPrimaryKeyOrUniqueIndexNames, readSchema, readTableColumn, readTableNames, readTableUniqueIndices, resolveJdbcType, resolveNativeType, rollback, setAutoCommit, singleResultMapper, tableTypes, username

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

SERVER_TIMEZONE_PROP_NAME

public static final String SERVER_TIMEZONE_PROP_NAME

See Also:

Constant Field Values

INSTANCE_NAME

public static final String INSTANCE_NAME

See Also:

Constant Field Values

GET_DATABASE_NAME

private static final String GET_DATABASE_NAME

See Also:

Constant Field Values

LOGGER

private static final org.slf4j.Logger LOGGER

STATEMENTS_PLACEHOLDER

private static final String STATEMENTS_PLACEHOLDER

See Also:

Constant Field Values

GET_MAX_LSN

private static final String GET_MAX_LSN

See Also:

Constant Field Values

GET_MAX_LSN_SKIP_LOW_ACTIVTY

private static final String GET_MAX_LSN_SKIP_LOW_ACTIVTY

See Also:

Constant Field Values

GET_MIN_LSN

private static final String GET_MIN_LSN

See Also:

Constant Field Values

LOCK_TABLE

private static final String LOCK_TABLE

See Also:

Constant Field Values

SQL_SERVER_VERSION

private static final String SQL_SERVER_VERSION

See Also:

Constant Field Values

lsnToTimestamp

private final String lsnToTimestamp

INCREMENT_LSN

private static final String INCREMENT_LSN

See Also:

Constant Field Values

GET_ALL_CHANGES_FOR_TABLE

private static final String GET_ALL_CHANGES_FOR_TABLE

See Also:

Constant Field Values

GET_LIST_OF_CDC_ENABLED_TABLES

private static final String GET_LIST_OF_CDC_ENABLED_TABLES

See Also:

Constant Field Values

GET_LIST_OF_NEW_CDC_ENABLED_TABLES

private static final String GET_LIST_OF_NEW_CDC_ENABLED_TABLES

See Also:

Constant Field Values

GET_LIST_OF_KEY_COLUMNS

private static final String GET_LIST_OF_KEY_COLUMNS

See Also:

Constant Field Values

BRACKET_PATTERN

private static final Pattern BRACKET_PATTERN

CHANGE_TABLE_DATA_COLUMN_OFFSET

private static final int CHANGE_TABLE_DATA_COLUMN_OFFSET

See Also:

Constant Field Values

URL_PATTERN

private static final String URL_PATTERN

FACTORY

private static final JdbcConnection.ConnectionFactory FACTORY

realDatabaseName

private final String realDatabaseName

actual name of the database, which could differ in casing from the database name given in the connector config.

transactionTimezone

private final ZoneId transactionTimezone

sourceTimestampMode

private final SourceTimestampMode sourceTimestampMode

clock

private final Clock clock

queryFetchSize

private final int queryFetchSize

lsnToInstantCache

private final BoundedConcurrentHashMap<Lsn,Instant> lsnToInstantCache

defaultValueConverter

private final SqlServerDefaultValueConverter defaultValueConverter

Constructor Detail

SqlServerConnection

public SqlServerConnection(Configuration config,
                           Clock clock,
                           SourceTimestampMode sourceTimestampMode,
                           SqlServerValueConverters valueConverters)

Creates a new connection using the supplied configuration.

Parameters:

config - Configuration instance, may not be null.

clock - the clock

sourceTimestampMode - strategy for populating source.ts_ms.

valueConverters - SqlServerValueConverters instance

SqlServerConnection

public SqlServerConnection(Configuration config,
                           Clock clock,
                           SourceTimestampMode sourceTimestampMode,
                           SqlServerValueConverters valueConverters,
                           Supplier<ClassLoader> classLoaderSupplier)

Creates a new connection using the supplied configuration.

Parameters:

config - Configuration instance, may not be null.

clock - the clock

sourceTimestampMode - strategy for populating source.ts_ms.

valueConverters - SqlServerValueConverters instance

classLoaderSupplier - class loader supplier

Method Detail

getLsnToTimestamp

private static String getLsnToTimestamp(boolean supportsAtTimeZone)

Returns the query for obtaining the LSN-to-TIMESTAMP query. On SQL Server 2016 and newer, the query will normalize the value to UTC. This means that the SERVER_TIMEZONE_PROP_NAME is not necessary to be given. The returned TIMESTAMP will be adjusted by the JDBC driver using this VM's TZ (as required by the JDBC spec), and that same TZ will be applied when converting the TIMESTAMP value into an Instant.

getMaxLsn

public Lsn getMaxLsn()
              throws SQLException

Returns:

the current largest log sequence number

Throws:

SQLException

getMaxLsnResult

public MaxLsnResult getMaxLsnResult(boolean skipLowActivityLsnsEnabled)
                             throws SQLException

Throws:

SQLException

getMinLsn

public Lsn getMinLsn(String changeTableName)
              throws SQLException

Returns:

the smallest log sequence number of table

Throws:

SQLException

getChangesForTable

public void getChangesForTable(TableId tableId,
                               Lsn fromLsn,
                               Lsn toLsn,
                               JdbcConnection.ResultSetConsumer consumer)
                        throws SQLException

Provides all changes recorded by the SQL Server CDC capture process for a given table.

Parameters:

tableId - - the requested table changes

fromLsn - - closed lower bound of interval of changes to be provided

toLsn - - closed upper bound of interval of changes to be provided

consumer - - the change processor

Throws:

SQLException

getChangesForTables

public void getChangesForTables(SqlServerChangeTable[] changeTables,
                                Lsn intervalFromLsn,
                                Lsn intervalToLsn,
                                JdbcConnection.BlockingMultiResultSetConsumer consumer)
                         throws SQLException,
                                InterruptedException

Provides all changes recorder by the SQL Server CDC capture process for a set of tables.

Parameters:

changeTables - - the requested tables to obtain changes for

intervalFromLsn - - closed lower bound of interval of changes to be provided

intervalToLsn - - closed upper bound of interval of changes to be provided

consumer - - the change processor

Throws:

SQLException

InterruptedException

getFromLsn

private Lsn getFromLsn(SqlServerChangeTable changeTable,
                       Lsn intervalFromLsn)
                throws SQLException

Throws:

SQLException

incrementLsn

public Lsn incrementLsn(Lsn lsn)
                 throws SQLException

Obtain the next available position in the database log.

Parameters:

lsn - - LSN of the current position

Returns:

LSN of the next position in the database

Throws:

SQLException

timestampOfLsn

public Instant timestampOfLsn(Lsn lsn)
                       throws SQLException

Map a commit LSN to a point in time when the commit happened.

Parameters:

lsn - - LSN of the commit

Returns:

time when the commit was recorded into the database log or the current time, depending on the setting for the "source timestamp mode" option

Throws:

SQLException

normalize

private Instant normalize(Timestamp timestamp)

lockTable

public void lockTable(TableId tableId)
               throws SQLException

Creates an exclusive lock for a given table.

Parameters:

tableId - to be locked

Throws:

SQLException

cdcNameForTable

private String cdcNameForTable(TableId tableId)

listOfChangeTables

public Set<SqlServerChangeTable> listOfChangeTables()
                                             throws SQLException

Throws:

SQLException

listOfNewChangeTables

public Set<SqlServerChangeTable> listOfNewChangeTables(Lsn fromLsn,
                                                       Lsn toLsn)
                                                throws SQLException

Throws:

SQLException

getTableSchemaFromTable

public Table getTableSchemaFromTable(SqlServerChangeTable changeTable)
                              throws SQLException

Throws:

SQLException

getTableSchemaFromChangeTable

public Table getTableSchemaFromChangeTable(SqlServerChangeTable changeTable)
                                    throws SQLException

Throws:

SQLException

getNameOfChangeTable

public String getNameOfChangeTable(String captureName)

getRealDatabaseName

public String getRealDatabaseName()

retrieveTransactionTimezone

private ZoneId retrieveTransactionTimezone(boolean supportsAtTimeZone)

retrieveRealDatabaseName

private String retrieveRealDatabaseName()

supportsAtTimeZone

private boolean supportsAtTimeZone()

SELECT ... AT TIME ZONE only works on SQL Server 2016 and newer.

getSqlServerVersion

private Optional<Integer> getSqlServerVersion()

getDefaultValue

protected Optional<Object> getDefaultValue(Column column,
                                           String defaultValue)

Overrides:

getDefaultValue in class JdbcConnection