Package io.debezium.connector.sqlserver

Class SqlServerConnection

java.lang.Object

io.debezium.jdbc.JdbcConnection

io.debezium.connector.sqlserver.SqlServerConnection

All Implemented Interfaces:

AutoCloseable

public class SqlServerConnection
extends JdbcConnection

JdbcConnection extension to be used with Microsoft SQL Server

Author:

Horia Chiorean (hchiorea@redhat.com), Jiri Pechanec

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	SqlServerConnection.CdcEnabledTable

Nested classes/interfaces inherited from class io.debezium.jdbc.JdbcConnection

JdbcConnection.BlockingMultiResultSetConsumer, JdbcConnection.BlockingResultSetConsumer, JdbcConnection.CallPreparer, JdbcConnection.ConnectionFactory, JdbcConnection.MultiResultSetConsumer, JdbcConnection.Operations, JdbcConnection.ParameterResultSetConsumer, JdbcConnection.ResultSetConsumer, JdbcConnection.ResultSetExtractor<T extends Object>, JdbcConnection.ResultSetMapper<T extends Object>, JdbcConnection.StatementFactory, JdbcConnection.StatementPreparer

Field Summary

Fields

Modifier and Type	Field	Description
private static String	CLOSING_QUOTING_CHARACTER
private static String	DATABASE_NAME_PLACEHOLDER
private SqlServerDefaultValueConverter	defaultValueConverter
private String	get_all_changes_for_table
private static String	GET_ALL_CHANGES_FOR_TABLE
private static String	GET_CAPTURED_COLUMNS	Queries the list of captured column names and their change table identifiers in the given database.
private static String	GET_CHANGE_TABLES	Queries the list of capture instances in the given database.
private static String	GET_DATABASE_NAME
private static String	GET_MAX_LSN
private static String	GET_MAX_TRANSACTION_LSN
private static String	GET_MIN_LSN
private static String	GET_NEW_CHANGE_TABLES
private static String	GET_NTH_TRANSACTION_LSN_FROM_BEGINNING
private static String	GET_NTH_TRANSACTION_LSN_FROM_LAST
private String	getAllChangesForTable
private static String	INCREMENT_LSN
static String	INSTANCE_NAME
private static String	LOCK_TABLE
private static org.slf4j.Logger	LOGGER
protected static String	LSN_TIMESTAMP_SELECT_STATEMENT
private boolean	multiPartitionMode
private static String	OPENING_QUOTING_CHARACTER
private boolean	optionRecompile
private int	queryFetchSize
static String	SERVER_TIMEZONE_PROP_NAME	Deprecated. The connector will determine the database server timezone offset automatically.
private static String	STATEMENTS_PLACEHOLDER
private static String	URL_PATTERN

Constructor Summary

Constructors

Constructor	Description
SqlServerConnection(Configuration config, SourceTimestampMode sourceTimestampMode, SqlServerValueConverters valueConverters, Supplier<ClassLoader> classLoaderSupplier, Set<Envelope.Operation> skippedOperations, boolean multiPartitionMode)	Creates a new connection using the supplied configuration.
SqlServerConnection(Configuration config, SourceTimestampMode sourceTimestampMode, SqlServerValueConverters valueConverters, Supplier<ClassLoader> classLoaderSupplier, Set<Envelope.Operation> skippedOperations, boolean multiPartitionMode, boolean optionRecompile)	Creates a new connection using the supplied configuration.

Method Summary

Modifier and Type	Method	Description
String	buildSelectWithRowLimits(TableId tableId, int limit, String projection, Optional<String> condition, String orderBy)
private String	cdcNameForTable(TableId tableId)
Connection	connection(boolean executeOnConnect)
String	connectionString()	Returns a JDBC connection string for the current configuration.
private static JdbcConnection.ConnectionFactory	createConnectionFactory(boolean multiPartitionMode)
private static String	createUrlPattern(boolean multiPartitionMode)
void	getChangesForTables(String databaseName, SqlServerChangeTable[] changeTables, Lsn intervalFromLsn, Lsn intervalToLsn, JdbcConnection.BlockingMultiResultSetConsumer consumer)	Provides all changes recorder by the SQL Server CDC capture process for a set of tables.
List<SqlServerChangeTable>	getChangeTables(String databaseName)
List<SqlServerChangeTable>	getChangeTables(String databaseName, Lsn toLsn)
<T extends DatabaseSchema<TableId>> Object	getColumnValue(ResultSet rs, int columnIndex, Column column, Table table, T schema)
SqlServerDefaultValueConverter	getDefaultValueConverter()
private Lsn	getFromLsn(String databaseName, SqlServerChangeTable changeTable, Lsn intervalFromLsn)
Lsn	getMaxLsn(String databaseName)
Lsn	getMaxTransactionLsn(String databaseName)
Lsn	getMinLsn(String databaseName, String changeTableName)
String	getNameOfChangeTable(String captureName)
List<SqlServerChangeTable>	getNewChangeTables(String databaseName, Lsn fromLsn, Lsn toLsn)
Lsn	getNthTransactionLsnFromBeginning(String databaseName, int maxOffset)
Lsn	getNthTransactionLsnFromLast(String databaseName, Lsn lastLsn, int maxOffset)
Table	getTableSchemaFromTable(String databaseName, SqlServerChangeTable changeTable)
Lsn	incrementLsn(String databaseName, Lsn lsn)	Obtain the next available position in the database log.
protected boolean	isTableUniqueIndexIncluded(String indexName, String columnName)
void	lockTable(TableId tableId)	Creates an exclusive lock for a given table.
String	quotedTableIdString(TableId tableId)
private String	replaceDatabaseNamePlaceholder(String sql, String databaseName)
String	retrieveRealDatabaseName(String databaseName)	Retrieve the name of the database in the original case as it's defined on the server.

Methods inherited from class io.debezium.jdbc.JdbcConnection

call, close, commit, config, connect, connection, connectionString, database, execute, execute, executeWithoutCommitting, isConnected, isNullable, isValid, overrideColumn, parseSqlStatementString, patternBasedFactory, patternBasedFactory, prepareQuery, prepareQuery, prepareQuery, prepareQuery, prepareQuery, prepareQueryAndMap, prepareQueryWithBlockingConsumer, prepareUpdate, print, print, query, query, queryAndMap, queryAndMap, querySingleValue, queryWithBlockingConsumer, quotedColumnIdString, readAllCatalogNames, readAllSchemaNames, readAllTableNames, readPrimaryKeyNames, readPrimaryKeyOrUniqueIndexNames, readSchema, readTableColumn, readTableNames, readTablePreparedStatement, readTableStatement, readTableUniqueIndices, resolveCatalogName, resolveJdbcType, resolveNativeType, rollback, rowToArray, setAutoCommit, singleResultMapper, tableTypes, username

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

SERVER_TIMEZONE_PROP_NAME

@Deprecated
public static final String SERVER_TIMEZONE_PROP_NAME

Deprecated.

The connector will determine the database server timezone offset automatically.

See Also:

Constant Field Values

INSTANCE_NAME

public static final String INSTANCE_NAME

See Also:

Constant Field Values

GET_DATABASE_NAME

private static final String GET_DATABASE_NAME

See Also:

Constant Field Values

LOGGER

private static final org.slf4j.Logger LOGGER

STATEMENTS_PLACEHOLDER

private static final String STATEMENTS_PLACEHOLDER

See Also:

Constant Field Values

DATABASE_NAME_PLACEHOLDER

private static final String DATABASE_NAME_PLACEHOLDER

See Also:

Constant Field Values

GET_MAX_LSN

private static final String GET_MAX_LSN

See Also:

Constant Field Values

GET_MAX_TRANSACTION_LSN

private static final String GET_MAX_TRANSACTION_LSN

See Also:

Constant Field Values

GET_NTH_TRANSACTION_LSN_FROM_BEGINNING

private static final String GET_NTH_TRANSACTION_LSN_FROM_BEGINNING

See Also:

Constant Field Values

GET_NTH_TRANSACTION_LSN_FROM_LAST

private static final String GET_NTH_TRANSACTION_LSN_FROM_LAST

See Also:

Constant Field Values

GET_MIN_LSN

private static final String GET_MIN_LSN

See Also:

Constant Field Values

LOCK_TABLE

private static final String LOCK_TABLE

See Also:

Constant Field Values

INCREMENT_LSN

private static final String INCREMENT_LSN

See Also:

Constant Field Values

GET_ALL_CHANGES_FOR_TABLE

private static final String GET_ALL_CHANGES_FOR_TABLE

See Also:

Constant Field Values

get_all_changes_for_table

private final String get_all_changes_for_table

LSN_TIMESTAMP_SELECT_STATEMENT

protected static final String LSN_TIMESTAMP_SELECT_STATEMENT

See Also:

Constant Field Values

GET_CAPTURED_COLUMNS

private static final String GET_CAPTURED_COLUMNS

Queries the list of captured column names and their change table identifiers in the given database.

See Also:

Constant Field Values

GET_CHANGE_TABLES

private static final String GET_CHANGE_TABLES

Queries the list of capture instances in the given database. If two or more capture instances with the same start LSN are available for a given source table, only the newest one will be returned. We use a query instead of sys.sp_cdc_help_change_data_capture because: 1. The stored procedure doesn't allow filtering capture instances by start LSN. 2. There is no way to use the result returned by a stored procedure in a query.

See Also:

Constant Field Values

GET_NEW_CHANGE_TABLES

private static final String GET_NEW_CHANGE_TABLES

See Also:

Constant Field Values

OPENING_QUOTING_CHARACTER

private static final String OPENING_QUOTING_CHARACTER

See Also:

Constant Field Values

CLOSING_QUOTING_CHARACTER

private static final String CLOSING_QUOTING_CHARACTER

See Also:

Constant Field Values

URL_PATTERN

private static final String URL_PATTERN

multiPartitionMode

private final boolean multiPartitionMode

getAllChangesForTable

private final String getAllChangesForTable

queryFetchSize

private final int queryFetchSize

defaultValueConverter

private final SqlServerDefaultValueConverter defaultValueConverter

optionRecompile

private boolean optionRecompile

Constructor Detail

SqlServerConnection

public SqlServerConnection(Configuration config,
                           SourceTimestampMode sourceTimestampMode,
                           SqlServerValueConverters valueConverters,
                           Supplier<ClassLoader> classLoaderSupplier,
                           Set<Envelope.Operation> skippedOperations,
                           boolean multiPartitionMode)

Creates a new connection using the supplied configuration.

Parameters:

config - Configuration instance, may not be null.

sourceTimestampMode - strategy for populating source.ts_ms.

valueConverters - SqlServerValueConverters instance

classLoaderSupplier - class loader supplier

skippedOperations - a set of Envelope.Operation to skip in streaming

SqlServerConnection

public SqlServerConnection(Configuration config,
                           SourceTimestampMode sourceTimestampMode,
                           SqlServerValueConverters valueConverters,
                           Supplier<ClassLoader> classLoaderSupplier,
                           Set<Envelope.Operation> skippedOperations,
                           boolean multiPartitionMode,
                           boolean optionRecompile)

Creates a new connection using the supplied configuration.

Parameters:

config - Configuration instance, may not be null.

sourceTimestampMode - strategy for populating source.ts_ms.

valueConverters - SqlServerValueConverters instance

classLoaderSupplier - class loader supplier

skippedOperations - a set of Envelope.Operation to skip in streaming

optionRecompile - Includes query option RECOMPILE on incremental snapshots

Method Detail

createUrlPattern

private static String createUrlPattern(boolean multiPartitionMode)

createConnectionFactory

private static JdbcConnection.ConnectionFactory createConnectionFactory(boolean multiPartitionMode)

connectionString

public String connectionString()

Returns a JDBC connection string for the current configuration.

Returns:

a String where the variables in urlPattern are replaced with values from the configuration

connection

public Connection connection(boolean executeOnConnect)
                      throws SQLException

Overrides:

connection in class JdbcConnection

Throws:

SQLException

getMaxLsn

public Lsn getMaxLsn(String databaseName)
              throws SQLException

Returns:

the current largest log sequence number

Throws:

SQLException

getNthTransactionLsnFromBeginning

public Lsn getNthTransactionLsnFromBeginning(String databaseName,
                                             int maxOffset)
                                      throws SQLException

Returns:

the log sequence number of the most recent transaction that isn't further than maxOffset from the beginning.

Throws:

SQLException

getNthTransactionLsnFromLast

public Lsn getNthTransactionLsnFromLast(String databaseName,
                                        Lsn lastLsn,
                                        int maxOffset)
                                 throws SQLException

Returns:

the log sequence number of the most recent transaction that isn't further than maxOffset from lastLsn.

Throws:

SQLException

getMaxTransactionLsn

public Lsn getMaxTransactionLsn(String databaseName)
                         throws SQLException

Returns:

the log sequence number of the most recent transaction.

Throws:

SQLException

getMinLsn

public Lsn getMinLsn(String databaseName,
                     String changeTableName)
              throws SQLException

Returns:

the smallest log sequence number of table

Throws:

SQLException

getChangesForTables

public void getChangesForTables(String databaseName,
                                SqlServerChangeTable[] changeTables,
                                Lsn intervalFromLsn,
                                Lsn intervalToLsn,
                                JdbcConnection.BlockingMultiResultSetConsumer consumer)
                         throws SQLException,
                                InterruptedException

Provides all changes recorder by the SQL Server CDC capture process for a set of tables.

Parameters:

databaseName - - the name of the database to query

changeTables - - the requested tables to obtain changes for

intervalFromLsn - - closed lower bound of interval of changes to be provided

intervalToLsn - - closed upper bound of interval of changes to be provided

consumer - - the change processor

Throws:

SQLException

InterruptedException

getFromLsn

private Lsn getFromLsn(String databaseName,
                       SqlServerChangeTable changeTable,
                       Lsn intervalFromLsn)
                throws SQLException

Throws:

SQLException

incrementLsn

public Lsn incrementLsn(String databaseName,
                        Lsn lsn)
                 throws SQLException

Obtain the next available position in the database log.

Parameters:

databaseName - - the name of the database that the LSN belongs to

lsn - - LSN of the current position

Returns:

LSN of the next position in the database

Throws:

SQLException

lockTable

public void lockTable(TableId tableId)
               throws SQLException

Creates an exclusive lock for a given table.

Parameters:

tableId - to be locked

Throws:

SQLException

cdcNameForTable

private String cdcNameForTable(TableId tableId)

getChangeTables

public List<SqlServerChangeTable> getChangeTables(String databaseName)
                                           throws SQLException

Throws:

SQLException

getChangeTables

public List<SqlServerChangeTable> getChangeTables(String databaseName,
                                                  Lsn toLsn)
                                           throws SQLException

Throws:

SQLException

getNewChangeTables

public List<SqlServerChangeTable> getNewChangeTables(String databaseName,
                                                     Lsn fromLsn,
                                                     Lsn toLsn)
                                              throws SQLException

Throws:

SQLException

getTableSchemaFromTable

public Table getTableSchemaFromTable(String databaseName,
                                     SqlServerChangeTable changeTable)
                              throws SQLException

Throws:

SQLException

getNameOfChangeTable

public String getNameOfChangeTable(String captureName)

retrieveRealDatabaseName

public String retrieveRealDatabaseName(String databaseName)

Retrieve the name of the database in the original case as it's defined on the server. Although SQL Server supports case-insensitive collations, the connector uses the database name to build the produced records' source info and, subsequently, the keys of its committed offset messages. This value must remain the same during the lifetime of the connector regardless of the case used in the connector configuration.

isTableUniqueIndexIncluded

protected boolean isTableUniqueIndexIncluded(String indexName,
                                             String columnName)

Overrides:

isTableUniqueIndexIncluded in class JdbcConnection

getColumnValue

public <T extends DatabaseSchema<TableId>> Object getColumnValue(ResultSet rs,
                                                                 int columnIndex,
                                                                 Column column,
                                                                 Table table,
                                                                 T schema)
                                                          throws SQLException

Overrides:

getColumnValue in class JdbcConnection

Throws:

SQLException

buildSelectWithRowLimits

public String buildSelectWithRowLimits(TableId tableId,
                                       int limit,
                                       String projection,
                                       Optional<String> condition,
                                       String orderBy)

Overrides:

buildSelectWithRowLimits in class JdbcConnection

quotedTableIdString

public String quotedTableIdString(TableId tableId)

Overrides:

quotedTableIdString in class JdbcConnection

replaceDatabaseNamePlaceholder

private String replaceDatabaseNamePlaceholder(String sql,
                                              String databaseName)

getDefaultValueConverter

public SqlServerDefaultValueConverter getDefaultValueConverter()