public class SqlServerStreamingChangeEventSource extends Object implements StreamingChangeEventSource
A StreamingChangeEventSource based on SQL Server change data capture functionality.
A main loop polls database DDL change and change data tables and turns them into change events.
The connector uses CDC functionality of SQL Server that is implemented as as a process that monitors source table and write changes from the table into the change table.
The main loop keeps a pointer to the LSN of changes that were already processed. It queries all change tables and get result set of changes. It always finds the smallest LSN across all tables and the change is converted into the event message and sent downstream. The process repeats until all result sets are empty. The LSN is marked and the procedure repeats.
The schema changes detection follows the procedure recommended by SQL Server CDC documentation. The database operator should create one more capture process (and table) when a table schema is updated. The code detects presence of two change tables for a single source table. It decides which table is the new one depending on LSNs stored in them. The loop streams changes from the older table till there are events in new table with the LSN larger than in the old one. Then the change table is switched and streaming is executed from the new one.
| Modifier and Type | Class and Description |
|---|---|
private static class |
SqlServerStreamingChangeEventSource.ChangeTablePointer
The logical representation of a position for the change in the transaction log.
|
ChangeEventSource.ChangeEventSourceContext| Modifier and Type | Field and Description |
|---|---|
private Clock |
clock |
private static int |
COL_COMMIT_LSN |
private static int |
COL_DATA |
private static int |
COL_OPERATION |
private static int |
COL_ROW_LSN |
private SqlServerConnectorConfig |
connectorConfig |
private SqlServerConnection |
dataConnection
Connection used for reading CDC tables.
|
private EventDispatcher<TableId> |
dispatcher |
private ErrorHandler |
errorHandler |
private static org.slf4j.Logger |
LOGGER |
private SqlServerConnection |
metadataConnection
A separate connection for retrieving timestamps; without it, adaptive
buffering will not work.
|
private static Pattern |
MISSING_CDC_FUNCTION_CHANGES_ERROR |
private SqlServerOffsetContext |
offsetContext |
private Duration |
pollInterval |
private SqlServerDatabaseSchema |
schema |
| Constructor and Description |
|---|
SqlServerStreamingChangeEventSource(SqlServerConnectorConfig connectorConfig,
SqlServerOffsetContext offsetContext,
SqlServerConnection dataConnection,
SqlServerConnection metadataConnection,
EventDispatcher<TableId> dispatcher,
ErrorHandler errorHandler,
Clock clock,
SqlServerDatabaseSchema schema) |
| Modifier and Type | Method and Description |
|---|---|
void |
execute(ChangeEventSource.ChangeEventSourceContext context) |
private ChangeTable[] |
getCdcTablesToQuery() |
private void |
migrateTable(Queue<ChangeTable> schemaChangeCheckpoints) |
private ChangeTable[] |
processErrorFromChangeTableQuery(SQLException exception,
ChangeTable[] currentChangeTables) |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, waitcommitOffsetprivate static final int COL_COMMIT_LSN
private static final int COL_ROW_LSN
private static final int COL_OPERATION
private static final int COL_DATA
private static final Pattern MISSING_CDC_FUNCTION_CHANGES_ERROR
private static final org.slf4j.Logger LOGGER
private final SqlServerConnection dataConnection
private final SqlServerConnection metadataConnection
https://docs.microsoft.com/en-us/sql/connect/jdbc/using-adaptive-buffering?view=sql-server-2017#guidelines-for-using-adaptive-bufferingprivate final EventDispatcher<TableId> dispatcher
private final ErrorHandler errorHandler
private final Clock clock
private final SqlServerDatabaseSchema schema
private final SqlServerOffsetContext offsetContext
private final Duration pollInterval
private final SqlServerConnectorConfig connectorConfig
public SqlServerStreamingChangeEventSource(SqlServerConnectorConfig connectorConfig, SqlServerOffsetContext offsetContext, SqlServerConnection dataConnection, SqlServerConnection metadataConnection, EventDispatcher<TableId> dispatcher, ErrorHandler errorHandler, Clock clock, SqlServerDatabaseSchema schema)
public void execute(ChangeEventSource.ChangeEventSourceContext context) throws InterruptedException
execute in interface StreamingChangeEventSourceInterruptedExceptionprivate void migrateTable(Queue<ChangeTable> schemaChangeCheckpoints) throws InterruptedException, SQLException
InterruptedExceptionSQLExceptionprivate ChangeTable[] processErrorFromChangeTableQuery(SQLException exception, ChangeTable[] currentChangeTables) throws Exception
Exceptionprivate ChangeTable[] getCdcTablesToQuery() throws SQLException, InterruptedException
SQLExceptionInterruptedExceptionCopyright © 2020 JBoss by Red Hat. All rights reserved.