Package io.debezium.connector.mysql
Class MySqlConnectorIT
- java.lang.Object
-
- io.debezium.embedded.AbstractConnectorTest
-
- io.debezium.connector.mysql.MySqlConnectorIT
-
- All Implemented Interfaces:
Testing
public class MySqlConnectorIT extends AbstractConnectorTest
- Author:
- Randall Hauch
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description protected static classMySqlConnectorIT.BinlogPosition-
Nested classes/interfaces inherited from class io.debezium.embedded.AbstractConnectorTest
AbstractConnectorTest.SourceRecords
-
Nested classes/interfaces inherited from interface io.debezium.util.Testing
Testing.Debug, Testing.Files, Testing.InterruptableFunction, Testing.Network, Testing.Print, Testing.Timer
-
-
Field Summary
Fields Modifier and Type Field Description private Configurationconfigprivate UniqueDatabaseDATABASEprivate static PathDB_HISTORY_PATHprivate static intINITIAL_EVENT_COUNTprivate static intORDERS_TABLE_EVENT_COUNTprivate static intPRODUCTS_TABLE_EVENT_COUNTprivate UniqueDatabaseRO_DATABASE-
Fields inherited from class io.debezium.embedded.AbstractConnectorTest
engine, logger, logTestName, OFFSET_STORE_PATH, pollTimeoutInMs, skipTestRule
-
-
Constructor Summary
Constructors Constructor Description MySqlConnectorIT()
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description voidafterEach()voidbeforeEach()private voiddropDatabases()private org.apache.kafka.connect.data.StructgetAfter(org.apache.kafka.connect.source.SourceRecord record)private Optional<org.apache.kafka.connect.header.Header>getHeaderField(org.apache.kafka.connect.source.SourceRecord record, String fieldName)private Optional<org.apache.kafka.connect.header.Header>getPKUpdateNewKeyHeader(org.apache.kafka.connect.source.SourceRecord record)private Optional<org.apache.kafka.connect.header.Header>getPKUpdateOldKeyHeader(org.apache.kafka.connect.source.SourceRecord record)voidparseDeleteQuery()This test case validates that if you enable MySQL option binlog_rows_query_log_events, then the original SQL statement for a DELETE over a single row is parsed into the resulting event.voidparseMultipleInsertStatements()This test case validates that if you enable MySQL option binlog_rows_query_log_events, then the issue multiple INSERTs, the appropriate SQL statements are parsed into the resulting events.voidparseMultipleRowInsertStatement()This test case validates that if you enable MySQL option binlog_rows_query_log_events, then the issue single multi-row INSERT, the appropriate SQL statements are parsed into the resulting events.voidparseMultiRowDeleteQuery()This test case validates that if you enable MySQL option binlog_rows_query_log_events, then issue a multi-row DELETE, the resulting events get the original SQL statement.voidparseMultiRowUpdateQuery()This test case validates that if you enable MySQL option binlog_rows_query_log_events, then the original SQL statement for an UPDATE over a single row is parsed into the resulting event.voidparseUpdateQuery()This test case validates that if you enable MySQL option binlog_rows_query_log_events, then the original SQL statement for an UPDATE over a single row is parsed into the resulting event.private List<org.apache.kafka.connect.source.SourceRecord>recordsForTopicForRoProductsTable(AbstractConnectorTest.SourceRecords records)voidshouldConsumeAllEventsFromDatabaseUsingSnapshot()private voidshouldConsumeAllEventsFromDatabaseUsingSnapshotByField(Field dbIncludeListField, int serverId)voidshouldConsumeAllEventsFromDatabaseUsingSnapshotOld()voidshouldConsumeEventsWithIncludedColumns()voidshouldConsumeEventsWithMaskedAndBlacklistedColumns()voidshouldConsumeEventsWithMaskedHashedColumns()voidshouldConsumeEventsWithNoSnapshot()voidshouldConsumeEventsWithTruncatedColumns()voidshouldEmitHeadersOnPrimaryKeyUpdate()voidshouldEmitNoEventsForSkippedCreateOperations()voidshouldEmitNoEventsForSkippedUpdateAndDeleteOperations()voidshouldEmitNoSavepoints()voidshouldEmitNoTombstoneOnDelete()voidshouldEmitTombstoneOnDeleteByDefault()voidshouldFailToValidateAdaptivePrecisionMode()Specifying the adaptive time.precision.mode is no longer valid and a configuration validation problem should be reported when that configuration option is used.voidshouldFailToValidateInvalidConfiguration()voidshouldHandleIncludeListTables()voidshouldHandleWhitelistedTables()voidshouldIgnoreAlterTableForNonCapturedTablesNotStoredInHistory()voidshouldIgnoreAlterTableForNonCapturedTablesStoredInHistory()voidshouldIgnoreCreateIndexForNonCapturedTablesNotStoredInHistory()voidshouldNotParseQueryIfConnectorNotConfiguredTo()This test case validates that if you enable MySQL option binlog_rows_query_log_events, but configure the connector to NOT include the query, it will not be included in the event.voidshouldNotParseQueryIfServerOptionDisabled()This test case validates that if you disable MySQL option binlog_rows_query_log_events, then the original SQL statement for an INSERT statement is NOT parsed into the resulting event.voidshouldNotStartWithInvalidConfiguration()Verifies that the connector doesn't run with an invalid configuration.voidshouldOutputRecordsInCloudEventsFormat()voidshouldParseQueryIfAvailableAndConnectorOptionEnabled()This test case validates that if you enable MySQL option binlog_rows_query_log_events, then the original SQL statement for an INSERT statement is parsed into the resulting event.voidshouldProcessCreateUniqueIndex()voidshouldReceiveSchemaForNonWhitelistedTablesAndDatabases()voidshouldRewriteIdentityKey()voidshouldRewriteIdentityKeyWithMsgKeyColumnsFieldRegexValidation()voidshouldRewriteIdentityKeyWithWhitespace()voidshouldSaveSetCharacterSetWhenStoringOnlyCapturededTables()voidshouldUseMultipleOverriddenSelectStatementsDuringSnapshotting()voidshouldUseOverriddenSelectStatementDuringSnapshotting()voidshouldValidateAcceptableConfiguration()voidshouldValidateLockingModeNoneWithValidSnapshotModeConfiguration()Validates that SNAPSHOT_LOCKING_MODE 'none' is valid with all snapshot modesvoidshouldValidateValidConfigurationWithSSL()voidtestEmptySchemaLogWarningWithDatabaseWhitelist()voidtestEmptySchemaWarningWithTableWhitelist()voidtestNoEmptySchemaLogWarningWithDatabaseWhitelist()voidtestNoEmptySchemaWarningWithTableWhitelist()private voidwaitForStreamingRunning(String serverName)-
Methods inherited from class io.debezium.embedded.AbstractConnectorTest
assertBeginTransaction, assertConfigurationErrors, assertConfigurationErrors, assertConfigurationErrors, assertConnectorIsRunning, assertConnectorNotRunning, assertDelete, assertEndTransaction, assertEngineIsRunning, assertHasNoSourceQuery, assertInsert, assertKey, assertNoConfigurationErrors, assertNoRecordsToConsume, assertOffset, assertOffset, assertOnlyTransactionRecordsToConsume, assertRecordTransactionMetadata, assertSchemaMatchesStruct, assertSchemaMatchesStruct, assertSourceQuery, assertTombstone, assertTombstone, assertUpdate, assertValueField, configValue, consumeAvailableRecords, consumeDmlRecordsByTopic, consumeDmlRecordsByTopic, consumeDmlRecordsByTopic, consumeRecord, consumeRecords, consumeRecords, consumeRecords, consumeRecordsByTopic, consumeRecordsByTopic, consumeRecordsByTopic, debug, getMaximumEnqueuedRecordCount, getSnapshotMetricsObjectName, getStreamingMetricsObjectName, getStreamingMetricsObjectName, getStreamingNamespace, initializeConnectorTestFramework, isStreamingRunning, isStreamingRunning, isTransactionRecord, loggingCompletion, print, readLastCommittedOffset, readLastCommittedOffsets, setConsumeTimeout, skipAvroValidation, start, start, start, start, start, startAndConsumeTillEnd, stopConnector, stopConnector, validate, waitForAvailableRecords, waitForConnectorShutdown, waitForSnapshotToBeCompleted, waitForStreamingRunning, waitForStreamingRunning, waitTimeForRecords, waitTimeForRecordsAfterNulls
-
-
-
-
Field Detail
-
DB_HISTORY_PATH
private static final Path DB_HISTORY_PATH
-
DATABASE
private final UniqueDatabase DATABASE
-
RO_DATABASE
private final UniqueDatabase RO_DATABASE
-
PRODUCTS_TABLE_EVENT_COUNT
private static final int PRODUCTS_TABLE_EVENT_COUNT
- See Also:
- Constant Field Values
-
ORDERS_TABLE_EVENT_COUNT
private static final int ORDERS_TABLE_EVENT_COUNT
- See Also:
- Constant Field Values
-
INITIAL_EVENT_COUNT
private static final int INITIAL_EVENT_COUNT
- See Also:
- Constant Field Values
-
config
private Configuration config
-
-
Method Detail
-
beforeEach
public void beforeEach()
-
afterEach
public void afterEach()
-
shouldNotStartWithInvalidConfiguration
public void shouldNotStartWithInvalidConfiguration()
Verifies that the connector doesn't run with an invalid configuration. This does not actually connect to the MySQL server.
-
shouldFailToValidateInvalidConfiguration
public void shouldFailToValidateInvalidConfiguration()
-
shouldValidateValidConfigurationWithSSL
public void shouldValidateValidConfigurationWithSSL()
-
shouldValidateAcceptableConfiguration
public void shouldValidateAcceptableConfiguration()
-
shouldValidateLockingModeNoneWithValidSnapshotModeConfiguration
@FixFor("DBZ-639") public void shouldValidateLockingModeNoneWithValidSnapshotModeConfiguration()
Validates that SNAPSHOT_LOCKING_MODE 'none' is valid with all snapshot modes
-
getPKUpdateNewKeyHeader
private Optional<org.apache.kafka.connect.header.Header> getPKUpdateNewKeyHeader(org.apache.kafka.connect.source.SourceRecord record)
-
getPKUpdateOldKeyHeader
private Optional<org.apache.kafka.connect.header.Header> getPKUpdateOldKeyHeader(org.apache.kafka.connect.source.SourceRecord record)
-
getHeaderField
private Optional<org.apache.kafka.connect.header.Header> getHeaderField(org.apache.kafka.connect.source.SourceRecord record, String fieldName)
-
shouldConsumeAllEventsFromDatabaseUsingSnapshot
public void shouldConsumeAllEventsFromDatabaseUsingSnapshot() throws SQLException, InterruptedException- Throws:
SQLExceptionInterruptedException
-
shouldConsumeAllEventsFromDatabaseUsingSnapshotOld
public void shouldConsumeAllEventsFromDatabaseUsingSnapshotOld() throws SQLException, InterruptedException- Throws:
SQLExceptionInterruptedException
-
shouldConsumeAllEventsFromDatabaseUsingSnapshotByField
private void shouldConsumeAllEventsFromDatabaseUsingSnapshotByField(Field dbIncludeListField, int serverId) throws SQLException, InterruptedException
- Throws:
SQLExceptionInterruptedException
-
shouldUseOverriddenSelectStatementDuringSnapshotting
public void shouldUseOverriddenSelectStatementDuringSnapshotting() throws SQLException, InterruptedException- Throws:
SQLExceptionInterruptedException
-
shouldUseMultipleOverriddenSelectStatementsDuringSnapshotting
public void shouldUseMultipleOverriddenSelectStatementsDuringSnapshotting() throws SQLException, InterruptedException- Throws:
SQLExceptionInterruptedException
-
shouldIgnoreAlterTableForNonCapturedTablesNotStoredInHistory
@FixFor("DBZ-977") public void shouldIgnoreAlterTableForNonCapturedTablesNotStoredInHistory() throws SQLException, InterruptedException
- Throws:
SQLExceptionInterruptedException
-
shouldSaveSetCharacterSetWhenStoringOnlyCapturededTables
@FixFor("DBZ-1201") public void shouldSaveSetCharacterSetWhenStoringOnlyCapturededTables() throws SQLException, InterruptedException
- Throws:
SQLExceptionInterruptedException
-
shouldProcessCreateUniqueIndex
@FixFor("DBZ-1246") public void shouldProcessCreateUniqueIndex() throws SQLException, InterruptedException
- Throws:
SQLExceptionInterruptedException
-
shouldIgnoreAlterTableForNonCapturedTablesStoredInHistory
@FixFor("DBZ-977") public void shouldIgnoreAlterTableForNonCapturedTablesStoredInHistory() throws SQLException, InterruptedException
- Throws:
SQLExceptionInterruptedException
-
shouldIgnoreCreateIndexForNonCapturedTablesNotStoredInHistory
@FixFor("DBZ-1264") public void shouldIgnoreCreateIndexForNonCapturedTablesNotStoredInHistory() throws SQLException, InterruptedException
- Throws:
SQLExceptionInterruptedException
-
shouldReceiveSchemaForNonWhitelistedTablesAndDatabases
@FixFor("DBZ-683") public void shouldReceiveSchemaForNonWhitelistedTablesAndDatabases() throws SQLException, InterruptedException
- Throws:
SQLExceptionInterruptedException
-
shouldHandleIncludeListTables
@FixFor("DBZ-1546") public void shouldHandleIncludeListTables() throws SQLException, InterruptedException
- Throws:
SQLExceptionInterruptedException
-
shouldHandleWhitelistedTables
public void shouldHandleWhitelistedTables() throws SQLException, InterruptedException- Throws:
SQLExceptionInterruptedException
-
dropDatabases
private void dropDatabases() throws SQLException- Throws:
SQLException
-
getAfter
private org.apache.kafka.connect.data.Struct getAfter(org.apache.kafka.connect.source.SourceRecord record)
-
shouldConsumeEventsWithNoSnapshot
public void shouldConsumeEventsWithNoSnapshot() throws SQLException, InterruptedException- Throws:
SQLExceptionInterruptedException
-
shouldConsumeEventsWithIncludedColumns
@FixFor("DBZ-1962") public void shouldConsumeEventsWithIncludedColumns() throws SQLException, InterruptedException
- Throws:
SQLExceptionInterruptedException
-
shouldConsumeEventsWithMaskedAndBlacklistedColumns
public void shouldConsumeEventsWithMaskedAndBlacklistedColumns() throws SQLException, InterruptedException- Throws:
SQLExceptionInterruptedException
-
shouldConsumeEventsWithMaskedHashedColumns
@FixFor("DBZ-1692") public void shouldConsumeEventsWithMaskedHashedColumns() throws InterruptedException
- Throws:
InterruptedException
-
shouldConsumeEventsWithTruncatedColumns
@FixFor("DBZ-1972") public void shouldConsumeEventsWithTruncatedColumns() throws InterruptedException
- Throws:
InterruptedException
-
shouldEmitTombstoneOnDeleteByDefault
@FixFor("DBZ-582") public void shouldEmitTombstoneOnDeleteByDefault() throws Exception
- Throws:
Exception
-
shouldEmitNoTombstoneOnDelete
@FixFor("DBZ-582") public void shouldEmitNoTombstoneOnDelete() throws Exception
- Throws:
Exception
-
shouldEmitNoSavepoints
@FixFor("DBZ-794") public void shouldEmitNoSavepoints() throws Exception
- Throws:
Exception
-
shouldNotParseQueryIfServerOptionDisabled
@FixFor("DBZ-706") public void shouldNotParseQueryIfServerOptionDisabled() throws Exception
This test case validates that if you disable MySQL option binlog_rows_query_log_events, then the original SQL statement for an INSERT statement is NOT parsed into the resulting event.- Throws:
Exception
-
shouldNotParseQueryIfConnectorNotConfiguredTo
@FixFor("DBZ-706") public void shouldNotParseQueryIfConnectorNotConfiguredTo() throws Exception
This test case validates that if you enable MySQL option binlog_rows_query_log_events, but configure the connector to NOT include the query, it will not be included in the event.- Throws:
Exception
-
shouldParseQueryIfAvailableAndConnectorOptionEnabled
@FixFor("DBZ-706") public void shouldParseQueryIfAvailableAndConnectorOptionEnabled() throws Exception
This test case validates that if you enable MySQL option binlog_rows_query_log_events, then the original SQL statement for an INSERT statement is parsed into the resulting event.- Throws:
Exception
-
parseMultipleInsertStatements
@FixFor("DBZ-706") public void parseMultipleInsertStatements() throws Exception
This test case validates that if you enable MySQL option binlog_rows_query_log_events, then the issue multiple INSERTs, the appropriate SQL statements are parsed into the resulting events.- Throws:
Exception
-
parseMultipleRowInsertStatement
@FixFor("DBZ-706") public void parseMultipleRowInsertStatement() throws Exception
This test case validates that if you enable MySQL option binlog_rows_query_log_events, then the issue single multi-row INSERT, the appropriate SQL statements are parsed into the resulting events.- Throws:
Exception
-
parseDeleteQuery
@FixFor("DBZ-706") public void parseDeleteQuery() throws Exception
This test case validates that if you enable MySQL option binlog_rows_query_log_events, then the original SQL statement for a DELETE over a single row is parsed into the resulting event.- Throws:
Exception
-
parseMultiRowDeleteQuery
@FixFor("DBZ-706") public void parseMultiRowDeleteQuery() throws Exception
This test case validates that if you enable MySQL option binlog_rows_query_log_events, then issue a multi-row DELETE, the resulting events get the original SQL statement.- Throws:
Exception
-
parseUpdateQuery
@FixFor("DBZ-706") public void parseUpdateQuery() throws Exception
This test case validates that if you enable MySQL option binlog_rows_query_log_events, then the original SQL statement for an UPDATE over a single row is parsed into the resulting event.- Throws:
Exception
-
parseMultiRowUpdateQuery
@FixFor("DBZ-706") public void parseMultiRowUpdateQuery() throws Exception
This test case validates that if you enable MySQL option binlog_rows_query_log_events, then the original SQL statement for an UPDATE over a single row is parsed into the resulting event.- Throws:
Exception
-
shouldFailToValidateAdaptivePrecisionMode
@FixFor("DBZ-1234") public void shouldFailToValidateAdaptivePrecisionMode() throws InterruptedException
Specifying the adaptive time.precision.mode is no longer valid and a configuration validation problem should be reported when that configuration option is used.- Throws:
InterruptedException
-
testEmptySchemaLogWarningWithDatabaseWhitelist
@FixFor("DBZ-1242") public void testEmptySchemaLogWarningWithDatabaseWhitelist() throws Exception
- Throws:
Exception
-
testNoEmptySchemaLogWarningWithDatabaseWhitelist
@FixFor("DBZ-1242") public void testNoEmptySchemaLogWarningWithDatabaseWhitelist() throws Exception
- Throws:
Exception
-
testEmptySchemaWarningWithTableWhitelist
@FixFor("DBZ-1242") public void testEmptySchemaWarningWithTableWhitelist() throws Exception
- Throws:
Exception
-
testNoEmptySchemaWarningWithTableWhitelist
@FixFor("DBZ-1242") public void testNoEmptySchemaWarningWithTableWhitelist() throws Exception
- Throws:
Exception
-
shouldRewriteIdentityKey
@FixFor("DBZ-1015") public void shouldRewriteIdentityKey() throws InterruptedException, SQLException
- Throws:
InterruptedExceptionSQLException
-
shouldRewriteIdentityKeyWithWhitespace
@FixFor("DBZ-2957") public void shouldRewriteIdentityKeyWithWhitespace() throws InterruptedException, SQLException
- Throws:
InterruptedExceptionSQLException
-
shouldRewriteIdentityKeyWithMsgKeyColumnsFieldRegexValidation
@FixFor("DBZ-2957") public void shouldRewriteIdentityKeyWithMsgKeyColumnsFieldRegexValidation() throws InterruptedException, SQLException
- Throws:
InterruptedExceptionSQLException
-
shouldOutputRecordsInCloudEventsFormat
@FixFor("DBZ-1292") public void shouldOutputRecordsInCloudEventsFormat() throws Exception
- Throws:
Exception
-
waitForStreamingRunning
private void waitForStreamingRunning(String serverName) throws InterruptedException
- Throws:
InterruptedException
-
recordsForTopicForRoProductsTable
private List<org.apache.kafka.connect.source.SourceRecord> recordsForTopicForRoProductsTable(AbstractConnectorTest.SourceRecords records)
-
shouldEmitHeadersOnPrimaryKeyUpdate
@FixFor("DBZ-1531") public void shouldEmitHeadersOnPrimaryKeyUpdate() throws Exception
- Throws:
Exception
-
shouldEmitNoEventsForSkippedCreateOperations
@FixFor("DBZ-1895") public void shouldEmitNoEventsForSkippedCreateOperations() throws Exception
- Throws:
Exception
-
-