Package io.debezium.connector.mysql
Class MySqlConnectorConfig
- java.lang.Object
-
- io.debezium.config.CommonConnectorConfig
-
- io.debezium.relational.RelationalDatabaseConnectorConfig
-
- io.debezium.relational.HistorizedRelationalDatabaseConnectorConfig
-
- io.debezium.connector.mysql.MySqlConnectorConfig
-
public class MySqlConnectorConfig extends HistorizedRelationalDatabaseConnectorConfig
The configuration properties.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static classMySqlConnectorConfig.BigIntUnsignedHandlingModeThe set of predefined BigIntUnsignedHandlingMode options or aliases.static classMySqlConnectorConfig.GtidNewChannelPositionThe set of predefined Gtid New Channel Position options.static classMySqlConnectorConfig.SecureConnectionModeThe set of predefined SecureConnectionMode options or aliases.static classMySqlConnectorConfig.SnapshotLockingModeThe set of predefined Snapshot Locking Mode options.static classMySqlConnectorConfig.SnapshotModeThe set of predefined SnapshotMode options or aliases.static classMySqlConnectorConfig.SnapshotNewTables-
Nested classes/interfaces inherited from class io.debezium.relational.RelationalDatabaseConnectorConfig
RelationalDatabaseConnectorConfig.DecimalHandlingMode
-
Nested classes/interfaces inherited from class io.debezium.config.CommonConnectorConfig
CommonConnectorConfig.BinaryHandlingMode, CommonConnectorConfig.EventProcessingFailureHandlingMode, CommonConnectorConfig.Version
-
-
Field Summary
Fields Modifier and Type Field Description static Field.SetALL_FIELDSThe set ofFields defined as part of this configuration.static FieldBIGINT_UNSIGNED_HANDLING_MODEstatic FieldBUFFER_SIZE_FOR_BINLOG_READERprotected static Set<String>BUILT_IN_DB_NAMESprivate Configurationconfigprivate static ConfigDefinitionCONFIG_DEFINITIONstatic FieldCONNECTION_TIMEOUT_MSprivate DurationconnectionTimeoutstatic FieldDATABASE_HISTORYThe database history class is hidden in theconfigDef()since that is designed to work with a user interface, and in these situations using Kafka is the only way to go.private Predicate<String>ddlFilterprivate static intDEFAULT_BINLOG_BUFFER_SIZEDefault size of the binlog buffer used for examining transactions and deciding whether to propagate them or not.protected static intDEFAULT_PORTprotected static intDEFAULT_SNAPSHOT_FETCH_SIZEMinimum valueused for fetch size hint.static FieldENABLE_TIME_ADJUSTERstatic FieldEVENT_DESERIALIZATION_FAILURE_HANDLING_MODEprotected static Field.SetEXPOSED_FIELDSprivate MySqlConnectorConfig.GtidNewChannelPositiongitIdNewChannelPositionstatic FieldGTID_NEW_CHANNEL_POSITIONDeprecated.static FieldGTID_SOURCE_EXCLUDESA comma-separated list of regular expressions that match source UUIDs in the GTID set used to find the binlog position in the MySQL server.static FieldGTID_SOURCE_FILTER_DML_EVENTSIf set to true, we will only produce DML events into Kafka for transactions that were written on MySQL servers with UUIDs matching the filters defined by theGTID_SOURCE_INCLUDESorGTID_SOURCE_EXCLUDESconfiguration options, if they are specified.static FieldGTID_SOURCE_INCLUDESA comma-separated list of regular expressions that match source UUIDs in the GTID set used to find the binlog position in the MySQL server.private Predicate<String>gtidSourceFilterstatic FieldINCLUDE_SQL_QUERYstatic FieldINCONSISTENT_SCHEMA_HANDLING_MODEprivate CommonConnectorConfig.EventProcessingFailureHandlingModeinconsistentSchemaFailureHandlingModestatic FieldJDBC_DRIVERstatic FieldKEEP_ALIVEstatic FieldKEEP_ALIVE_INTERVAL_MSprivate booleanlegacyprivate static org.slf4j.LoggerLOGGERstatic FieldON_CONNECT_STATEMENTSstatic FieldPORTstatic FieldREAD_ONLY_CONNECTIONprivate booleanreadOnlyConnectionstatic FieldROW_COUNT_FOR_STREAMING_RESULT_SETSstatic FieldSERVER_IDstatic FieldSERVER_ID_OFFSETstatic FieldSERVER_NAMEstatic FieldSNAPSHOT_LOCKING_MODEstatic FieldSNAPSHOT_MODEstatic FieldSNAPSHOT_NEW_TABLESprivate MySqlConnectorConfig.SnapshotLockingModesnapshotLockingModeprivate MySqlConnectorConfig.SnapshotModesnapshotModeprivate MySqlConnectorConfig.SnapshotNewTablessnapshotNewTablesprivate SourceInfoStructMaker<? extends AbstractSourceInfo>sourceInfoStructMakerstatic FieldSSL_KEYSTOREstatic FieldSSL_KEYSTORE_PASSWORDstatic FieldSSL_MODEstatic FieldSSL_TRUSTSTOREstatic FieldSSL_TRUSTSTORE_PASSWORDstatic FieldTABLES_IGNORE_BUILTINprivate TemporalPrecisionModetemporalPrecisionMode(package private) static StringTEST_DISABLE_GLOBAL_LOCKINGIt is not possible to test disabled global locking locally as regular MySQL build always provides global locking.static FieldTIME_PRECISION_MODE-
Fields inherited from class io.debezium.relational.RelationalDatabaseConnectorConfig
COLUMN_BLACKLIST, COLUMN_EXCLUDE_LIST, COLUMN_INCLUDE_LIST, COLUMN_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG, COLUMN_WHITELIST, COLUMN_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG, DATABASE_BLACKLIST, DATABASE_BLACKLIST_NAME, DATABASE_EXCLUDE_LIST, DATABASE_EXCLUDE_LIST_NAME, DATABASE_INCLUDE_LIST, DATABASE_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG, DATABASE_INCLUDE_LIST_NAME, DATABASE_NAME, DATABASE_WHITELIST, DATABASE_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG, DATABASE_WHITELIST_NAME, DECIMAL_HANDLING_MODE, DEFAULT_SNAPSHOT_LOCK_TIMEOUT_MILLIS, HOSTNAME, INCLUDE_SCHEMA_CHANGES, INCLUDE_SCHEMA_COMMENTS, MASK_COLUMN, MASK_COLUMN_WITH_HASH, MSG_KEY_COLUMNS, PASSWORD, PROPAGATE_COLUMN_SOURCE_TYPE, PROPAGATE_DATATYPE_SOURCE_TYPE, SCHEMA_BLACKLIST, SCHEMA_EXCLUDE_LIST, SCHEMA_EXCLUDE_LIST_NAME, SCHEMA_INCLUDE_LIST, SCHEMA_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG, SCHEMA_INCLUDE_LIST_NAME, SCHEMA_WHITELIST, SCHEMA_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG, SERVER_NAME_PATTERN, SNAPSHOT_FULL_COLUMN_SCAN_FORCE, SNAPSHOT_LOCK_TIMEOUT_MS, SNAPSHOT_SELECT_STATEMENT_OVERRIDES_BY_TABLE, TABLE_BLACKLIST, TABLE_BLACKLIST_NAME, TABLE_EXCLUDE_LIST, TABLE_EXCLUDE_LIST_NAME, TABLE_IGNORE_BUILTIN, TABLE_INCLUDE_LIST, TABLE_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG, TABLE_INCLUDE_LIST_NAME, TABLE_WHITELIST, TABLE_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG, TABLE_WHITELIST_NAME, TRUNCATE_COLUMN, USER
-
Fields inherited from class io.debezium.config.CommonConnectorConfig
BINARY_HANDLING_MODE, CUSTOM_CONVERTERS, DATABASE_CONFIG_PREFIX, DEFAULT_MAX_BATCH_SIZE, DEFAULT_MAX_QUEUE_SIZE, DEFAULT_MAX_QUEUE_SIZE_IN_BYTES, DEFAULT_POLL_INTERVAL_MILLIS, DEFAULT_QUERY_FETCH_SIZE, DEFAULT_RETRIABLE_RESTART_WAIT, EVENT_PROCESSING_FAILURE_HANDLING_MODE, INCREMENTAL_SNAPSHOT_CHUNK_SIZE, MAX_BATCH_SIZE, MAX_QUEUE_SIZE, MAX_QUEUE_SIZE_IN_BYTES, POLL_INTERVAL_MS, PROVIDE_TRANSACTION_METADATA, QUERY_FETCH_SIZE, RETRIABLE_RESTART_WAIT, SANITIZE_FIELD_NAMES, SIGNAL_DATA_COLLECTION, SKIPPED_OPERATIONS, SNAPSHOT_DELAY_MS, SNAPSHOT_FETCH_SIZE, SNAPSHOT_MAX_THREADS, SNAPSHOT_MODE_TABLES, SOURCE_STRUCT_MAKER_VERSION, TOMBSTONES_ON_DELETE, TRANSACTION_TOPIC
-
-
Constructor Summary
Constructors Constructor Description MySqlConnectorConfig(Configuration config)
-
Method Summary
-
Methods inherited from class io.debezium.relational.HistorizedRelationalDatabaseConnectorConfig
getDatabaseHistory, useCatalogBeforeSchema
-
Methods inherited from class io.debezium.relational.RelationalDatabaseConnectorConfig
getColumnFilter, getDecimalMode, getHeartbeatActionQuery, getJdbcConfig, getKeyMapper, getSnapshotSelectOverridesByTable, getTableFilters, getTableIdMapper, isFullColummnScanRequired, isSchemaChangesHistoryEnabled, isSchemaCommentsHistoryEnabled, schemaExcludeList, schemaIncludeList, snapshotLockTimeout, tableExcludeList, tableIncludeList
-
Methods inherited from class io.debezium.config.CommonConnectorConfig
binaryHandlingMode, customConverterRegistry, getConfig, getDataCollectionsToBeSnapshotted, getEventProcessingFailureHandlingMode, getHeartbeatInterval, getHeartbeatTopicsPrefix, getIncrementalSnashotChunkSize, getLogicalName, getMaxBatchSize, getMaxQueueSize, getMaxQueueSizeInBytes, getPollInterval, getQueryFetchSize, getRetriableRestartWait, getSanitizeFieldNames, getSignalingDataCollectionId, getSkippedOperations, getSnapshotDelay, getSnapshotFetchSize, getSnapshotMaxThreads, getTransactionTopic, isEmitTombstoneOnDelete, legacyGetDataCollectionsToBeSnapshotted, shouldProvideTransactionMetadata, validate, validateAndRecord, validateServerNameIsDifferentFromHistoryTopicName
-
-
-
-
Field Detail
-
LOGGER
private static final org.slf4j.Logger LOGGER
-
TEST_DISABLE_GLOBAL_LOCKING
static final String TEST_DISABLE_GLOBAL_LOCKING
It is not possible to test disabled global locking locally as regular MySQL build always provides global locking. So to bypass this limitation it is necessary to provide a backdoor to connector to disable it on its own.- See Also:
- Constant Field Values
-
DEFAULT_SNAPSHOT_FETCH_SIZE
protected static final int DEFAULT_SNAPSHOT_FETCH_SIZE
Minimum valueused for fetch size hint. See DBZ-94 for details.- See Also:
- Constant Field Values
-
DEFAULT_PORT
protected static final int DEFAULT_PORT
- See Also:
- Constant Field Values
-
DEFAULT_BINLOG_BUFFER_SIZE
private static final int DEFAULT_BINLOG_BUFFER_SIZE
Default size of the binlog buffer used for examining transactions and deciding whether to propagate them or not. A size of 0 disables the buffer, all events will be passed on directly as they are passed by the binlog client.- See Also:
- Constant Field Values
-
PORT
public static final Field PORT
-
ON_CONNECT_STATEMENTS
public static final Field ON_CONNECT_STATEMENTS
-
SERVER_NAME
public static final Field SERVER_NAME
-
SERVER_ID
public static final Field SERVER_ID
-
SERVER_ID_OFFSET
public static final Field SERVER_ID_OFFSET
-
SSL_MODE
public static final Field SSL_MODE
-
SSL_KEYSTORE
public static final Field SSL_KEYSTORE
-
SSL_KEYSTORE_PASSWORD
public static final Field SSL_KEYSTORE_PASSWORD
-
SSL_TRUSTSTORE
public static final Field SSL_TRUSTSTORE
-
SSL_TRUSTSTORE_PASSWORD
public static final Field SSL_TRUSTSTORE_PASSWORD
-
TABLES_IGNORE_BUILTIN
public static final Field TABLES_IGNORE_BUILTIN
-
JDBC_DRIVER
public static final Field JDBC_DRIVER
-
GTID_SOURCE_INCLUDES
public static final Field GTID_SOURCE_INCLUDES
A comma-separated list of regular expressions that match source UUIDs in the GTID set used to find the binlog position in the MySQL server. Only the GTID ranges that have sources matching one of these include patterns will be used. Must not be used withGTID_SOURCE_EXCLUDES.
-
GTID_SOURCE_EXCLUDES
public static final Field GTID_SOURCE_EXCLUDES
A comma-separated list of regular expressions that match source UUIDs in the GTID set used to find the binlog position in the MySQL server. Only the GTID ranges that have sources matching none of these exclude patterns will be used. Must not be used withGTID_SOURCE_INCLUDES.
-
GTID_SOURCE_FILTER_DML_EVENTS
public static final Field GTID_SOURCE_FILTER_DML_EVENTS
If set to true, we will only produce DML events into Kafka for transactions that were written on MySQL servers with UUIDs matching the filters defined by theGTID_SOURCE_INCLUDESorGTID_SOURCE_EXCLUDESconfiguration options, if they are specified. Defaults to true. When true, eitherGTID_SOURCE_INCLUDESorGTID_SOURCE_EXCLUDESmust be set.
-
GTID_NEW_CHANNEL_POSITION
@Deprecated public static final Field GTID_NEW_CHANNEL_POSITION
Deprecated.If set to 'latest', connector when encountering new GTID channel after job restart will start reading it from the latest executed position (default). When set to 'earliest' the connector will start reading new GTID channels from the first available position. This is useful when in active-passive mysql setup during failover new GTID channel starts receiving writes, see DBZ-923. Defaults to latest.
-
CONNECTION_TIMEOUT_MS
public static final Field CONNECTION_TIMEOUT_MS
-
KEEP_ALIVE
public static final Field KEEP_ALIVE
-
KEEP_ALIVE_INTERVAL_MS
public static final Field KEEP_ALIVE_INTERVAL_MS
-
ROW_COUNT_FOR_STREAMING_RESULT_SETS
public static final Field ROW_COUNT_FOR_STREAMING_RESULT_SETS
-
BUFFER_SIZE_FOR_BINLOG_READER
public static final Field BUFFER_SIZE_FOR_BINLOG_READER
-
DATABASE_HISTORY
public static final Field DATABASE_HISTORY
The database history class is hidden in theconfigDef()since that is designed to work with a user interface, and in these situations using Kafka is the only way to go.
-
INCLUDE_SQL_QUERY
public static final Field INCLUDE_SQL_QUERY
-
SNAPSHOT_MODE
public static final Field SNAPSHOT_MODE
-
SNAPSHOT_LOCKING_MODE
public static final Field SNAPSHOT_LOCKING_MODE
-
SNAPSHOT_NEW_TABLES
public static final Field SNAPSHOT_NEW_TABLES
-
TIME_PRECISION_MODE
public static final Field TIME_PRECISION_MODE
-
BIGINT_UNSIGNED_HANDLING_MODE
public static final Field BIGINT_UNSIGNED_HANDLING_MODE
-
EVENT_DESERIALIZATION_FAILURE_HANDLING_MODE
public static final Field EVENT_DESERIALIZATION_FAILURE_HANDLING_MODE
-
INCONSISTENT_SCHEMA_HANDLING_MODE
public static final Field INCONSISTENT_SCHEMA_HANDLING_MODE
-
ENABLE_TIME_ADJUSTER
public static final Field ENABLE_TIME_ADJUSTER
-
READ_ONLY_CONNECTION
public static final Field READ_ONLY_CONNECTION
-
CONFIG_DEFINITION
private static final ConfigDefinition CONFIG_DEFINITION
-
ALL_FIELDS
public static Field.Set ALL_FIELDS
The set ofFields defined as part of this configuration.
-
EXPOSED_FIELDS
protected static Field.Set EXPOSED_FIELDS
-
config
private final Configuration config
-
snapshotMode
private final MySqlConnectorConfig.SnapshotMode snapshotMode
-
snapshotLockingMode
private final MySqlConnectorConfig.SnapshotLockingMode snapshotLockingMode
-
gitIdNewChannelPosition
private final MySqlConnectorConfig.GtidNewChannelPosition gitIdNewChannelPosition
-
snapshotNewTables
private final MySqlConnectorConfig.SnapshotNewTables snapshotNewTables
-
temporalPrecisionMode
private final TemporalPrecisionMode temporalPrecisionMode
-
connectionTimeout
private final Duration connectionTimeout
-
inconsistentSchemaFailureHandlingMode
private final CommonConnectorConfig.EventProcessingFailureHandlingMode inconsistentSchemaFailureHandlingMode
-
legacy
private final boolean legacy
-
sourceInfoStructMaker
private final SourceInfoStructMaker<? extends AbstractSourceInfo> sourceInfoStructMaker
-
readOnlyConnection
private final boolean readOnlyConnection
-
-
Constructor Detail
-
MySqlConnectorConfig
public MySqlConnectorConfig(Configuration config)
-
-
Method Detail
-
configDef
protected static org.apache.kafka.common.config.ConfigDef configDef()
-
supportsOperationFiltering
public boolean supportsOperationFiltering()
- Overrides:
supportsOperationFilteringin classCommonConnectorConfig
-
useCursorFetch
public boolean useCursorFetch()
-
getSnapshotLockingMode
public MySqlConnectorConfig.SnapshotLockingMode getSnapshotLockingMode()
-
gtidNewChannelPosition
public MySqlConnectorConfig.GtidNewChannelPosition gtidNewChannelPosition()
-
getSnapshotNewTables
public MySqlConnectorConfig.SnapshotNewTables getSnapshotNewTables()
-
validateGtidNewChannelPositionNotSet
private static int validateGtidNewChannelPositionNotSet(Configuration config, Field field, Field.ValidationOutput problems)
-
validateEventDeserializationFailureHandlingModeNotSet
private static int validateEventDeserializationFailureHandlingModeNotSet(Configuration config, Field field, Field.ValidationOutput problems)
-
validateInconsistentSchemaHandlingModeNotIgnore
private static int validateInconsistentSchemaHandlingModeNotIgnore(Configuration config, Field field, Field.ValidationOutput problems)
-
validateGtidSetExcludes
private static int validateGtidSetExcludes(Configuration config, Field field, Field.ValidationOutput problems)
-
validateSnapshotLockingMode
private static int validateSnapshotLockingMode(Configuration config, Field field, Field.ValidationOutput problems)
Validate the new snapshot.locking.mode configuration, which replaces snapshot.minimal.locking. If minimal.locking is explicitly defined and locking.mode is NOT explicitly defined: - coerce minimal.locking into the new snap.locking.mode property. If minimal.locking is NOT explicitly defined and locking.mode IS explicitly defined: - use new locking.mode property. If BOTH minimal.locking and locking.mode ARE defined: - Throw a validation error.
-
validateTimePrecisionMode
private static int validateTimePrecisionMode(Configuration config, Field field, Field.ValidationOutput problems)
Validate the time.precision.mode configuration. Ifadaptiveis specified, this option has the potential to cause overflow which is why the option was deprecated and no longer supported for this connector.
-
randomServerId
private static int randomServerId()
-
getSourceInfoStructMaker
protected SourceInfoStructMaker<? extends AbstractSourceInfo> getSourceInfoStructMaker(CommonConnectorConfig.Version version)
- Specified by:
getSourceInfoStructMakerin classCommonConnectorConfig
-
getContextName
public String getContextName()
- Specified by:
getContextNamein classCommonConnectorConfig
-
getConnectorName
public String getConnectorName()
- Specified by:
getConnectorNamein classCommonConnectorConfig
-
getTemporalPrecisionMode
public TemporalPrecisionMode getTemporalPrecisionMode()
- Overrides:
getTemporalPrecisionModein classRelationalDatabaseConnectorConfig
-
getSnapshotMode
public MySqlConnectorConfig.SnapshotMode getSnapshotMode()
-
getConnectionTimeout
public Duration getConnectionTimeout()
-
inconsistentSchemaFailureHandlingMode
public CommonConnectorConfig.EventProcessingFailureHandlingMode inconsistentSchemaFailureHandlingMode()
-
hostname
public String hostname()
-
port
public int port()
-
username
public String username()
-
password
public String password()
-
serverId
public long serverId()
-
sslMode
public MySqlConnectorConfig.SecureConnectionMode sslMode()
-
sslModeEnabled
public boolean sslModeEnabled()
-
bufferSizeForStreamingChangeEventSource
public int bufferSizeForStreamingChangeEventSource()
-
gtidSourceFilter
public Predicate<String> gtidSourceFilter()
Get the predicate function that will returntrueif a GTID source is to be included, orfalseif a GTID source is to be excluded.- Returns:
- the GTID source predicate function; never null
-
includeSchemaChangeRecords
public boolean includeSchemaChangeRecords()
-
includeSqlQuery
public boolean includeSqlQuery()
-
rowCountForLargeTable
public long rowCountForLargeTable()
-
getHistoryRecordComparator
protected HistoryRecordComparator getHistoryRecordComparator()
- Specified by:
getHistoryRecordComparatorin classHistorizedRelationalDatabaseConnectorConfig
-
isBuiltInDatabase
public static boolean isBuiltInDatabase(String databaseName)
-
isNotBuiltInTable
public static boolean isNotBuiltInTable(TableId id)
-
legacy
boolean legacy()
-
isReadOnlyConnection
public boolean isReadOnlyConnection()
-
useGlobalLock
boolean useGlobalLock()
Intended for testing only
-
getSourceInfoStructMaker
public SourceInfoStructMaker<? extends AbstractSourceInfo> getSourceInfoStructMaker()
- Overrides:
getSourceInfoStructMakerin classCommonConnectorConfig
-
-