Package io.debezium.connector.mysql
Class MySqlConnectorConfig
java.lang.Object
io.debezium.config.CommonConnectorConfig
io.debezium.relational.RelationalDatabaseConnectorConfig
io.debezium.relational.HistorizedRelationalDatabaseConnectorConfig
io.debezium.connector.mysql.MySqlConnectorConfig
The configuration properties.
-
Nested Class Summary
Nested ClassesModifier and TypeClassDescriptionstatic enumThe set of predefined BigIntUnsignedHandlingMode options or aliases.static enumThe set of predefined SecureConnectionMode options or aliases.static enumThe set of predefined Snapshot Locking Mode options.static enumThe set of predefined SnapshotMode options or aliases.static enumNested classes/interfaces inherited from class io.debezium.relational.RelationalDatabaseConnectorConfig
RelationalDatabaseConnectorConfig.DecimalHandlingMode, RelationalDatabaseConnectorConfig.SnapshotTablesRowCountOrderNested classes/interfaces inherited from class io.debezium.config.CommonConnectorConfig
CommonConnectorConfig.BinaryHandlingMode, CommonConnectorConfig.EventProcessingFailureHandlingMode, CommonConnectorConfig.FieldNameAdjustmentMode, CommonConnectorConfig.SchemaNameAdjustmentMode, CommonConnectorConfig.Version -
Field Summary
FieldsModifier and TypeFieldDescriptionstatic Field.SetThe set ofFields defined as part of this configuration.static final Fieldstatic final Fieldprivate final Configurationprivate static final ConfigDefinitionstatic final Fieldprivate final Durationprivate static final intDefault size of the binlog buffer used for examining transactions and deciding whether to propagate them or not.protected static final intprotected static final intMinimum valueused for fetch size hint.static final Fieldstatic final Fieldprotected static Field.Setstatic final FieldA comma-separated list of regular expressions that match source UUIDs in the GTID set used to find the binlog position in the MySQL server.static final FieldIf set to true, we will only produce DML events into Kafka for transactions that were written on MySQL servers with UUIDs matching the filters defined by theGTID_SOURCE_INCLUDESorGTID_SOURCE_EXCLUDESconfiguration options, if they are specified.static final FieldA comma-separated list of regular expressions that match source UUIDs in the GTID set used to find the binlog position in the MySQL server.static final Fieldstatic final Fieldprivate final CommonConnectorConfig.EventProcessingFailureHandlingModestatic final Fieldstatic final Fieldstatic final Fieldstatic final Fieldprivate static final org.slf4j.Loggerstatic final Fieldstatic final Fieldstatic final Fieldprivate final booleanstatic final Fieldstatic final Fieldstatic final Fieldstatic final Fieldstatic final Fieldstatic final Fieldprivate final MySqlConnectorConfig.SnapshotLockingModeprivate final MySqlConnectorConfig.SnapshotModeprivate final MySqlConnectorConfig.SnapshotNewTablesstatic final Fieldstatic final Fieldstatic final Fieldstatic final Fieldstatic final Fieldstatic final Fieldstatic final Fieldstatic final Fieldprivate final TemporalPrecisionMode(package private) static final StringIt is not possible to test disabled global locking locally as regular MySQL build always provides global locking.static final Fieldstatic final FieldFields inherited from class io.debezium.relational.HistorizedRelationalDatabaseConnectorConfig
SCHEMA_HISTORY, SKIP_UNPARSEABLE_DDL_STATEMENTS, skipUnparseableDDL, STORE_ONLY_CAPTURED_TABLES_DDL, storeOnlyCapturedDatabasesDdl, storeOnlyCapturedTablesDdlFields inherited from class io.debezium.relational.RelationalDatabaseConnectorConfig
COLUMN_EXCLUDE_LIST, COLUMN_INCLUDE_LIST, COLUMN_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG, DATABASE_EXCLUDE_LIST, DATABASE_EXCLUDE_LIST_NAME, DATABASE_INCLUDE_LIST, DATABASE_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG, DATABASE_INCLUDE_LIST_NAME, DATABASE_NAME, DECIMAL_HANDLING_MODE, DEFAULT_SNAPSHOT_LOCK_TIMEOUT_MILLIS, DEFAULT_UNAVAILABLE_VALUE_PLACEHOLDER, HOSTNAME, HOSTNAME_PATTERN, INCLUDE_SCHEMA_CHANGES, INCLUDE_SCHEMA_COMMENTS, MASK_COLUMN, MASK_COLUMN_WITH_HASH, MSG_KEY_COLUMNS, PASSWORD, PROPAGATE_COLUMN_SOURCE_TYPE, PROPAGATE_DATATYPE_SOURCE_TYPE, SCHEMA_EXCLUDE_LIST, SCHEMA_EXCLUDE_LIST_NAME, SCHEMA_INCLUDE_LIST, SCHEMA_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG, SCHEMA_INCLUDE_LIST_NAME, SNAPSHOT_FULL_COLUMN_SCAN_FORCE, SNAPSHOT_LOCK_TIMEOUT_MS, SNAPSHOT_SELECT_STATEMENT_OVERRIDES_BY_TABLE, SNAPSHOT_TABLES_ORDER_BY_ROW_COUNT, TABLE_EXCLUDE_LIST, TABLE_EXCLUDE_LIST_NAME, TABLE_IGNORE_BUILTIN, TABLE_INCLUDE_LIST, TABLE_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG, TABLE_INCLUDE_LIST_NAME, TRUNCATE_COLUMN, UNAVAILABLE_VALUE_PLACEHOLDER, USERFields inherited from class io.debezium.config.CommonConnectorConfig
BINARY_HANDLING_MODE, CUSTOM_CONVERTERS, CUSTOM_METRIC_TAGS, CUSTOM_RETRIABLE_EXCEPTION, DATABASE_CONFIG_PREFIX, DEFAULT_MAX_BATCH_SIZE, DEFAULT_MAX_QUEUE_SIZE, DEFAULT_MAX_QUEUE_SIZE_IN_BYTES, DEFAULT_MAX_RETRIES, DEFAULT_POLL_INTERVAL_MILLIS, DEFAULT_QUERY_FETCH_SIZE, DEFAULT_RETRIABLE_RESTART_WAIT, DRIVER_CONFIG_PREFIX, ERRORS_MAX_RETRIES, EVENT_PROCESSING_FAILURE_HANDLING_MODE, FIELD_NAME_ADJUSTMENT_MODE, INCREMENTAL_SNAPSHOT_ALLOW_SCHEMA_CHANGES, INCREMENTAL_SNAPSHOT_CHUNK_SIZE, logicalName, MAX_BATCH_SIZE, MAX_QUEUE_SIZE, MAX_QUEUE_SIZE_IN_BYTES, MAX_RETRIES_ON_ERROR, MULTI_PARTITION_MODE, NOTIFICATION_CONFIGURATION_FIELD_PREFIX_STRING, NOTIFICATION_ENABLED_CHANNELS, POLL_INTERVAL_MS, PROVIDE_TRANSACTION_METADATA, QUERY_FETCH_SIZE, RETRIABLE_RESTART_WAIT, SCHEMA_NAME_ADJUSTMENT_MODE, SIGNAL_DATA_COLLECTION, SIGNAL_ENABLED_CHANNELS, SIGNAL_POLL_INTERVAL_MS, SKIP_MESSAGES_WITHOUT_CHANGE, SKIPPED_OPERATIONS, SNAPSHOT_DELAY_MS, SNAPSHOT_FETCH_SIZE, SNAPSHOT_MAX_THREADS, SNAPSHOT_MODE_TABLES, TASK_ID, TOMBSTONES_ON_DELETE, TOPIC_NAME_PATTERN, TOPIC_PREFIX -
Constructor Summary
Constructors -
Method Summary
Modifier and TypeMethodDescriptionintprotected static org.apache.kafka.common.config.ConfigDefprotected HistoryRecordComparatorprotected SourceInfoStructMaker<? extends AbstractSourceInfo>Get the predicate function that will returntrueif a GTID source is to be included, orfalseif a GTID source is to be excluded.hostname()booleanbooleanstatic booleanisBuiltInDatabase(String databaseName) static booleanbooleanpassword()intport()longlongserverId()sslMode()booleanbooleanprotected booleanboolean(package private) booleanIntended for testing onlyusername()private static intvalidateEventDeserializationFailureHandlingModeNotSet(Configuration config, Field field, Field.ValidationOutput problems) private static intvalidateGtidSetExcludes(Configuration config, Field field, Field.ValidationOutput problems) private static intvalidateSnapshotLockingMode(Configuration config, Field field, Field.ValidationOutput problems) Validate the new snapshot.locking.mode configuration, which replaces snapshot.minimal.locking.private static intvalidateTimePrecisionMode(Configuration config, Field field, Field.ValidationOutput problems) Validate the time.precision.mode configuration.Methods inherited from class io.debezium.relational.HistorizedRelationalDatabaseConnectorConfig
ddlFilter, getSchemaHistory, multiPartitionMode, skipUnparseableDdlStatements, storeOnlyCapturedDatabases, storeOnlyCapturedTables, useCatalogBeforeSchemaMethods inherited from class io.debezium.relational.RelationalDatabaseConnectorConfig
createHeartbeat, getColumnFilter, getDecimalMode, getFieldNamer, getHeartbeatActionQuery, getJdbcConfig, getKeyMapper, getSnapshotSelectOverridesByTable, getTableFilters, getTableIdMapper, getUnavailableValuePlaceholder, isColumnsFiltered, isFullColumnScanRequired, isSchemaChangesHistoryEnabled, isSchemaCommentsHistoryEnabled, schemaExcludeList, schemaIncludeList, snapshotLockTimeout, snapshotOrderByRowCount, tableExcludeList, tableIncludeListMethods inherited from class io.debezium.config.CommonConnectorConfig
binaryHandlingMode, createCustomMetricTags, customConverterRegistry, customRetriableException, fieldNameAdjuster, getConfig, getCustomMetricTags, getDataCollectionsToBeSnapshotted, getEnabledChannels, getEnabledNotificationChannels, getEventProcessingFailureHandlingMode, getHeartbeatInterval, getHeartbeatTopicsPrefix, getIncrementalSnapshotChunkSize, getLogicalName, getMaxBatchSize, getMaxQueueSize, getMaxQueueSizeInBytes, getMaxRetriesOnError, getNotificationTopic, getPollInterval, getQueryFetchSize, getRetriableRestartWait, getSignalingDataCollectionId, getSignalPollInterval, getSkippedOperations, getSnapshotDelay, getSnapshotFetchSize, getSnapshotMaxThreads, getSourceInfoStructMaker, getSourceInfoStructMaker, getTaskId, getTopicNamingStrategy, getTopicNamingStrategy, isEmitTombstoneOnDelete, isIncrementalSnapshotSchemaChangesEnabled, isSignalDataCollection, parseSignallingMessage, schemaNameAdjuster, shouldProvideTransactionMetadata, skipMessagesWithoutChange, validate, validateAndRecord, validateSkippedOperation, validateTopicName
-
Field Details
-
LOGGER
private static final org.slf4j.Logger LOGGER -
TEST_DISABLE_GLOBAL_LOCKING
It is not possible to test disabled global locking locally as regular MySQL build always provides global locking. So to bypass this limitation it is necessary to provide a backdoor to connector to disable it on its own.- See Also:
-
DEFAULT_SNAPSHOT_FETCH_SIZE
protected static final int DEFAULT_SNAPSHOT_FETCH_SIZEMinimum valueused for fetch size hint. See DBZ-94 for details.- See Also:
-
DEFAULT_PORT
protected static final int DEFAULT_PORT- See Also:
-
DEFAULT_BINLOG_BUFFER_SIZE
private static final int DEFAULT_BINLOG_BUFFER_SIZEDefault size of the binlog buffer used for examining transactions and deciding whether to propagate them or not. A size of 0 disables the buffer, all events will be passed on directly as they are passed by the binlog client.- See Also:
-
PORT
-
ON_CONNECT_STATEMENTS
-
SERVER_ID
-
SERVER_ID_OFFSET
-
SSL_MODE
-
SSL_KEYSTORE
-
SSL_KEYSTORE_PASSWORD
-
SSL_TRUSTSTORE
-
SSL_TRUSTSTORE_PASSWORD
-
TABLES_IGNORE_BUILTIN
-
JDBC_DRIVER
-
JDBC_PROTOCOL
-
GTID_SOURCE_INCLUDES
A comma-separated list of regular expressions that match source UUIDs in the GTID set used to find the binlog position in the MySQL server. Only the GTID ranges that have sources matching one of these include patterns will be used. Must not be used withGTID_SOURCE_EXCLUDES. -
GTID_SOURCE_EXCLUDES
A comma-separated list of regular expressions that match source UUIDs in the GTID set used to find the binlog position in the MySQL server. Only the GTID ranges that have sources matching none of these exclude patterns will be used. Must not be used withGTID_SOURCE_INCLUDES. -
GTID_SOURCE_FILTER_DML_EVENTS
If set to true, we will only produce DML events into Kafka for transactions that were written on MySQL servers with UUIDs matching the filters defined by theGTID_SOURCE_INCLUDESorGTID_SOURCE_EXCLUDESconfiguration options, if they are specified. Defaults to true. When true, eitherGTID_SOURCE_INCLUDESorGTID_SOURCE_EXCLUDESmust be set. -
CONNECTION_TIMEOUT_MS
-
KEEP_ALIVE
-
KEEP_ALIVE_INTERVAL_MS
-
ROW_COUNT_FOR_STREAMING_RESULT_SETS
-
BUFFER_SIZE_FOR_BINLOG_READER
-
TOPIC_NAMING_STRATEGY
-
INCLUDE_SQL_QUERY
-
SNAPSHOT_MODE
-
SNAPSHOT_LOCKING_MODE
-
SNAPSHOT_NEW_TABLES
-
TIME_PRECISION_MODE
-
BIGINT_UNSIGNED_HANDLING_MODE
-
EVENT_DESERIALIZATION_FAILURE_HANDLING_MODE
-
INCONSISTENT_SCHEMA_HANDLING_MODE
-
ENABLE_TIME_ADJUSTER
-
READ_ONLY_CONNECTION
-
SOURCE_INFO_STRUCT_MAKER
-
STORE_ONLY_CAPTURED_DATABASES_DDL
-
CONFIG_DEFINITION
-
ALL_FIELDS
The set ofFields defined as part of this configuration. -
EXPOSED_FIELDS
-
BUILT_IN_DB_NAMES
-
config
-
snapshotMode
-
snapshotLockingMode
-
snapshotNewTables
-
temporalPrecisionMode
-
connectionTimeout
-
gtidSourceFilter
-
inconsistentSchemaFailureHandlingMode
private final CommonConnectorConfig.EventProcessingFailureHandlingMode inconsistentSchemaFailureHandlingMode -
readOnlyConnection
private final boolean readOnlyConnection
-
-
Constructor Details
-
MySqlConnectorConfig
-
-
Method Details
-
configDef
protected static org.apache.kafka.common.config.ConfigDef configDef() -
supportsOperationFiltering
public boolean supportsOperationFiltering()- Overrides:
supportsOperationFilteringin classCommonConnectorConfig
-
supportsSchemaChangesDuringIncrementalSnapshot
protected boolean supportsSchemaChangesDuringIncrementalSnapshot()- Overrides:
supportsSchemaChangesDuringIncrementalSnapshotin classCommonConnectorConfig
-
useCursorFetch
public boolean useCursorFetch() -
getSnapshotLockingMode
-
getSnapshotNewTables
-
validateEventDeserializationFailureHandlingModeNotSet
private static int validateEventDeserializationFailureHandlingModeNotSet(Configuration config, Field field, Field.ValidationOutput problems) -
validateGtidSetExcludes
private static int validateGtidSetExcludes(Configuration config, Field field, Field.ValidationOutput problems) -
validateSnapshotLockingMode
private static int validateSnapshotLockingMode(Configuration config, Field field, Field.ValidationOutput problems) Validate the new snapshot.locking.mode configuration, which replaces snapshot.minimal.locking. If minimal.locking is explicitly defined and locking.mode is NOT explicitly defined: - coerce minimal.locking into the new snap.locking.mode property. If minimal.locking is NOT explicitly defined and locking.mode IS explicitly defined: - use new locking.mode property. If BOTH minimal.locking and locking.mode ARE defined: - Throw a validation error. -
validateTimePrecisionMode
private static int validateTimePrecisionMode(Configuration config, Field field, Field.ValidationOutput problems) Validate the time.precision.mode configuration. Ifadaptiveis specified, this option has the potential to cause overflow which is why the option was deprecated and no longer supported for this connector. -
getSourceInfoStructMaker
protected SourceInfoStructMaker<? extends AbstractSourceInfo> getSourceInfoStructMaker(CommonConnectorConfig.Version version) - Specified by:
getSourceInfoStructMakerin classCommonConnectorConfig
-
getContextName
- Specified by:
getContextNamein classCommonConnectorConfig
-
getConnectorName
- Specified by:
getConnectorNamein classCommonConnectorConfig
-
getTemporalPrecisionMode
- Overrides:
getTemporalPrecisionModein classRelationalDatabaseConnectorConfig
-
getSnapshotMode
-
getConnectionTimeout
-
inconsistentSchemaFailureHandlingMode
public CommonConnectorConfig.EventProcessingFailureHandlingMode inconsistentSchemaFailureHandlingMode() -
hostname
-
port
public int port() -
username
-
password
-
serverId
public long serverId() -
sslMode
-
sslModeEnabled
public boolean sslModeEnabled() -
bufferSizeForStreamingChangeEventSource
public int bufferSizeForStreamingChangeEventSource() -
gtidSourceFilter
Get the predicate function that will returntrueif a GTID source is to be included, orfalseif a GTID source is to be excluded.- Returns:
- the GTID source predicate function; never null
-
includeSchemaChangeRecords
public boolean includeSchemaChangeRecords() -
includeSqlQuery
public boolean includeSqlQuery() -
rowCountForLargeTable
public long rowCountForLargeTable() -
getHistoryRecordComparator
- Specified by:
getHistoryRecordComparatorin classHistorizedRelationalDatabaseConnectorConfig
-
isBuiltInDatabase
-
isNotBuiltInTable
-
isReadOnlyConnection
public boolean isReadOnlyConnection() -
useGlobalLock
boolean useGlobalLock()Intended for testing only
-