Package io.debezium.connector.mongodb

Class MongoDbConnectorConfig

java.lang.Object

io.debezium.config.CommonConnectorConfig

io.debezium.connector.mongodb.MongoDbConnectorConfig

public class MongoDbConnectorConfig
extends CommonConnectorConfig

The configuration properties.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static enum	MongoDbConnectorConfig.CaptureMode	The set off different ways how connector can capture changes.
static enum	MongoDbConnectorConfig.SnapshotMode	The set of predefined SnapshotMode options or aliases.

Nested classes/interfaces inherited from class io.debezium.config.CommonConnectorConfig

CommonConnectorConfig.BinaryHandlingMode, CommonConnectorConfig.EventProcessingFailureHandlingMode, CommonConnectorConfig.SchemaNameAdjustmentMode, CommonConnectorConfig.Version

Field Summary

Fields

Modifier and Type	Field	Description
static Field.Set	ALL_FIELDS	The set of Fields defined as part of this configuration.
static final Field	AUTH_SOURCE
static final Field	AUTO_DISCOVER_MEMBERS
static final Field	CAPTURE_MODE
private MongoDbConnectorConfig.CaptureMode	captureMode
static final Field	COLLECTION_EXCLUDE_LIST	A comma-separated list of regular expressions that match the fully-qualified namespaces of collections to be excluded from monitoring.
static final Field	COLLECTION_INCLUDE_LIST	A comma-separated list of regular expressions that match the fully-qualified namespaces of collections to be monitored.
protected static final String	COLLECTION_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG
private static final ConfigDefinition	CONFIG_DEFINITION
static final Field	CONNECT_BACKOFF_INITIAL_DELAY_MS
static final Field	CONNECT_BACKOFF_MAX_DELAY_MS
static final Field	CONNECT_TIMEOUT_MS
static final Field	CURSOR_MAX_AWAIT_TIME_MS
private final int	cursorMaxAwaitTimeMs
static final Field	DATABASE_EXCLUDE_LIST	A comma-separated list of regular expressions that match the databases to be excluded.
static final Field	DATABASE_INCLUDE_LIST	A comma-separated list of regular expressions that match the databases to be monitored.
protected static final String	DATABASE_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG
protected static final int	DEFAULT_SNAPSHOT_FETCH_SIZE
protected static Field.Set	EXPOSED_FIELDS
static final Field	FIELD_EXCLUDE_LIST	A comma-separated list of the fully-qualified names of fields that should be excluded from change event message values.
protected static final Pattern	FIELD_EXCLUDE_LIST_PATTERN
static final Field	FIELD_RENAMES	A comma-separated list of the fully-qualified replacements of fields that should be used to rename fields in change event message values.
protected static final Pattern	FIELD_RENAMES_PATTERN
static final Field	HOSTS	The comma-separated list of hostname and port pairs (in the form 'host' or 'host:port') of the MongoDB servers in the replica set.
private static final org.slf4j.Logger	LOGGER
static final Field	LOGICAL_NAME
static final Field	MAX_FAILED_CONNECTIONS
static final Field	MONGODB_POLL_INTERVAL_MS
static final Field	PASSWORD
protected static final Pattern	PATTERN_SPILT
protected static final String	QUALIFIED_FIELD_EXCLUDE_LIST_PATTERN
protected static final String	QUALIFIED_FIELD_RENAMES_PATTERN
static final Field	SERVER_SELECTION_TIMEOUT_MS
static final Field	SNAPSHOT_FILTER_QUERY_BY_COLLECTION
static final Field	SNAPSHOT_MODE
private final int	snapshotMaxThreads
private final MongoDbConnectorConfig.SnapshotMode	snapshotMode
static final Field	SOCKET_TIMEOUT_MS
static final Field	SSL_ALLOW_INVALID_HOSTNAMES
static final Field	SSL_ENABLED
protected static final Field	TASK_ID
static final Field	TOPIC_NAMING_STRATEGY
static final Field	USER

Fields inherited from class io.debezium.config.CommonConnectorConfig

BINARY_HANDLING_MODE, CUSTOM_CONVERTERS, CUSTOM_RETRIABLE_EXCEPTION, DATABASE_CONFIG_PREFIX, DEFAULT_MAX_BATCH_SIZE, DEFAULT_MAX_QUEUE_SIZE, DEFAULT_MAX_QUEUE_SIZE_IN_BYTES, DEFAULT_POLL_INTERVAL_MILLIS, DEFAULT_QUERY_FETCH_SIZE, DEFAULT_RETRIABLE_RESTART_WAIT, EVENT_PROCESSING_FAILURE_HANDLING_MODE, INCREMENTAL_SNAPSHOT_ALLOW_SCHEMA_CHANGES, INCREMENTAL_SNAPSHOT_CHUNK_SIZE, MAX_BATCH_SIZE, MAX_QUEUE_SIZE, MAX_QUEUE_SIZE_IN_BYTES, MULTI_PARTITION_MODE, POLL_INTERVAL_MS, PROVIDE_TRANSACTION_METADATA, QUERY_FETCH_SIZE, RETRIABLE_RESTART_WAIT, SANITIZE_FIELD_NAMES, SCHEMA_NAME_ADJUSTMENT_MODE, SIGNAL_DATA_COLLECTION, SKIPPED_OPERATIONS, SNAPSHOT_DELAY_MS, SNAPSHOT_FETCH_SIZE, SNAPSHOT_MAX_THREADS, SNAPSHOT_MODE_TABLES, TOMBSTONES_ON_DELETE

Constructor Summary

Constructors

Constructor	Description
MongoDbConnectorConfig(Configuration config)

Method Summary

Modifier and Type	Method	Description
static org.apache.kafka.common.config.ConfigDef	configDef()
MongoDbConnectorConfig.CaptureMode	getCaptureMode()	Provides statically configured capture mode.
String	getConnectorName()
String	getContextName()
int	getCursorMaxAwaitTime()
Map<String,String>	getSnapshotFilterQueryByCollection()
Optional<String>	getSnapshotFilterQueryForCollection(CollectionId collectionId)
int	getSnapshotMaxThreads()
MongoDbConnectorConfig.SnapshotMode	getSnapshotMode()
protected SourceInfoStructMaker<? extends AbstractSourceInfo>	getSourceInfoStructMaker(CommonConnectorConfig.Version version)
boolean	isSignalDataCollection(io.debezium.spi.schema.DataCollectionId dataCollectionId)
Optional<String[]>	parseSignallingMessage(org.apache.kafka.connect.data.Struct value)
private static int	resolveSnapshotMaxThreads(Configuration config)
boolean	supportsOperationFiltering()
private static int	validateCollectionExcludeList(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateDatabaseExcludeList(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateFieldExcludeList(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateFieldRenamesList(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateHosts(Configuration config, Field field, Field.ValidationOutput problems)

Methods inherited from class io.debezium.config.CommonConnectorConfig

binaryHandlingMode, createHeartbeat, customConverterRegistry, customRetriableException, getConfig, getDataCollectionsToBeSnapshotted, getEventProcessingFailureHandlingMode, getHeartbeatInterval, getHeartbeatTopicsPrefix, getIncrementalSnashotChunkSize, getLogicalName, getMaxBatchSize, getMaxQueueSize, getMaxQueueSizeInBytes, getPollInterval, getQueryFetchSize, getRetriableRestartWait, getSanitizeFieldNames, getSignalingDataCollectionId, getSkippedOperations, getSnapshotDelay, getSnapshotFetchSize, getSourceInfoStructMaker, getTaskId, getTopicNamingStrategy, getTopicNamingStrategy, isEmitTombstoneOnDelete, isIncrementalSnapshotSchemaChangesEnabled, isSchemaChangesHistoryEnabled, isSchemaCommentsHistoryEnabled, legacyGetDataCollectionsToBeSnapshotted, schemaNameAdjustmentMode, shouldProvideTransactionMetadata, supportsSchemaChangesDuringIncrementalSnapshot, validate, validateAndRecord, validateSkippedOperation

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOGGER

private static final org.slf4j.Logger LOGGER

COLLECTION_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG

protected static final String COLLECTION_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG

See Also:

• Constant Field Values

DATABASE_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG

protected static final String DATABASE_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG

See Also:

• Constant Field Values

PATTERN_SPILT

protected static final Pattern PATTERN_SPILT

FIELD_EXCLUDE_LIST_PATTERN

protected static final Pattern FIELD_EXCLUDE_LIST_PATTERN

QUALIFIED_FIELD_EXCLUDE_LIST_PATTERN

protected static final String QUALIFIED_FIELD_EXCLUDE_LIST_PATTERN

See Also:

• Constant Field Values

FIELD_RENAMES_PATTERN

protected static final Pattern FIELD_RENAMES_PATTERN

QUALIFIED_FIELD_RENAMES_PATTERN

protected static final String QUALIFIED_FIELD_RENAMES_PATTERN

See Also:

• Constant Field Values

DEFAULT_SNAPSHOT_FETCH_SIZE

protected static final int DEFAULT_SNAPSHOT_FETCH_SIZE

See Also:

• Constant Field Values

HOSTS

public static final Field HOSTS

The comma-separated list of hostname and port pairs (in the form 'host' or 'host:port') of the MongoDB servers in the replica set.

LOGICAL_NAME

public static final Field LOGICAL_NAME

USER

public static final Field USER

PASSWORD

public static final Field PASSWORD

AUTH_SOURCE

public static final Field AUTH_SOURCE

MONGODB_POLL_INTERVAL_MS

public static final Field MONGODB_POLL_INTERVAL_MS

SSL_ENABLED

public static final Field SSL_ENABLED

SSL_ALLOW_INVALID_HOSTNAMES

public static final Field SSL_ALLOW_INVALID_HOSTNAMES

CONNECT_BACKOFF_INITIAL_DELAY_MS

public static final Field CONNECT_BACKOFF_INITIAL_DELAY_MS

CONNECT_BACKOFF_MAX_DELAY_MS

public static final Field CONNECT_BACKOFF_MAX_DELAY_MS

MAX_FAILED_CONNECTIONS

public static final Field MAX_FAILED_CONNECTIONS

AUTO_DISCOVER_MEMBERS

public static final Field AUTO_DISCOVER_MEMBERS

DATABASE_INCLUDE_LIST

public static final Field DATABASE_INCLUDE_LIST

A comma-separated list of regular expressions that match the databases to be monitored. Must not be used with DATABASE_EXCLUDE_LIST.

DATABASE_EXCLUDE_LIST

public static final Field DATABASE_EXCLUDE_LIST

A comma-separated list of regular expressions that match the databases to be excluded. Must not be used with DATABASE_INCLUDE_LIST.

COLLECTION_INCLUDE_LIST

public static final Field COLLECTION_INCLUDE_LIST

A comma-separated list of regular expressions that match the fully-qualified namespaces of collections to be monitored. Fully-qualified namespaces for collections are of the form <databaseName>.<collectionName>. Must not be used with COLLECTION_EXCLUDE_LIST.

COLLECTION_EXCLUDE_LIST

public static final Field COLLECTION_EXCLUDE_LIST

A comma-separated list of regular expressions that match the fully-qualified namespaces of collections to be excluded from monitoring. Fully-qualified namespaces for collections are of the form <databaseName>.<collectionName>. Must not be used with COLLECTION_INCLUDE_LIST.

FIELD_EXCLUDE_LIST

public static final Field FIELD_EXCLUDE_LIST

A comma-separated list of the fully-qualified names of fields that should be excluded from change event message values. Fully-qualified names for fields are of the form <databaseName>.<collectionName>.<fieldName>.<nestedFieldName>, where <databaseName> and <collectionName> may contain the wildcard (*) which matches any characters.

FIELD_RENAMES

public static final Field FIELD_RENAMES

A comma-separated list of the fully-qualified replacements of fields that should be used to rename fields in change event message values. Fully-qualified replacements for fields are of the form <databaseName>.<collectionName>.<fieldName>.<nestedFieldName>:<newNestedFieldName>, where <databaseName> and <collectionName> may contain the wildcard (*) which matches any characters, the colon character (:) is used to determine rename mapping of field.

SNAPSHOT_MODE

public static final Field SNAPSHOT_MODE

CAPTURE_MODE

public static final Field CAPTURE_MODE

CONNECT_TIMEOUT_MS

public static final Field CONNECT_TIMEOUT_MS

SERVER_SELECTION_TIMEOUT_MS

public static final Field SERVER_SELECTION_TIMEOUT_MS

SOCKET_TIMEOUT_MS

public static final Field SOCKET_TIMEOUT_MS

TASK_ID

protected static final Field TASK_ID

SNAPSHOT_FILTER_QUERY_BY_COLLECTION

public static final Field SNAPSHOT_FILTER_QUERY_BY_COLLECTION

CURSOR_MAX_AWAIT_TIME_MS

public static final Field CURSOR_MAX_AWAIT_TIME_MS

TOPIC_NAMING_STRATEGY

public static final Field TOPIC_NAMING_STRATEGY

CONFIG_DEFINITION

private static final ConfigDefinition CONFIG_DEFINITION

ALL_FIELDS

public static Field.Set ALL_FIELDS

The set of Fields defined as part of this configuration.

EXPOSED_FIELDS

protected static Field.Set EXPOSED_FIELDS

snapshotMode

private final MongoDbConnectorConfig.SnapshotMode snapshotMode

captureMode

private MongoDbConnectorConfig.CaptureMode captureMode

snapshotMaxThreads

private final int snapshotMaxThreads

cursorMaxAwaitTimeMs

private final int cursorMaxAwaitTimeMs

Constructor Details

MongoDbConnectorConfig

public MongoDbConnectorConfig(Configuration config)

Method Details

configDef

public static org.apache.kafka.common.config.ConfigDef configDef()

validateHosts

private static int validateHosts(Configuration config,
 Field field,
 Field.ValidationOutput problems)

validateFieldExcludeList

private static int validateFieldExcludeList(Configuration config,
 Field field,
 Field.ValidationOutput problems)

validateFieldRenamesList

private static int validateFieldRenamesList(Configuration config,
 Field field,
 Field.ValidationOutput problems)

validateCollectionExcludeList

private static int validateCollectionExcludeList(Configuration config,
 Field field,
 Field.ValidationOutput problems)

validateDatabaseExcludeList

private static int validateDatabaseExcludeList(Configuration config,
 Field field,
 Field.ValidationOutput problems)

getSnapshotMode

public MongoDbConnectorConfig.SnapshotMode getSnapshotMode()

getCaptureMode

public MongoDbConnectorConfig.CaptureMode getCaptureMode()

Provides statically configured capture mode. The configured value can be overrided upon connector start if offsets stored were created by a different capture mode. See MongoDbTaskContext.getCaptureMode()

Returns:

capture mode requested by configuration

getCursorMaxAwaitTime

public int getCursorMaxAwaitTime()

getSnapshotMaxThreads

public int getSnapshotMaxThreads()

Overrides:

getSnapshotMaxThreads in class CommonConnectorConfig

getSourceInfoStructMaker

protected SourceInfoStructMaker<? extends AbstractSourceInfo> getSourceInfoStructMaker(CommonConnectorConfig.Version version)

Specified by:

getSourceInfoStructMaker in class CommonConnectorConfig

getSnapshotFilterQueryForCollection

public Optional<String> getSnapshotFilterQueryForCollection(CollectionId collectionId)

getSnapshotFilterQueryByCollection

public Map<String,String> getSnapshotFilterQueryByCollection()

supportsOperationFiltering

public boolean supportsOperationFiltering()

Overrides:

supportsOperationFiltering in class CommonConnectorConfig

getContextName

public String getContextName()

Specified by:

getContextName in class CommonConnectorConfig

getConnectorName

public String getConnectorName()

Specified by:

getConnectorName in class CommonConnectorConfig

resolveSnapshotMaxThreads

private static int resolveSnapshotMaxThreads(Configuration config)

parseSignallingMessage

public Optional<String[]> parseSignallingMessage(org.apache.kafka.connect.data.Struct value)

Overrides:

parseSignallingMessage in class CommonConnectorConfig

isSignalDataCollection

public boolean isSignalDataCollection(io.debezium.spi.schema.DataCollectionId dataCollectionId)

Overrides:

isSignalDataCollection in class CommonConnectorConfig