Package io.debezium.connector.mongodb

Class MongoDbConnectorConfig

java.lang.Object

io.debezium.config.CommonConnectorConfig

io.debezium.connector.mongodb.MongoDbConnectorConfig

public class MongoDbConnectorConfig
extends CommonConnectorConfig

The configuration properties.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static enum	MongoDbConnectorConfig.CaptureMode	The set off different ways how connector can capture changes.
static enum	MongoDbConnectorConfig.ConnectionMode	The set of predefined MongoDbConnectionMode options or aliases.
static enum	MongoDbConnectorConfig.SnapshotMode	The set of predefined SnapshotMode options or aliases.

Nested classes/interfaces inherited from class io.debezium.config.CommonConnectorConfig

CommonConnectorConfig.BinaryHandlingMode, CommonConnectorConfig.EventProcessingFailureHandlingMode, CommonConnectorConfig.FieldNameAdjustmentMode, CommonConnectorConfig.SchemaNameAdjustmentMode, CommonConnectorConfig.Version

Field Summary

Fields

Modifier and Type	Field	Description
static Field.Set	ALL_FIELDS	The set of Fields defined as part of this configuration.
static final Field	AUTH_SOURCE
static final Field	AUTO_DISCOVER_MEMBERS	Deprecated.
static final Field	CAPTURE_MODE
private MongoDbConnectorConfig.CaptureMode	captureMode
static final Field	COLLECTION_EXCLUDE_LIST	A comma-separated list of regular expressions that match the fully-qualified namespaces of collections to be excluded from monitoring.
static final Field	COLLECTION_INCLUDE_LIST	A comma-separated list of regular expressions that match the fully-qualified namespaces of collections to be monitored.
protected static final String	COLLECTION_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG
private static final ConfigDefinition	CONFIG_DEFINITION
static final Field	CONNECT_TIMEOUT_MS
static final Field	CONNECTION_MODE
static final Field	CONNECTION_STRING
private final MongoDbConnectorConfig.ConnectionMode	connectionMode
static final Field	CURSOR_MAX_AWAIT_TIME_MS
static final Field	CURSOR_PIPELINE
private final int	cursorMaxAwaitTimeMs
static final Field	DATABASE_EXCLUDE_LIST	A comma-separated list of regular expressions that match the databases to be excluded.
static final Field	DATABASE_INCLUDE_LIST	A comma-separated list of regular expressions that match the databases to be monitored.
protected static final String	DATABASE_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG
protected static final int	DEFAULT_SNAPSHOT_FETCH_SIZE
protected static Field.Set	EXPOSED_FIELDS
static final Field	FIELD_EXCLUDE_LIST	A comma-separated list of the fully-qualified names of fields that should be excluded from change event message values.
protected static final Pattern	FIELD_EXCLUDE_LIST_PATTERN
static final Field	FIELD_RENAMES	A comma-separated list of the fully-qualified replacements of fields that should be used to rename fields in change event message values.
protected static final Pattern	FIELD_RENAMES_PATTERN
static final Field	HEARTBEAT_FREQUENCY_MS
static final Field	HOSTS	Deprecated.
private static final org.slf4j.Logger	LOGGER
static final Field	MONGODB_POLL_INTERVAL_MS
static final Field	PASSWORD
protected static final Pattern	PATTERN_SPILT
protected static final String	QUALIFIED_FIELD_EXCLUDE_LIST_PATTERN
protected static final String	QUALIFIED_FIELD_RENAMES_PATTERN
static final Field	REPLICA_SETS	The comma-separated list of replica set names
private final ReplicaSets	replicaSets
static final Field	SERVER_SELECTION_TIMEOUT_MS
static final Field	SNAPSHOT_FILTER_QUERY_BY_COLLECTION
static final Field	SNAPSHOT_MODE
private final int	snapshotMaxThreads
private final MongoDbConnectorConfig.SnapshotMode	snapshotMode
static final Field	SOCKET_TIMEOUT_MS
static final Field	SSL_ALLOW_INVALID_HOSTNAMES
static final Field	SSL_ENABLED
protected static final Field	TASK_ID
static final Field	TOPIC_NAMING_STRATEGY
static final Field	USER

Fields inherited from class io.debezium.config.CommonConnectorConfig

BINARY_HANDLING_MODE, CUSTOM_CONVERTERS, CUSTOM_RETRIABLE_EXCEPTION, DATABASE_CONFIG_PREFIX, DEFAULT_MAX_BATCH_SIZE, DEFAULT_MAX_QUEUE_SIZE, DEFAULT_MAX_QUEUE_SIZE_IN_BYTES, DEFAULT_POLL_INTERVAL_MILLIS, DEFAULT_QUERY_FETCH_SIZE, DEFAULT_RETRIABLE_RESTART_WAIT, DRIVER_CONFIG_PREFIX, EVENT_PROCESSING_FAILURE_HANDLING_MODE, FIELD_NAME_ADJUSTMENT_MODE, INCREMENTAL_SNAPSHOT_ALLOW_SCHEMA_CHANGES, INCREMENTAL_SNAPSHOT_CHUNK_SIZE, logicalName, MAX_BATCH_SIZE, MAX_QUEUE_SIZE, MAX_QUEUE_SIZE_IN_BYTES, MULTI_PARTITION_MODE, POLL_INTERVAL_MS, PROVIDE_TRANSACTION_METADATA, QUERY_FETCH_SIZE, RETRIABLE_RESTART_WAIT, SCHEMA_NAME_ADJUSTMENT_MODE, SIGNAL_DATA_COLLECTION, SKIPPED_OPERATIONS, SNAPSHOT_DELAY_MS, SNAPSHOT_FETCH_SIZE, SNAPSHOT_MAX_THREADS, SNAPSHOT_MODE_TABLES, TOMBSTONES_ON_DELETE, TOPIC_NAME_PATTERN, TOPIC_PREFIX

Constructor Summary

Constructors

Constructor	Description
MongoDbConnectorConfig(Configuration config)

Method Summary

Modifier and Type	Method	Description
static org.apache.kafka.common.config.ConfigDef	configDef()
MongoDbConnectorConfig.CaptureMode	getCaptureMode()	Provides statically configured capture mode.
MongoDbConnectorConfig.ConnectionMode	getConnectionMode()
String	getConnectorName()
String	getContextName()
int	getCursorMaxAwaitTime()
ReplicaSets	getReplicaSets()
Map<String,String>	getSnapshotFilterQueryByCollection()
Optional<String>	getSnapshotFilterQueryForCollection(CollectionId collectionId)
int	getSnapshotMaxThreads()
MongoDbConnectorConfig.SnapshotMode	getSnapshotMode()
protected SourceInfoStructMaker<? extends AbstractSourceInfo>	getSourceInfoStructMaker(CommonConnectorConfig.Version version)
boolean	isSignalDataCollection(io.debezium.spi.schema.DataCollectionId dataCollectionId)
Optional<String[]>	parseSignallingMessage(org.apache.kafka.connect.data.Struct value)
private static ReplicaSets	resolveReplicaSets(Configuration config, MongoDbConnectorConfig.ConnectionMode connectionMode)
private static int	resolveSnapshotMaxThreads(Configuration config)
boolean	supportsOperationFiltering()
private static int	validateChangeStreamPipeline(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateCollectionExcludeList(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateConnectionString(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateDatabaseExcludeList(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateFieldExcludeList(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateFieldRenamesList(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateHosts(Configuration config, Field field, Field.ValidationOutput problems)

Methods inherited from class io.debezium.config.CommonConnectorConfig

binaryHandlingMode, createHeartbeat, customConverterRegistry, customRetriableException, fieldNameAdjuster, getConfig, getDataCollectionsToBeSnapshotted, getEventProcessingFailureHandlingMode, getHeartbeatInterval, getHeartbeatTopicsPrefix, getIncrementalSnashotChunkSize, getLogicalName, getMaxBatchSize, getMaxQueueSize, getMaxQueueSizeInBytes, getMaxRetriesOnError, getPollInterval, getQueryFetchSize, getRetriableRestartWait, getSignalingDataCollectionId, getSkippedOperations, getSnapshotDelay, getSnapshotFetchSize, getSourceInfoStructMaker, getTaskId, getTopicNamingStrategy, getTopicNamingStrategy, isEmitTombstoneOnDelete, isIncrementalSnapshotSchemaChangesEnabled, isSchemaChangesHistoryEnabled, isSchemaCommentsHistoryEnabled, schemaNameAdjuster, shouldProvideTransactionMetadata, supportsSchemaChangesDuringIncrementalSnapshot, validate, validateAndRecord, validateSkippedOperation, validateTopicName

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOGGER

private static final org.slf4j.Logger LOGGER

COLLECTION_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG

protected static final String COLLECTION_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG

See Also:

• Constant Field Values

DATABASE_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG

protected static final String DATABASE_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG

See Also:

• Constant Field Values

PATTERN_SPILT

protected static final Pattern PATTERN_SPILT

FIELD_EXCLUDE_LIST_PATTERN

protected static final Pattern FIELD_EXCLUDE_LIST_PATTERN

QUALIFIED_FIELD_EXCLUDE_LIST_PATTERN

protected static final String QUALIFIED_FIELD_EXCLUDE_LIST_PATTERN

See Also:

• Constant Field Values

FIELD_RENAMES_PATTERN

protected static final Pattern FIELD_RENAMES_PATTERN

QUALIFIED_FIELD_RENAMES_PATTERN

protected static final String QUALIFIED_FIELD_RENAMES_PATTERN

See Also:

• Constant Field Values

DEFAULT_SNAPSHOT_FETCH_SIZE

protected static final int DEFAULT_SNAPSHOT_FETCH_SIZE

See Also:

• Constant Field Values

REPLICA_SETS

public static final Field REPLICA_SETS

The comma-separated list of replica set names

CONNECTION_STRING

public static final Field CONNECTION_STRING

CONNECTION_MODE

public static final Field CONNECTION_MODE

USER

public static final Field USER

PASSWORD

public static final Field PASSWORD

MONGODB_POLL_INTERVAL_MS

public static final Field MONGODB_POLL_INTERVAL_MS

SSL_ENABLED

public static final Field SSL_ENABLED

SSL_ALLOW_INVALID_HOSTNAMES

public static final Field SSL_ALLOW_INVALID_HOSTNAMES

CONNECT_TIMEOUT_MS

public static final Field CONNECT_TIMEOUT_MS

AUTH_SOURCE

public static final Field AUTH_SOURCE

SERVER_SELECTION_TIMEOUT_MS

public static final Field SERVER_SELECTION_TIMEOUT_MS

SOCKET_TIMEOUT_MS

public static final Field SOCKET_TIMEOUT_MS

HEARTBEAT_FREQUENCY_MS

public static final Field HEARTBEAT_FREQUENCY_MS

DATABASE_INCLUDE_LIST

public static final Field DATABASE_INCLUDE_LIST

A comma-separated list of regular expressions that match the databases to be monitored. Must not be used with DATABASE_EXCLUDE_LIST.

DATABASE_EXCLUDE_LIST

public static final Field DATABASE_EXCLUDE_LIST

A comma-separated list of regular expressions that match the databases to be excluded. Must not be used with DATABASE_INCLUDE_LIST.

COLLECTION_INCLUDE_LIST

public static final Field COLLECTION_INCLUDE_LIST

A comma-separated list of regular expressions that match the fully-qualified namespaces of collections to be monitored. Fully-qualified namespaces for collections are of the form <databaseName>.<collectionName>. Must not be used with COLLECTION_EXCLUDE_LIST.

COLLECTION_EXCLUDE_LIST

public static final Field COLLECTION_EXCLUDE_LIST

A comma-separated list of regular expressions that match the fully-qualified namespaces of collections to be excluded from monitoring. Fully-qualified namespaces for collections are of the form <databaseName>.<collectionName>. Must not be used with COLLECTION_INCLUDE_LIST.

FIELD_EXCLUDE_LIST

public static final Field FIELD_EXCLUDE_LIST

A comma-separated list of the fully-qualified names of fields that should be excluded from change event message values. Fully-qualified names for fields are of the form <databaseName>.<collectionName>.<fieldName>.<nestedFieldName>, where <databaseName> and <collectionName> may contain the wildcard (*) which matches any characters.

FIELD_RENAMES

public static final Field FIELD_RENAMES

A comma-separated list of the fully-qualified replacements of fields that should be used to rename fields in change event message values. Fully-qualified replacements for fields are of the form <databaseName>.<collectionName>.<fieldName>.<nestedFieldName>:<newNestedFieldName>, where <databaseName> and <collectionName> may contain the wildcard (*) which matches any characters, the colon character (:) is used to determine rename mapping of field.

CAPTURE_MODE

public static final Field CAPTURE_MODE

TASK_ID

protected static final Field TASK_ID

SNAPSHOT_MODE

public static final Field SNAPSHOT_MODE

SNAPSHOT_FILTER_QUERY_BY_COLLECTION

public static final Field SNAPSHOT_FILTER_QUERY_BY_COLLECTION

CURSOR_MAX_AWAIT_TIME_MS

public static final Field CURSOR_MAX_AWAIT_TIME_MS

CURSOR_PIPELINE

public static final Field CURSOR_PIPELINE

TOPIC_NAMING_STRATEGY

public static final Field TOPIC_NAMING_STRATEGY

HOSTS

@Deprecated
public static final Field HOSTS

Deprecated.

The comma-separated list of hostname and port pairs (in the form 'host' or 'host:port') of the MongoDB servers in the replica set.

AUTO_DISCOVER_MEMBERS

@Deprecated
public static final Field AUTO_DISCOVER_MEMBERS

Deprecated.

CONFIG_DEFINITION

private static final ConfigDefinition CONFIG_DEFINITION

ALL_FIELDS

public static Field.Set ALL_FIELDS

The set of Fields defined as part of this configuration.

EXPOSED_FIELDS

protected static Field.Set EXPOSED_FIELDS

snapshotMode

private final MongoDbConnectorConfig.SnapshotMode snapshotMode

captureMode

private MongoDbConnectorConfig.CaptureMode captureMode

connectionMode

private final MongoDbConnectorConfig.ConnectionMode connectionMode

snapshotMaxThreads

private final int snapshotMaxThreads

cursorMaxAwaitTimeMs

private final int cursorMaxAwaitTimeMs

replicaSets

private final ReplicaSets replicaSets

Constructor Details

MongoDbConnectorConfig

public MongoDbConnectorConfig(Configuration config)

Method Details

configDef

public static org.apache.kafka.common.config.ConfigDef configDef()

validateHosts

private static int validateHosts(Configuration config,
 Field field,
 Field.ValidationOutput problems)

validateChangeStreamPipeline

private static int validateChangeStreamPipeline(Configuration config,
 Field field,
 Field.ValidationOutput problems)

validateConnectionString

private static int validateConnectionString(Configuration config,
 Field field,
 Field.ValidationOutput problems)

validateFieldExcludeList

private static int validateFieldExcludeList(Configuration config,
 Field field,
 Field.ValidationOutput problems)

validateFieldRenamesList

private static int validateFieldRenamesList(Configuration config,
 Field field,
 Field.ValidationOutput problems)

validateCollectionExcludeList

private static int validateCollectionExcludeList(Configuration config,
 Field field,
 Field.ValidationOutput problems)

validateDatabaseExcludeList

private static int validateDatabaseExcludeList(Configuration config,
 Field field,
 Field.ValidationOutput problems)

getSnapshotMode

public MongoDbConnectorConfig.SnapshotMode getSnapshotMode()

getCaptureMode

public MongoDbConnectorConfig.CaptureMode getCaptureMode()

Provides statically configured capture mode. The configured value can be overrided upon connector start if offsets stored were created by a different capture mode. See MongoDbTaskContext.getCaptureMode()

Returns:

capture mode requested by configuration

getConnectionMode

public MongoDbConnectorConfig.ConnectionMode getConnectionMode()

getReplicaSets

public ReplicaSets getReplicaSets()

getCursorMaxAwaitTime

public int getCursorMaxAwaitTime()

getSnapshotMaxThreads

public int getSnapshotMaxThreads()

Overrides:

getSnapshotMaxThreads in class CommonConnectorConfig

getSourceInfoStructMaker

protected SourceInfoStructMaker<? extends AbstractSourceInfo> getSourceInfoStructMaker(CommonConnectorConfig.Version version)

Specified by:

getSourceInfoStructMaker in class CommonConnectorConfig

getSnapshotFilterQueryForCollection

public Optional<String> getSnapshotFilterQueryForCollection(CollectionId collectionId)

getSnapshotFilterQueryByCollection

public Map<String,String> getSnapshotFilterQueryByCollection()

supportsOperationFiltering

public boolean supportsOperationFiltering()

Overrides:

supportsOperationFiltering in class CommonConnectorConfig

getContextName

public String getContextName()

Specified by:

getContextName in class CommonConnectorConfig

getConnectorName

public String getConnectorName()

Specified by:

getConnectorName in class CommonConnectorConfig

resolveSnapshotMaxThreads

private static int resolveSnapshotMaxThreads(Configuration config)

resolveReplicaSets

private static ReplicaSets resolveReplicaSets(Configuration config,
 MongoDbConnectorConfig.ConnectionMode connectionMode)

parseSignallingMessage

public Optional<String[]> parseSignallingMessage(org.apache.kafka.connect.data.Struct value)

Overrides:

parseSignallingMessage in class CommonConnectorConfig

isSignalDataCollection

public boolean isSignalDataCollection(io.debezium.spi.schema.DataCollectionId dataCollectionId)

Overrides:

isSignalDataCollection in class CommonConnectorConfig