Package io.debezium.connector.mongodb

Class MongoDbConnectorConfig

java.lang.Object

io.debezium.config.CommonConnectorConfig

io.debezium.connector.mongodb.MongoDbConnectorConfig

public class MongoDbConnectorConfig
extends CommonConnectorConfig

The configuration properties.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	MongoDbConnectorConfig.CaptureMode	The set off different ways how connector can capture changes.
static class	MongoDbConnectorConfig.SnapshotMode	The set of predefined SnapshotMode options or aliases.

Nested classes/interfaces inherited from class io.debezium.config.CommonConnectorConfig

CommonConnectorConfig.BinaryHandlingMode, CommonConnectorConfig.EventProcessingFailureHandlingMode, CommonConnectorConfig.Version

Field Summary

Fields

Modifier and Type	Field	Description
static Field.Set	ALL_FIELDS	The set of Fields defined as part of this configuration.
static Field	AUTH_SOURCE
static Field	AUTO_DISCOVER_MEMBERS
static Field	CAPTURE_MODE
private MongoDbConnectorConfig.CaptureMode	captureMode
static Field	COLLECTION_BLACKLIST	Deprecated.
static Field	COLLECTION_EXCLUDE_LIST	A comma-separated list of regular expressions that match the fully-qualified namespaces of collections to be excluded from monitoring.
static Field	COLLECTION_INCLUDE_LIST	A comma-separated list of regular expressions that match the fully-qualified namespaces of collections to be monitored.
protected static String	COLLECTION_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG
static Field	COLLECTION_WHITELIST	Deprecated.
protected static String	COLLECTION_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG
private static ConfigDefinition	CONFIG_DEFINITION
static Field	CONNECT_BACKOFF_INITIAL_DELAY_MS
static Field	CONNECT_BACKOFF_MAX_DELAY_MS
static Field	CONNECT_TIMEOUT_MS
static Field	CURSOR_MAX_AWAIT_TIME_MS
private int	cursorMaxAwaitTimeMs
static Field	DATABASE_BLACKLIST	Deprecated.
static Field	DATABASE_EXCLUDE_LIST	A comma-separated list of regular expressions that match the databases to be excluded.
static Field	DATABASE_INCLUDE_LIST	A comma-separated list of regular expressions that match the databases to be monitored.
protected static String	DATABASE_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG
static Field	DATABASE_WHITELIST	Deprecated.
protected static String	DATABASE_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG
protected static int	DEFAULT_SNAPSHOT_FETCH_SIZE
protected static Field.Set	EXPOSED_FIELDS
static Field	FIELD_BLACKLIST	Deprecated.
static Field	FIELD_EXCLUDE_LIST	A comma-separated list of the fully-qualified names of fields that should be excluded from change event message values.
protected static Pattern	FIELD_EXCLUDE_LIST_PATTERN
static Field	FIELD_RENAMES	A comma-separated list of the fully-qualified replacements of fields that should be used to rename fields in change event message values.
protected static Pattern	FIELD_RENAMES_PATTERN
static Field	HOSTS	The comma-separated list of hostname and port pairs (in the form 'host' or 'host:port') of the MongoDB servers in the replica set.
private static org.slf4j.Logger	LOGGER
static Field	LOGICAL_NAME
static Field	MAX_COPY_THREADS	Deprecated.
static Field	MAX_FAILED_CONNECTIONS
static Field	MONGODB_POLL_INTERVAL_MS
static Field	PASSWORD
protected static Pattern	PATTERN_SPILT
static Field	POLL_INTERVAL_SEC	Deprecated.
protected static String	QUALIFIED_FIELD_EXCLUDE_LIST_PATTERN
protected static String	QUALIFIED_FIELD_RENAMES_PATTERN
static Field	SERVER_SELECTION_TIMEOUT_MS
static Field	SNAPSHOT_FILTER_QUERY_BY_COLLECTION
static Field	SNAPSHOT_MODE
private int	snapshotMaxThreads
private MongoDbConnectorConfig.SnapshotMode	snapshotMode
static Field	SOCKET_TIMEOUT_MS
static Field	SSL_ALLOW_INVALID_HOSTNAMES
static Field	SSL_ENABLED
protected static Field	TASK_ID
static Field	USER

Fields inherited from class io.debezium.config.CommonConnectorConfig

BINARY_HANDLING_MODE, CUSTOM_CONVERTERS, CUSTOM_RETRIABLE_EXCEPTION, DATABASE_CONFIG_PREFIX, DEFAULT_MAX_BATCH_SIZE, DEFAULT_MAX_QUEUE_SIZE, DEFAULT_MAX_QUEUE_SIZE_IN_BYTES, DEFAULT_POLL_INTERVAL_MILLIS, DEFAULT_QUERY_FETCH_SIZE, DEFAULT_RETRIABLE_RESTART_WAIT, EVENT_PROCESSING_FAILURE_HANDLING_MODE, INCREMENTAL_SNAPSHOT_ALLOW_SCHEMA_CHANGES, INCREMENTAL_SNAPSHOT_CHUNK_SIZE, MAX_BATCH_SIZE, MAX_QUEUE_SIZE, MAX_QUEUE_SIZE_IN_BYTES, POLL_INTERVAL_MS, PROVIDE_TRANSACTION_METADATA, QUERY_FETCH_SIZE, RETRIABLE_RESTART_WAIT, SANITIZE_FIELD_NAMES, SIGNAL_DATA_COLLECTION, SKIPPED_OPERATIONS, SNAPSHOT_DELAY_MS, SNAPSHOT_FETCH_SIZE, SNAPSHOT_MAX_THREADS, SNAPSHOT_MODE_TABLES, SOURCE_STRUCT_MAKER_VERSION, TOMBSTONES_ON_DELETE, TRANSACTION_TOPIC

Constructor Summary

Constructors

Constructor	Description
MongoDbConnectorConfig(Configuration config)

Method Summary

Modifier and Type	Method	Description
static org.apache.kafka.common.config.ConfigDef	configDef()
MongoDbConnectorConfig.CaptureMode	getCaptureMode()	Provides statically configured capture mode.
String	getConnectorName()
String	getContextName()
int	getCursorMaxAwaitTime()
Map<String,String>	getSnapshotFilterQueryByCollection()
Optional<String>	getSnapshotFilterQueryForCollection(CollectionId collectionId)
int	getSnapshotMaxThreads()
MongoDbConnectorConfig.SnapshotMode	getSnapshotMode()
protected SourceInfoStructMaker<? extends AbstractSourceInfo>	getSourceInfoStructMaker(CommonConnectorConfig.Version version)
boolean	isSignalDataCollection(DataCollectionId dataCollectionId)
Optional<String[]>	parseSignallingMessage(org.apache.kafka.connect.data.Struct value)
private static int	resolveSnapshotMaxThreads(Configuration config)
boolean	supportsOperationFiltering()
private static int	validateCollectionBlacklist(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateCollectionExcludeList(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateDatabaseBlacklist(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateDatabaseExcludeList(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateFieldExcludeList(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateFieldRenamesList(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateHosts(Configuration config, Field field, Field.ValidationOutput problems)

Methods inherited from class io.debezium.config.CommonConnectorConfig

binaryHandlingMode, customConverterRegistry, customRetriableException, getConfig, getDataCollectionsToBeSnapshotted, getEventProcessingFailureHandlingMode, getHeartbeatInterval, getHeartbeatTopicsPrefix, getIncrementalSnashotChunkSize, getLogicalName, getMaxBatchSize, getMaxQueueSize, getMaxQueueSizeInBytes, getPollInterval, getQueryFetchSize, getRetriableRestartWait, getSanitizeFieldNames, getSignalingDataCollectionId, getSkippedOperations, getSnapshotDelay, getSnapshotFetchSize, getSourceInfoStructMaker, getTransactionTopic, isEmitTombstoneOnDelete, isIncrementalSnapshotSchemaChangesEnabled, isSchemaChangesHistoryEnabled, isSchemaCommentsHistoryEnabled, legacyGetDataCollectionsToBeSnapshotted, shouldProvideTransactionMetadata, supportsSchemaChangesDuringIncrementalSnapshot, validate, validateAndRecord, validateServerNameIsDifferentFromHistoryTopicName, validateSkippedOperation

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOGGER

private static final org.slf4j.Logger LOGGER

COLLECTION_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG

protected static final String COLLECTION_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG

See Also:

Constant Field Values

COLLECTION_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG

protected static final String COLLECTION_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG

See Also:

Constant Field Values

DATABASE_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG

protected static final String DATABASE_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG

See Also:

Constant Field Values

DATABASE_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG

protected static final String DATABASE_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG

See Also:

Constant Field Values

PATTERN_SPILT

protected static final Pattern PATTERN_SPILT

FIELD_EXCLUDE_LIST_PATTERN

protected static final Pattern FIELD_EXCLUDE_LIST_PATTERN

QUALIFIED_FIELD_EXCLUDE_LIST_PATTERN

protected static final String QUALIFIED_FIELD_EXCLUDE_LIST_PATTERN

See Also:

Constant Field Values

FIELD_RENAMES_PATTERN

protected static final Pattern FIELD_RENAMES_PATTERN

QUALIFIED_FIELD_RENAMES_PATTERN

protected static final String QUALIFIED_FIELD_RENAMES_PATTERN

See Also:

Constant Field Values

DEFAULT_SNAPSHOT_FETCH_SIZE

protected static final int DEFAULT_SNAPSHOT_FETCH_SIZE

See Also:

Constant Field Values

HOSTS

public static final Field HOSTS

The comma-separated list of hostname and port pairs (in the form 'host' or 'host:port') of the MongoDB servers in the replica set.

LOGICAL_NAME

public static final Field LOGICAL_NAME

USER

public static final Field USER

PASSWORD

public static final Field PASSWORD

AUTH_SOURCE

public static final Field AUTH_SOURCE

POLL_INTERVAL_SEC

@Deprecated
public static final Field POLL_INTERVAL_SEC

Deprecated.

MONGODB_POLL_INTERVAL_MS

public static final Field MONGODB_POLL_INTERVAL_MS

SSL_ENABLED

public static final Field SSL_ENABLED

SSL_ALLOW_INVALID_HOSTNAMES

public static final Field SSL_ALLOW_INVALID_HOSTNAMES

MAX_COPY_THREADS

@Deprecated
public static final Field MAX_COPY_THREADS

Deprecated.

CONNECT_BACKOFF_INITIAL_DELAY_MS

public static final Field CONNECT_BACKOFF_INITIAL_DELAY_MS

CONNECT_BACKOFF_MAX_DELAY_MS

public static final Field CONNECT_BACKOFF_MAX_DELAY_MS

MAX_FAILED_CONNECTIONS

public static final Field MAX_FAILED_CONNECTIONS

AUTO_DISCOVER_MEMBERS

public static final Field AUTO_DISCOVER_MEMBERS

DATABASE_INCLUDE_LIST

public static final Field DATABASE_INCLUDE_LIST

A comma-separated list of regular expressions that match the databases to be monitored. Must not be used with DATABASE_EXCLUDE_LIST.

DATABASE_WHITELIST

@Deprecated
public static final Field DATABASE_WHITELIST

Deprecated.

Old, backwards-compatible "whitelist" property.

DATABASE_EXCLUDE_LIST

public static final Field DATABASE_EXCLUDE_LIST

A comma-separated list of regular expressions that match the databases to be excluded. Must not be used with DATABASE_INCLUDE_LIST.

DATABASE_BLACKLIST

@Deprecated
public static final Field DATABASE_BLACKLIST

Deprecated.

Old, backwards-compatible "blacklist" property.

COLLECTION_INCLUDE_LIST

public static final Field COLLECTION_INCLUDE_LIST

A comma-separated list of regular expressions that match the fully-qualified namespaces of collections to be monitored. Fully-qualified namespaces for collections are of the form <databaseName>.<collectionName>. Must not be used with COLLECTION_EXCLUDE_LIST.

COLLECTION_WHITELIST

@Deprecated
public static final Field COLLECTION_WHITELIST

Deprecated.

Old, backwards-compatible "whitelist" property.

COLLECTION_EXCLUDE_LIST

public static final Field COLLECTION_EXCLUDE_LIST

A comma-separated list of regular expressions that match the fully-qualified namespaces of collections to be excluded from monitoring. Fully-qualified namespaces for collections are of the form <databaseName>.<collectionName>. Must not be used with COLLECTION_INCLUDE_LIST.

COLLECTION_BLACKLIST

@Deprecated
public static final Field COLLECTION_BLACKLIST

Deprecated.

Old, backwards-compatible "blacklist" property.

FIELD_EXCLUDE_LIST

public static final Field FIELD_EXCLUDE_LIST

A comma-separated list of the fully-qualified names of fields that should be excluded from change event message values. Fully-qualified names for fields are of the form <databaseName>.<collectionName>.<fieldName>.<nestedFieldName>, where <databaseName> and <collectionName> may contain the wildcard (*) which matches any characters.

FIELD_BLACKLIST

@Deprecated
public static final Field FIELD_BLACKLIST

Deprecated.

Old, backwards-compatible "blacklist" property.

FIELD_RENAMES

public static final Field FIELD_RENAMES

A comma-separated list of the fully-qualified replacements of fields that should be used to rename fields in change event message values. Fully-qualified replacements for fields are of the form <databaseName>.<collectionName>.<fieldName>.<nestedFieldName>:<newNestedFieldName>, where <databaseName> and <collectionName> may contain the wildcard (*) which matches any characters, the colon character (:) is used to determine rename mapping of field.

SNAPSHOT_MODE

public static final Field SNAPSHOT_MODE

CAPTURE_MODE

public static final Field CAPTURE_MODE

CONNECT_TIMEOUT_MS

public static final Field CONNECT_TIMEOUT_MS

SERVER_SELECTION_TIMEOUT_MS

public static final Field SERVER_SELECTION_TIMEOUT_MS

SOCKET_TIMEOUT_MS

public static final Field SOCKET_TIMEOUT_MS

TASK_ID

protected static final Field TASK_ID

SNAPSHOT_FILTER_QUERY_BY_COLLECTION

public static final Field SNAPSHOT_FILTER_QUERY_BY_COLLECTION

CURSOR_MAX_AWAIT_TIME_MS

public static final Field CURSOR_MAX_AWAIT_TIME_MS

CONFIG_DEFINITION

private static final ConfigDefinition CONFIG_DEFINITION

ALL_FIELDS

public static Field.Set ALL_FIELDS

The set of Fields defined as part of this configuration.

EXPOSED_FIELDS

protected static Field.Set EXPOSED_FIELDS

snapshotMode

private final MongoDbConnectorConfig.SnapshotMode snapshotMode

captureMode

private MongoDbConnectorConfig.CaptureMode captureMode

snapshotMaxThreads

private final int snapshotMaxThreads

cursorMaxAwaitTimeMs

private final int cursorMaxAwaitTimeMs

Constructor Detail

MongoDbConnectorConfig

public MongoDbConnectorConfig(Configuration config)

Method Detail

configDef

public static org.apache.kafka.common.config.ConfigDef configDef()

validateHosts

private static int validateHosts(Configuration config,
                                 Field field,
                                 Field.ValidationOutput problems)

validateFieldExcludeList

private static int validateFieldExcludeList(Configuration config,
                                            Field field,
                                            Field.ValidationOutput problems)

validateFieldRenamesList

private static int validateFieldRenamesList(Configuration config,
                                            Field field,
                                            Field.ValidationOutput problems)

validateCollectionExcludeList

private static int validateCollectionExcludeList(Configuration config,
                                                 Field field,
                                                 Field.ValidationOutput problems)

validateCollectionBlacklist

private static int validateCollectionBlacklist(Configuration config,
                                               Field field,
                                               Field.ValidationOutput problems)

validateDatabaseExcludeList

private static int validateDatabaseExcludeList(Configuration config,
                                               Field field,
                                               Field.ValidationOutput problems)

validateDatabaseBlacklist

private static int validateDatabaseBlacklist(Configuration config,
                                             Field field,
                                             Field.ValidationOutput problems)

getSnapshotMode

public MongoDbConnectorConfig.SnapshotMode getSnapshotMode()

getCaptureMode

public MongoDbConnectorConfig.CaptureMode getCaptureMode()

Provides statically configured capture mode. The configured value can be overrided upon connector start if offsets stored were created by a different capture mode. See MongoDbTaskContext.getCaptureMode()

Returns:

capture mode requested by configuration

getCursorMaxAwaitTime

public int getCursorMaxAwaitTime()

getSnapshotMaxThreads

public int getSnapshotMaxThreads()

Overrides:

getSnapshotMaxThreads in class CommonConnectorConfig

getSourceInfoStructMaker

protected SourceInfoStructMaker<? extends AbstractSourceInfo> getSourceInfoStructMaker(CommonConnectorConfig.Version version)

Specified by:

getSourceInfoStructMaker in class CommonConnectorConfig

getSnapshotFilterQueryForCollection

public Optional<String> getSnapshotFilterQueryForCollection(CollectionId collectionId)

getSnapshotFilterQueryByCollection

public Map<String,String> getSnapshotFilterQueryByCollection()

supportsOperationFiltering

public boolean supportsOperationFiltering()

Overrides:

supportsOperationFiltering in class CommonConnectorConfig

getContextName

public String getContextName()

Specified by:

getContextName in class CommonConnectorConfig

getConnectorName

public String getConnectorName()

Specified by:

getConnectorName in class CommonConnectorConfig

resolveSnapshotMaxThreads

private static int resolveSnapshotMaxThreads(Configuration config)

parseSignallingMessage

public Optional<String[]> parseSignallingMessage(org.apache.kafka.connect.data.Struct value)

Overrides:

parseSignallingMessage in class CommonConnectorConfig

isSignalDataCollection

public boolean isSignalDataCollection(DataCollectionId dataCollectionId)

Overrides:

isSignalDataCollection in class CommonConnectorConfig