io.debezium.connector.mongodb

Class MongoDbConnectorConfig

java.lang.Object

io.debezium.config.CommonConnectorConfig

io.debezium.connector.mongodb.MongoDbConnectorConfig

public class MongoDbConnectorConfig
extends CommonConnectorConfig

The configuration properties.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	MongoDbConnectorConfig.SnapshotMode The set of predefined SnapshotMode options or aliases.

Nested classes/interfaces inherited from class io.debezium.config.CommonConnectorConfig

CommonConnectorConfig.BinaryHandlingMode, CommonConnectorConfig.EventProcessingFailureHandlingMode, CommonConnectorConfig.Version

Field Summary

Fields

Modifier and Type	Field and Description
static Field.Set	ALL_FIELDS The set of Fields defined as part of this configuration.
static Field	AUTH_SOURCE
static Field	AUTO_DISCOVER_MEMBERS
static Field	COLLECTION_BLACKLIST Deprecated.
static Field	COLLECTION_EXCLUDE_LIST A comma-separated list of regular expressions that match the fully-qualified namespaces of collections to be excluded from monitoring.
static Field	COLLECTION_INCLUDE_LIST A comma-separated list of regular expressions that match the fully-qualified namespaces of collections to be monitored.
protected static String	COLLECTION_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG
static Field	COLLECTION_WHITELIST Deprecated.
protected static String	COLLECTION_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG
private static ConfigDefinition	CONFIG_DEFINITION
static Field	CONNECT_BACKOFF_INITIAL_DELAY_MS
static Field	CONNECT_BACKOFF_MAX_DELAY_MS
static Field	CONNECT_TIMEOUT_MS
static Field	DATABASE_BLACKLIST Deprecated.
static Field	DATABASE_EXCLUDE_LIST A comma-separated list of regular expressions that match the databases to be excluded.
static Field	DATABASE_INCLUDE_LIST A comma-separated list of regular expressions that match the databases to be monitored.
protected static String	DATABASE_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG
static Field	DATABASE_WHITELIST Deprecated.
protected static String	DATABASE_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG
protected static int	DEFAULT_SNAPSHOT_FETCH_SIZE
protected static Field.Set	EXPOSED_FIELDS
static Field	FIELD_BLACKLIST Deprecated.
static Field	FIELD_EXCLUDE_LIST A comma-separated list of the fully-qualified names of fields that should be excluded from change event message values.
static Field	FIELD_RENAMES A comma-separated list of the fully-qualified replacements of fields that should be used to rename fields in change event message values.
static Field	HOSTS The comma-separated list of hostname and port pairs (in the form 'host' or 'host:port') of the MongoDB servers in the replica set.
private static org.slf4j.Logger	LOGGER
static Field	LOGICAL_NAME
static Field	MAX_COPY_THREADS Deprecated.
static Field	MAX_FAILED_CONNECTIONS
static Field	MONGODB_POLL_INTERVAL_MS
static Field	PASSWORD
static Field	POLL_INTERVAL_SEC Deprecated.
static Field	SERVER_SELECTION_TIMEOUT_MS
static Field	SNAPSHOT_FILTER_QUERY_BY_COLLECTION
static Field	SNAPSHOT_MODE
private int	snapshotMaxThreads
private MongoDbConnectorConfig.SnapshotMode	snapshotMode
static Field	SOCKET_TIMEOUT_MS
static Field	SSL_ALLOW_INVALID_HOSTNAMES
static Field	SSL_ENABLED
protected static Field	TASK_ID
static Field	USER

Fields inherited from class io.debezium.config.CommonConnectorConfig

BINARY_HANDLING_MODE, CUSTOM_CONVERTERS, DATABASE_CONFIG_PREFIX, DEFAULT_MAX_BATCH_SIZE, DEFAULT_MAX_QUEUE_SIZE, DEFAULT_MAX_QUEUE_SIZE_IN_BYTES, DEFAULT_POLL_INTERVAL_MILLIS, DEFAULT_QUERY_FETCH_SIZE, DEFAULT_RETRIABLE_RESTART_WAIT, EVENT_PROCESSING_FAILURE_HANDLING_MODE, MAX_BATCH_SIZE, MAX_QUEUE_SIZE, MAX_QUEUE_SIZE_IN_BYTES, POLL_INTERVAL_MS, PROVIDE_TRANSACTION_METADATA, QUERY_FETCH_SIZE, RETRIABLE_RESTART_WAIT, SANITIZE_FIELD_NAMES, SIGNAL_DATA_COLLECTION, SKIPPED_OPERATIONS, SNAPSHOT_DELAY_MS, SNAPSHOT_FETCH_SIZE, SNAPSHOT_MAX_THREADS, SNAPSHOT_MODE_TABLES, SOURCE_STRUCT_MAKER_VERSION, TOMBSTONES_ON_DELETE

Constructor Summary

Constructors

Constructor and Description
MongoDbConnectorConfig(Configuration config)

Method Summary

Modifier and Type	Method and Description
static org.apache.kafka.common.config.ConfigDef	configDef()
String	getConnectorName()
String	getContextName()
Map<String,String>	getSnapshotFilterQueryByCollection()
Optional<String>	getSnapshotFilterQueryForCollection(CollectionId collectionId)
int	getSnapshotMaxThreads()
MongoDbConnectorConfig.SnapshotMode	getSnapshotMode()
protected SourceInfoStructMaker<? extends AbstractSourceInfo>	getSourceInfoStructMaker(CommonConnectorConfig.Version version)
private static int	resolveSnapshotMaxThreads(Configuration config)
private static int	validateCollectionBlacklist(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateCollectionExcludeList(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateDatabaseBlacklist(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateDatabaseExcludeList(Configuration config, Field field, Field.ValidationOutput problems)
private static int	validateHosts(Configuration config, Field field, Field.ValidationOutput problems)

Methods inherited from class io.debezium.config.CommonConnectorConfig

binaryHandlingMode, customConverterRegistry, getConfig, getDataCollectionsToBeSnapshotted, getEventProcessingFailureHandlingMode, getHeartbeatTopicsPrefix, getLogicalName, getMaxBatchSize, getMaxQueueSize, getMaxQueueSizeInBytes, getPollInterval, getQueryFetchSize, getRetriableRestartWait, getSanitizeFieldNames, getSignalingDataCollectionId, getSkippedOps, getSnapshotDelay, getSnapshotFetchSize, getSourceInfoStructMaker, isEmitTombstoneOnDelete, isSchemaChangesHistoryEnabled, shouldProvideTransactionMetadata, validate, validateAndRecord, validateServerNameIsDifferentFromHistoryTopicName

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOGGER

private static final org.slf4j.Logger LOGGER

COLLECTION_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG

protected static final String COLLECTION_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG

See Also:

Constant Field Values

COLLECTION_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG

protected static final String COLLECTION_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG

See Also:

Constant Field Values

DATABASE_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG

protected static final String DATABASE_INCLUDE_LIST_ALREADY_SPECIFIED_ERROR_MSG

See Also:

Constant Field Values

DATABASE_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG

protected static final String DATABASE_WHITELIST_ALREADY_SPECIFIED_ERROR_MSG

See Also:

Constant Field Values

DEFAULT_SNAPSHOT_FETCH_SIZE

protected static final int DEFAULT_SNAPSHOT_FETCH_SIZE

See Also:

Constant Field Values

HOSTS

public static final Field HOSTS

The comma-separated list of hostname and port pairs (in the form 'host' or 'host:port') of the MongoDB servers in the replica set.

LOGICAL_NAME

public static final Field LOGICAL_NAME

USER

public static final Field USER

PASSWORD

public static final Field PASSWORD

AUTH_SOURCE

public static final Field AUTH_SOURCE

POLL_INTERVAL_SEC

@Deprecated
public static final Field POLL_INTERVAL_SEC

Deprecated.

MONGODB_POLL_INTERVAL_MS

public static final Field MONGODB_POLL_INTERVAL_MS

SSL_ENABLED

public static final Field SSL_ENABLED

SSL_ALLOW_INVALID_HOSTNAMES

public static final Field SSL_ALLOW_INVALID_HOSTNAMES

MAX_COPY_THREADS

@Deprecated
public static final Field MAX_COPY_THREADS

Deprecated.

CONNECT_BACKOFF_INITIAL_DELAY_MS

public static final Field CONNECT_BACKOFF_INITIAL_DELAY_MS

CONNECT_BACKOFF_MAX_DELAY_MS

public static final Field CONNECT_BACKOFF_MAX_DELAY_MS

MAX_FAILED_CONNECTIONS

public static final Field MAX_FAILED_CONNECTIONS

AUTO_DISCOVER_MEMBERS

public static final Field AUTO_DISCOVER_MEMBERS

DATABASE_INCLUDE_LIST

public static final Field DATABASE_INCLUDE_LIST

A comma-separated list of regular expressions that match the databases to be monitored. Must not be used with DATABASE_EXCLUDE_LIST.

DATABASE_WHITELIST

@Deprecated
public static final Field DATABASE_WHITELIST

Deprecated.

Old, backwards-compatible "whitelist" property.

DATABASE_EXCLUDE_LIST

public static final Field DATABASE_EXCLUDE_LIST

A comma-separated list of regular expressions that match the databases to be excluded. Must not be used with DATABASE_INCLUDE_LIST.

DATABASE_BLACKLIST

@Deprecated
public static final Field DATABASE_BLACKLIST

Deprecated.

Old, backwards-compatible "blacklist" property.

COLLECTION_INCLUDE_LIST

public static final Field COLLECTION_INCLUDE_LIST

A comma-separated list of regular expressions that match the fully-qualified namespaces of collections to be monitored. Fully-qualified namespaces for collections are of the form <databaseName>.<collectionName>. Must not be used with COLLECTION_EXCLUDE_LIST.

COLLECTION_WHITELIST

@Deprecated
public static final Field COLLECTION_WHITELIST

Deprecated.

Old, backwards-compatible "whitelist" property.

COLLECTION_EXCLUDE_LIST

public static final Field COLLECTION_EXCLUDE_LIST

A comma-separated list of regular expressions that match the fully-qualified namespaces of collections to be excluded from monitoring. Fully-qualified namespaces for collections are of the form <databaseName>.<collectionName>. Must not be used with COLLECTION_INCLUDE_LIST.

COLLECTION_BLACKLIST

@Deprecated
public static final Field COLLECTION_BLACKLIST

Deprecated.

Old, backwards-compatible "blacklist" property.

FIELD_EXCLUDE_LIST

public static final Field FIELD_EXCLUDE_LIST

A comma-separated list of the fully-qualified names of fields that should be excluded from change event message values. Fully-qualified names for fields are of the form <databaseName>.<collectionName>.<fieldName>.<nestedFieldName>, where <databaseName> and <collectionName> may contain the wildcard (*) which matches any characters.

FIELD_BLACKLIST

@Deprecated
public static final Field FIELD_BLACKLIST

Deprecated.

Old, backwards-compatible "blacklist" property.

FIELD_RENAMES

public static final Field FIELD_RENAMES

A comma-separated list of the fully-qualified replacements of fields that should be used to rename fields in change event message values. Fully-qualified replacements for fields are of the form <databaseName>.<collectionName>.<fieldName>.<nestedFieldName>:<newNestedFieldName>, where <databaseName> and <collectionName> may contain the wildcard (*) which matches any characters, the colon character (:) is used to determine rename mapping of field.

SNAPSHOT_MODE

public static final Field SNAPSHOT_MODE

CONNECT_TIMEOUT_MS

public static final Field CONNECT_TIMEOUT_MS

SERVER_SELECTION_TIMEOUT_MS

public static final Field SERVER_SELECTION_TIMEOUT_MS

SOCKET_TIMEOUT_MS

public static final Field SOCKET_TIMEOUT_MS

TASK_ID

protected static final Field TASK_ID

SNAPSHOT_FILTER_QUERY_BY_COLLECTION

public static final Field SNAPSHOT_FILTER_QUERY_BY_COLLECTION

CONFIG_DEFINITION

private static final ConfigDefinition CONFIG_DEFINITION

ALL_FIELDS

public static Field.Set ALL_FIELDS

The set of Fields defined as part of this configuration.

EXPOSED_FIELDS

protected static Field.Set EXPOSED_FIELDS

snapshotMode

private final MongoDbConnectorConfig.SnapshotMode snapshotMode

snapshotMaxThreads

private final int snapshotMaxThreads

Constructor Detail

MongoDbConnectorConfig

public MongoDbConnectorConfig(Configuration config)

Method Detail

configDef

public static org.apache.kafka.common.config.ConfigDef configDef()

validateHosts

private static int validateHosts(Configuration config,
                                 Field field,
                                 Field.ValidationOutput problems)

validateCollectionExcludeList

private static int validateCollectionExcludeList(Configuration config,
                                                 Field field,
                                                 Field.ValidationOutput problems)

validateCollectionBlacklist

private static int validateCollectionBlacklist(Configuration config,
                                               Field field,
                                               Field.ValidationOutput problems)

validateDatabaseExcludeList

private static int validateDatabaseExcludeList(Configuration config,
                                               Field field,
                                               Field.ValidationOutput problems)

validateDatabaseBlacklist

private static int validateDatabaseBlacklist(Configuration config,
                                             Field field,
                                             Field.ValidationOutput problems)

getSnapshotMode

public MongoDbConnectorConfig.SnapshotMode getSnapshotMode()

getSnapshotMaxThreads

public int getSnapshotMaxThreads()

Overrides:

getSnapshotMaxThreads in class CommonConnectorConfig

getSourceInfoStructMaker

protected SourceInfoStructMaker<? extends AbstractSourceInfo> getSourceInfoStructMaker(CommonConnectorConfig.Version version)

Specified by:

getSourceInfoStructMaker in class CommonConnectorConfig

getSnapshotFilterQueryForCollection

public Optional<String> getSnapshotFilterQueryForCollection(CollectionId collectionId)

getSnapshotFilterQueryByCollection

public Map<String,String> getSnapshotFilterQueryByCollection()

getContextName

public String getContextName()

Specified by:

getContextName in class CommonConnectorConfig

getConnectorName

public String getConnectorName()

Specified by:

getConnectorName in class CommonConnectorConfig

resolveSnapshotMaxThreads

private static int resolveSnapshotMaxThreads(Configuration config)