Package io.debezium.connector.mongodb

Class MongoDbSnapshotChangeEventSource

java.lang.Object

io.debezium.pipeline.source.AbstractSnapshotChangeEventSource<MongoDbPartition,MongoDbOffsetContext>

io.debezium.connector.mongodb.MongoDbSnapshotChangeEventSource

All Implemented Interfaces:

ChangeEventSource, SnapshotChangeEventSource<MongoDbPartition,MongoDbOffsetContext>, AutoCloseable

public class MongoDbSnapshotChangeEventSource
extends AbstractSnapshotChangeEventSource<MongoDbPartition,MongoDbOffsetContext>

A SnapshotChangeEventSource that performs multithreaded snapshots of replica sets.

Author:

Chris Cranford

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
private static class	MongoDbSnapshotChangeEventSource.MongoDbSnapshotContext	Mutable context that is populated in the course of snapshotting.

Nested classes/interfaces inherited from class io.debezium.pipeline.source.AbstractSnapshotChangeEventSource

AbstractSnapshotChangeEventSource.SnapshotContext<P extends Partition,O extends OffsetContext>

Nested classes/interfaces inherited from interface io.debezium.pipeline.source.spi.ChangeEventSource

ChangeEventSource.ChangeEventSourceContext

Field Summary

Fields

Modifier and Type	Field	Description
private final Clock	clock
private final MongoDbConnectorConfig	connectorConfig
private final EventDispatcher<MongoDbPartition,CollectionId>	dispatcher
private final ErrorHandler	errorHandler
private static final org.slf4j.Logger	LOGGER
private final SnapshotProgressListener<MongoDbPartition>	snapshotProgressListener
private final SnapshotterService	snapshotterService
private final MongoDbTaskContext	taskContext

Fields inherited from class io.debezium.pipeline.source.AbstractSnapshotChangeEventSource

LOG_INTERVAL, notificationService

Constructor Summary

Constructors

Constructor	Description
MongoDbSnapshotChangeEventSource(MongoDbConnectorConfig connectorConfig, MongoDbTaskContext taskContext, EventDispatcher<MongoDbPartition,CollectionId> dispatcher, Clock clock, SnapshotProgressListener<MongoDbPartition> snapshotProgressListener, ErrorHandler errorHandler, NotificationService<MongoDbPartition,MongoDbOffsetContext> notificationService, SnapshotterService snapshotterService)

Method Summary

Modifier and Type	Method	Description
private void	createDataEvents(ChangeEventSource.ChangeEventSourceContext sourceContext, MongoDbSnapshotChangeEventSource.MongoDbSnapshotContext snapshotContext, EventDispatcher.SnapshotReceiver<MongoDbPartition> snapshotReceiver, MongoDbConnection mongo, SnapshottingTask snapshottingTask)	Dispatches the data change events for the records of a single replica-set.
private void	createDataEventsForCollection(ChangeEventSource.ChangeEventSourceContext sourceContext, MongoDbSnapshotChangeEventSource.MongoDbSnapshotContext snapshotContext, EventDispatcher.SnapshotReceiver<MongoDbPartition> snapshotReceiver, CollectionId collectionId, MongoDbConnection mongo, Map<String,String> snapshotFilterQueryForCollection)
protected <T extends io.debezium.spi.schema.DataCollectionId> Stream<T>	determineDataCollectionsToBeSnapshotted(Collection<T> allDataCollections, Set<Pattern> snapshotAllowedDataCollections)
private Optional<String>	determineSnapshotQuery(Map<String,String> snapshotFilterQueryForCollection, CollectionId collectionId)
protected SnapshotResult<MongoDbOffsetContext>	doExecute(ChangeEventSource.ChangeEventSourceContext context, MongoDbOffsetContext prevOffsetCtx, AbstractSnapshotChangeEventSource.SnapshotContext<MongoDbPartition,MongoDbOffsetContext> snapshotContext, SnapshottingTask snapshottingTask)
private void	doSnapshot(ChangeEventSource.ChangeEventSourceContext sourceCtx, MongoDbSnapshotChangeEventSource.MongoDbSnapshotContext snapshotCtx, SnapshottingTask snapshottingTask)
SnapshottingTask	getBlockingSnapshottingTask(MongoDbPartition partition, MongoDbOffsetContext previousOffset, SnapshotConfiguration snapshotConfiguration)
private ChangeRecordEmitter<MongoDbPartition>	getChangeRecordEmitter(AbstractSnapshotChangeEventSource.SnapshotContext<MongoDbPartition,MongoDbOffsetContext> snapshotContext, CollectionId collectionId, org.bson.BsonDocument document)
private Clock	getClock()
SnapshottingTask	getSnapshottingTask(MongoDbPartition partition, MongoDbOffsetContext previousOffset)
private void	initSnapshotStartOffsets(MongoDbSnapshotChangeEventSource.MongoDbSnapshotContext snapshotCtx)
private void	initSnapshotStartOffsets(MongoDbSnapshotChangeEventSource.MongoDbSnapshotContext snapshotCtx, MongoDbConnection mongo)
protected AbstractSnapshotChangeEventSource.SnapshotContext<MongoDbPartition,MongoDbOffsetContext>	prepare(MongoDbPartition partition, boolean onDemand)

Methods inherited from class io.debezium.pipeline.source.AbstractSnapshotChangeEventSource

aborted, close, completed, delaySnapshotIfNeeded, execute, getDataCollectionPattern, getOffsets

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOGGER

private static final org.slf4j.Logger LOGGER

connectorConfig

private final MongoDbConnectorConfig connectorConfig

taskContext

private final MongoDbTaskContext taskContext

dispatcher

private final EventDispatcher<MongoDbPartition,CollectionId> dispatcher

clock

private final Clock clock

snapshotProgressListener

private final SnapshotProgressListener<MongoDbPartition> snapshotProgressListener

errorHandler

private final ErrorHandler errorHandler

snapshotterService

private final SnapshotterService snapshotterService

Constructor Details

MongoDbSnapshotChangeEventSource

public MongoDbSnapshotChangeEventSource(MongoDbConnectorConfig connectorConfig,
 MongoDbTaskContext taskContext,
 EventDispatcher<MongoDbPartition,CollectionId> dispatcher,
 Clock clock,
 SnapshotProgressListener<MongoDbPartition> snapshotProgressListener,
 ErrorHandler errorHandler,
 NotificationService<MongoDbPartition,MongoDbOffsetContext> notificationService,
 SnapshotterService snapshotterService)

Method Details

doExecute

protected SnapshotResult<MongoDbOffsetContext> doExecute(ChangeEventSource.ChangeEventSourceContext context,
 MongoDbOffsetContext prevOffsetCtx,
 AbstractSnapshotChangeEventSource.SnapshotContext<MongoDbPartition,MongoDbOffsetContext> snapshotContext,
 SnapshottingTask snapshottingTask)

Specified by:

doExecute in class AbstractSnapshotChangeEventSource<MongoDbPartition,MongoDbOffsetContext>

getBlockingSnapshottingTask

public SnapshottingTask getBlockingSnapshottingTask(MongoDbPartition partition,
 MongoDbOffsetContext previousOffset,
 SnapshotConfiguration snapshotConfiguration)

getSnapshottingTask

public SnapshottingTask getSnapshottingTask(MongoDbPartition partition,
 MongoDbOffsetContext previousOffset)

prepare

protected AbstractSnapshotChangeEventSource.SnapshotContext<MongoDbPartition,MongoDbOffsetContext> prepare(MongoDbPartition partition,
 boolean onDemand)

Specified by:

prepare in class AbstractSnapshotChangeEventSource<MongoDbPartition,MongoDbOffsetContext>

doSnapshot

private void doSnapshot(ChangeEventSource.ChangeEventSourceContext sourceCtx,
 MongoDbSnapshotChangeEventSource.MongoDbSnapshotContext snapshotCtx,
 SnapshottingTask snapshottingTask)
                 throws InterruptedException

Throws:

InterruptedException

initSnapshotStartOffsets

private void initSnapshotStartOffsets(MongoDbSnapshotChangeEventSource.MongoDbSnapshotContext snapshotCtx)

initSnapshotStartOffsets

private void initSnapshotStartOffsets(MongoDbSnapshotChangeEventSource.MongoDbSnapshotContext snapshotCtx,
 MongoDbConnection mongo)
                               throws InterruptedException

Throws:

InterruptedException

createDataEvents

private void createDataEvents(ChangeEventSource.ChangeEventSourceContext sourceContext,
 MongoDbSnapshotChangeEventSource.MongoDbSnapshotContext snapshotContext,
 EventDispatcher.SnapshotReceiver<MongoDbPartition> snapshotReceiver,
 MongoDbConnection mongo,
 SnapshottingTask snapshottingTask)
                       throws InterruptedException

Dispatches the data change events for the records of a single replica-set.

Throws:

InterruptedException

determineDataCollectionsToBeSnapshotted

protected <T extends io.debezium.spi.schema.DataCollectionId>
Stream<T> determineDataCollectionsToBeSnapshotted(Collection<T> allDataCollections,
 Set<Pattern> snapshotAllowedDataCollections)

Overrides:

determineDataCollectionsToBeSnapshotted in class AbstractSnapshotChangeEventSource<MongoDbPartition,MongoDbOffsetContext>

createDataEventsForCollection

private void createDataEventsForCollection(ChangeEventSource.ChangeEventSourceContext sourceContext,
 MongoDbSnapshotChangeEventSource.MongoDbSnapshotContext snapshotContext,
 EventDispatcher.SnapshotReceiver<MongoDbPartition> snapshotReceiver,
 CollectionId collectionId,
 MongoDbConnection mongo,
 Map<String,String> snapshotFilterQueryForCollection)
                                    throws InterruptedException

Throws:

InterruptedException

determineSnapshotQuery

private Optional<String> determineSnapshotQuery(Map<String,String> snapshotFilterQueryForCollection,
 CollectionId collectionId)

getChangeRecordEmitter

private ChangeRecordEmitter<MongoDbPartition> getChangeRecordEmitter(AbstractSnapshotChangeEventSource.SnapshotContext<MongoDbPartition,MongoDbOffsetContext> snapshotContext,
 CollectionId collectionId,
 org.bson.BsonDocument document)

getClock

private Clock getClock()