Package io.debezium.connector.mongodb

Class MongoDbSnapshotChangeEventSource

java.lang.Object

io.debezium.pipeline.source.AbstractSnapshotChangeEventSource<MongoDbPartition,MongoDbOffsetContext>

io.debezium.connector.mongodb.MongoDbSnapshotChangeEventSource

All Implemented Interfaces:

ChangeEventSource, SnapshotChangeEventSource<MongoDbPartition,MongoDbOffsetContext>

public class MongoDbSnapshotChangeEventSource
extends AbstractSnapshotChangeEventSource<MongoDbPartition,MongoDbOffsetContext>

A SnapshotChangeEventSource that performs multi-threaded snapshots of replica sets.

Author:

Chris Cranford

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
private static class	MongoDbSnapshotChangeEventSource.MongoDbSnapshotContext	Mutable context that is populated in the course of snapshotting.
static class	MongoDbSnapshotChangeEventSource.MongoDbSnapshottingTask	A configuration describing the task to be performed during snapshotting.

Nested classes/interfaces inherited from class io.debezium.pipeline.source.AbstractSnapshotChangeEventSource

AbstractSnapshotChangeEventSource.SnapshotContext<P extends Partition,O extends OffsetContext>, AbstractSnapshotChangeEventSource.SnapshottingTask

Nested classes/interfaces inherited from interface io.debezium.pipeline.source.spi.ChangeEventSource

ChangeEventSource.ChangeEventSourceContext

Field Summary

Fields

Modifier and Type	Field	Description
private AtomicBoolean	aborted
private static String	AUTHORIZATION_FAILURE_MESSAGE
protected Clock	clock
private ConnectionContext	connectionContext
private MongoDbConnectorConfig	connectorConfig
private EventDispatcher<CollectionId>	dispatcher
private ErrorHandler	errorHandler
private static org.slf4j.Logger	LOGGER
private ReplicaSets	replicaSets
private SnapshotProgressListener	snapshotProgressListener
private MongoDbTaskContext	taskContext

Fields inherited from class io.debezium.pipeline.source.AbstractSnapshotChangeEventSource

LOG_INTERVAL

Constructor Summary

Constructors

Constructor	Description
MongoDbSnapshotChangeEventSource(MongoDbConnectorConfig connectorConfig, MongoDbTaskContext taskContext, ReplicaSets replicaSets, EventDispatcher<CollectionId> dispatcher, Clock clock, SnapshotProgressListener snapshotProgressListener, ErrorHandler errorHandler)

Method Summary

Modifier and Type	Method	Description
private void	createDataEvents(ChangeEventSource.ChangeEventSourceContext sourceContext, MongoDbSnapshotChangeEventSource.MongoDbSnapshotContext snapshotContext, ReplicaSet replicaSet, ConnectionContext.MongoPrimary primaryClient)
private void	createDataEventsForCollection(ChangeEventSource.ChangeEventSourceContext sourceContext, MongoDbSnapshotChangeEventSource.MongoDbSnapshotContext snapshotContext, EventDispatcher.SnapshotReceiver snapshotReceiver, ReplicaSet replicaSet, CollectionId collectionId, ConnectionContext.MongoPrimary primaryClient)
private void	createDataEventsForReplicaSet(ChangeEventSource.ChangeEventSourceContext sourceContext, MongoDbSnapshotChangeEventSource.MongoDbSnapshotContext snapshotContext, EventDispatcher.SnapshotReceiver snapshotReceiver, ReplicaSet replicaSet, ConnectionContext.MongoPrimary primaryClient)	Dispatches the data change events for the records of a single replica-set.
protected void	determineSnapshotOffsets(MongoDbSnapshotChangeEventSource.MongoDbSnapshotContext ctx, ReplicaSets replicaSets)
protected SnapshotResult<MongoDbOffsetContext>	doExecute(ChangeEventSource.ChangeEventSourceContext context, MongoDbOffsetContext previousOffset, AbstractSnapshotChangeEventSource.SnapshotContext<MongoDbPartition,MongoDbOffsetContext> snapshotContext, AbstractSnapshotChangeEventSource.SnapshottingTask snapshottingTask)
private ConnectionContext.MongoPrimary	establishConnectionToPrimary(ReplicaSet replicaSet)
protected ChangeRecordEmitter	getChangeRecordEmitter(AbstractSnapshotChangeEventSource.SnapshotContext<MongoDbPartition,MongoDbOffsetContext> snapshotContext, CollectionId collectionId, org.bson.Document document, ReplicaSet replicaSet)
protected Clock	getClock()
protected AbstractSnapshotChangeEventSource.SnapshottingTask	getSnapshottingTask(MongoDbOffsetContext previousOffset)
private boolean	isSnapshotExpected(ConnectionContext.MongoPrimary primaryClient, ReplicaSetOffsetContext offsetContext)
protected AbstractSnapshotChangeEventSource.SnapshotContext<MongoDbPartition,MongoDbOffsetContext>	prepare(MongoDbPartition partition)
private void	snapshotReplicaSet(ChangeEventSource.ChangeEventSourceContext sourceContext, MongoDbSnapshotChangeEventSource.MongoDbSnapshotContext ctx, ReplicaSet replicaSet)

Methods inherited from class io.debezium.pipeline.source.AbstractSnapshotChangeEventSource

complete, delaySnapshotIfNeeded, determineDataCollectionsToBeSnapshotted, execute

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOGGER

private static final org.slf4j.Logger LOGGER

AUTHORIZATION_FAILURE_MESSAGE

private static final String AUTHORIZATION_FAILURE_MESSAGE

See Also:

Constant Field Values

connectorConfig

private final MongoDbConnectorConfig connectorConfig

taskContext

private final MongoDbTaskContext taskContext

connectionContext

private final ConnectionContext connectionContext

replicaSets

private final ReplicaSets replicaSets

dispatcher

private final EventDispatcher<CollectionId> dispatcher

clock

protected final Clock clock

snapshotProgressListener

private final SnapshotProgressListener snapshotProgressListener

errorHandler

private final ErrorHandler errorHandler

aborted

private AtomicBoolean aborted

Constructor Detail

MongoDbSnapshotChangeEventSource

public MongoDbSnapshotChangeEventSource(MongoDbConnectorConfig connectorConfig,
                                        MongoDbTaskContext taskContext,
                                        ReplicaSets replicaSets,
                                        EventDispatcher<CollectionId> dispatcher,
                                        Clock clock,
                                        SnapshotProgressListener snapshotProgressListener,
                                        ErrorHandler errorHandler)

Method Detail

doExecute

protected SnapshotResult<MongoDbOffsetContext> doExecute(ChangeEventSource.ChangeEventSourceContext context,
                                                         MongoDbOffsetContext previousOffset,
                                                         AbstractSnapshotChangeEventSource.SnapshotContext<MongoDbPartition,MongoDbOffsetContext> snapshotContext,
                                                         AbstractSnapshotChangeEventSource.SnapshottingTask snapshottingTask)
                                                  throws Exception

Specified by:

doExecute in class AbstractSnapshotChangeEventSource<MongoDbPartition,MongoDbOffsetContext>

Throws:

Exception

getSnapshottingTask

protected AbstractSnapshotChangeEventSource.SnapshottingTask getSnapshottingTask(MongoDbOffsetContext previousOffset)

Specified by:

getSnapshottingTask in class AbstractSnapshotChangeEventSource<MongoDbPartition,MongoDbOffsetContext>

prepare

protected AbstractSnapshotChangeEventSource.SnapshotContext<MongoDbPartition,MongoDbOffsetContext> prepare(MongoDbPartition partition)
                                                                                                          throws Exception

Specified by:

prepare in class AbstractSnapshotChangeEventSource<MongoDbPartition,MongoDbOffsetContext>

Throws:

Exception

snapshotReplicaSet

private void snapshotReplicaSet(ChangeEventSource.ChangeEventSourceContext sourceContext,
                                MongoDbSnapshotChangeEventSource.MongoDbSnapshotContext ctx,
                                ReplicaSet replicaSet)
                         throws InterruptedException

Throws:

InterruptedException

establishConnectionToPrimary

private ConnectionContext.MongoPrimary establishConnectionToPrimary(ReplicaSet replicaSet)

isSnapshotExpected

private boolean isSnapshotExpected(ConnectionContext.MongoPrimary primaryClient,
                                   ReplicaSetOffsetContext offsetContext)

determineSnapshotOffsets

protected void determineSnapshotOffsets(MongoDbSnapshotChangeEventSource.MongoDbSnapshotContext ctx,
                                        ReplicaSets replicaSets)

createDataEvents

private void createDataEvents(ChangeEventSource.ChangeEventSourceContext sourceContext,
                              MongoDbSnapshotChangeEventSource.MongoDbSnapshotContext snapshotContext,
                              ReplicaSet replicaSet,
                              ConnectionContext.MongoPrimary primaryClient)
                       throws InterruptedException

Throws:

InterruptedException

createDataEventsForReplicaSet

private void createDataEventsForReplicaSet(ChangeEventSource.ChangeEventSourceContext sourceContext,
                                           MongoDbSnapshotChangeEventSource.MongoDbSnapshotContext snapshotContext,
                                           EventDispatcher.SnapshotReceiver snapshotReceiver,
                                           ReplicaSet replicaSet,
                                           ConnectionContext.MongoPrimary primaryClient)
                                    throws InterruptedException

Dispatches the data change events for the records of a single replica-set.

Throws:

InterruptedException

createDataEventsForCollection

private void createDataEventsForCollection(ChangeEventSource.ChangeEventSourceContext sourceContext,
                                           MongoDbSnapshotChangeEventSource.MongoDbSnapshotContext snapshotContext,
                                           EventDispatcher.SnapshotReceiver snapshotReceiver,
                                           ReplicaSet replicaSet,
                                           CollectionId collectionId,
                                           ConnectionContext.MongoPrimary primaryClient)
                                    throws InterruptedException

Throws:

InterruptedException

getChangeRecordEmitter

protected ChangeRecordEmitter getChangeRecordEmitter(AbstractSnapshotChangeEventSource.SnapshotContext<MongoDbPartition,MongoDbOffsetContext> snapshotContext,
                                                     CollectionId collectionId,
                                                     org.bson.Document document,
                                                     ReplicaSet replicaSet)

getClock

protected Clock getClock()