Package io.debezium.connector.mongodb

Class SourceInfo

java.lang.Object
io.debezium.connector.AbstractSourceInfo
io.debezium.connector.common.BaseSourceInfo
io.debezium.connector.mongodb.SourceInfo
@NotThreadSafe public final class SourceInfo extends BaseSourceInfo
Information about the source of information, which includes the partitions and offsets within those partitions. The MongoDB connector considers each MongoDB database a separate "partitions" (in Kafka Connect terminology), and each partition has an "offset" (if Kafka Connect terminology) that defines the position within that partition/database where the connector finds a particular record. As the connector uses the Kafka Connect framework to process each record in a partition, Kafka Connect keeps track of the most recent offset for that partition.

The source partition information identifies the particular MongoDB replica set and the connector's logical name of the MongoDB server. A JSON-like representation of the source partition for a database named "customers" hosted in a MongoDB replica set named "myMongoServer" is as follows: 

 {
     "server_id" : "myMongoServer",
     "replicaSetName" : "rs0"
 }

The source offset information describes the position within a particular partition of each record. Since each event in MongoDB's oplog is identified by a BSONTimestamp that tracks the time and the order of the event for that particular time (e.g., multiple events that occur at the same time will have unique orders), the offset includes the BSONTimetamp representation. (The event's h field is the unique ID for the operation, so this is also included in the offset.) And, if an initial sync is in progress, the offset will include the initsync field.

Here's a JSON-like representation of an example timestamp: 

 {
         "sec" = 1422998530,
         "ord" = 0,
         "h" = 398278732173914406,
         "initsync" = true
 }
Author:
Randall Hauch

  • Field Details

  • Constructor Details

  • Method Details

    • collectionId

      CollectionId collectionId()

    • position

      SourceInfo.Position position()

    • lastResumeToken

      public String lastResumeToken()

    • lastTimestamp

      public org.bson.BsonTimestamp lastTimestamp()

    • collectionEvent

      public void collectionEvent(CollectionId collectionId, long wallTime)
      Get a Struct representation of the source partition and offset information where we have last read. The Struct complies with the AbstractSourceInfo.schema() for the MongoDB connector.
      Parameters:
      collectionId - the event's collection identifier; may not be null
      See Also:

    • initEvent

      public void initEvent(com.mongodb.client.MongoChangeStreamCursor<com.mongodb.client.model.changestream.ChangeStreamDocument<org.bson.BsonDocument>> cursor)

    • noEvent

      public void noEvent(BufferingChangeStreamCursor.ResumableChangeStreamEvent<org.bson.BsonDocument> event)

    • noEvent

      public void noEvent(com.mongodb.client.MongoChangeStreamCursor<?> cursor)

    • noEvent

      public void noEvent(org.bson.BsonTimestamp timestamp)

    • noEvent

      private void noEvent(String resumeToken)

    • noEvent

      private void noEvent(SourceInfo.Position position)

    • changeStreamEvent

      public void changeStreamEvent(com.mongodb.client.model.changestream.ChangeStreamDocument<org.bson.BsonDocument> changeStreamEvent)

    • onEvent

      private void onEvent(CollectionId collectionId, SourceInfo.Position position, long wallTime)

    • hasPosition

      public boolean hasPosition()
      Determine whether we have previously recorded a MongoDB timestamp for the replica set.
      Returns:
      true if an offset has been recorded for the replica set, or false if the replica set has not yet been seen

    • setPosition

      public void setPosition(SourceInfo.Position position)

    • startInitialSnapshot

      public void startInitialSnapshot()
      Record that an initial sync has started for the given replica set.

    • stopInitialSnapshot

      public void stopInitialSnapshot()
      Record that an initial sync has stopped for the given replica set.

    • isSnapshotRunning

      public boolean isSnapshotRunning()
      Returns whether any replica sets are still running a snapshot.

    • timestamp

      protected Instant timestamp()
      Specified by:
      timestamp in class AbstractSourceInfo

    • snapshot

      public SnapshotRecord snapshot()
      Overrides:
      snapshot in class BaseSourceInfo

    • database

      protected String database()
      Specified by:
      database in class AbstractSourceInfo

    • wallTime

      long wallTime()

    • toString

      public String toString()
      Overrides:
      toString in class Object