Package io.debezium.connector.binlog

Class BinlogOffsetContext<T extends BinlogSourceInfo>

java.lang.Object
io.debezium.pipeline.CommonOffsetContext<T>
io.debezium.connector.binlog.BinlogOffsetContext<T>
All Implemented Interfaces:
OffsetContext
public class BinlogOffsetContext<T extends BinlogSourceInfo> extends CommonOffsetContext<T>
A concrete implementation of the CommonOffsetContext for binlog-based connectors that tracks the current offset within the source database change stream. This reflects the committed offset state and the "source" information Struct in the change events.
Author:
Chris Cranford

  • Field Details

    • SNAPSHOT_COMPLETED_KEY

      public static final String SNAPSHOT_COMPLETED_KEY
      See Also:

    • EVENTS_TO_SKIP_OFFSET_KEY

      public static final String EVENTS_TO_SKIP_OFFSET_KEY
      See Also:

    • TIMESTAMP_KEY

      public static final String TIMESTAMP_KEY
      See Also:

    • GTID_SET_KEY

      public static final String GTID_SET_KEY
      See Also:

    • NON_GTID_TRANSACTION_ID_FORMAT

      public static final String NON_GTID_TRANSACTION_ID_FORMAT
      See Also:

    • sourceInfoSchema

      private final org.apache.kafka.connect.data.Schema sourceInfoSchema

    • transactionContext

      private final TransactionContext transactionContext

    • incrementalSnapshotContext

      private final IncrementalSnapshotContext<TableId> incrementalSnapshotContext

    • restartGtidSet

      private String restartGtidSet

    • currentGtidSet

      private String currentGtidSet

    • restartBinlogFilename

      private String restartBinlogFilename

    • restartBinlogPosition

      private long restartBinlogPosition

    • restartRowsToSkip

      private int restartRowsToSkip

    • restartEventsToSkip

      private long restartEventsToSkip

    • currentEventLengthInBytes

      private long currentEventLengthInBytes

    • inTransaction

      private boolean inTransaction

    • snapshotCompleted

      private boolean snapshotCompleted

    • transactionId

      private String transactionId

  • Constructor Details

  • Method Details

    • getOffset

      public Map<String,?> getOffset()

    • getSourceInfoSchema

      public org.apache.kafka.connect.data.Schema getSourceInfoSchema()

    • isSnapshotRunning

      public boolean isSnapshotRunning()

    • preSnapshotStart

      public void preSnapshotStart()

    • preSnapshotCompletion

      public void preSnapshotCompletion()

    • event

      public void event(io.debezium.spi.schema.DataCollectionId tableId, Instant timestamp)

    • getTransactionContext

      public TransactionContext getTransactionContext()

    • getIncrementalSnapshotContext

      public IncrementalSnapshotContext<?> getIncrementalSnapshotContext()

    • isSnapshotCompleted

      public boolean isSnapshotCompleted()

    • getTransactionId

      public String getTransactionId()

    • setInitialSkips

      public void setInitialSkips(long restartEventsToSkip, int restartRowsToSkip)

    • databaseEvent

      public void databaseEvent(String database, Instant timestamp)

    • tableEvent

      public void tableEvent(String database, Set<TableId> tableIds, Instant timestamp)

    • setBinlogStartPoint

      public void setBinlogStartPoint(String binlogFilename, long positionOfFirstEvent)
      Set the position in the MySQL binlog where we will start reading.
      Parameters:
      binlogFilename - the name of the binary log file; may not be null
      positionOfFirstEvent - the position in the binary log file to begin processing

    • setCompletedGtidSet

      public void setCompletedGtidSet(String gtidSet)
      Set the GTID set that captures all of the GTID transactions that have been completely processed.
      Parameters:
      gtidSet - the string representation of the GTID set; may not be null, but may be an empty string if no GTIDs have been previously processed

    • gtidSet

      public String gtidSet()
      Get the string representation of the GTID range for the MySQL binary log file.
      Returns:
      the string representation of the binlog GTID ranges; may be null

    • startGtid

      public void startGtid(String gtid, String gtidSet)
      Record that a new GTID transaction has been started and has been included in the set of GTIDs known to the MySQL server.
      Parameters:
      gtid - the string representation of a specific GTID that has been begun; may not be null
      gtidSet - the string representation of GTID set that includes the newly begun GTID; may not be null

    • getSource

      public T getSource()

    • startNextTransaction

      public void startNextTransaction()

    • commitTransaction

      public void commitTransaction()

    • completeEvent

      public void completeEvent()
      Capture that we're starting a new event.

    • setEventPosition

      public void setEventPosition(long positionOfCurrentEvent, long eventSizeInBytes)
      Set the position within the MySQL binary log file of the current event.
      Parameters:
      positionOfCurrentEvent - the position within the binary log file of the current event
      eventSizeInBytes - the size in bytes of this event

    • setQuery

      public void setQuery(String query)
      Set the original SQL query.
      Parameters:
      query - the original SQL query that generated the event.

    • changeEventCompleted

      public void changeEventCompleted()

    • eventsToSkipUponRestart

      public long eventsToSkipUponRestart()
      Get the number of events after the last transaction BEGIN that we've already processed.
      Returns:
      the number of events in the transaction that have been processed completely
      See Also:

    • rowsToSkipUponRestart

      public int rowsToSkipUponRestart()
      Get the number of rows beyond the last completely processed event to be skipped upon restart.
      Returns:
      the number of rows to be skipped

    • setRowNumber

      public void setRowNumber(int eventRowNumber, int totalNumberOfRows)
      Given the row number within a binlog event and the total number of rows in that event, compute the Kafka Connect offset that is to be included in the produced change event describing the row.

      This method should always be called before AbstractSourceInfo.struct().

      Parameters:
      eventRowNumber - the 0-based row number within the event for which the offset is to be produced
      totalNumberOfRows - the total number of rows within the event being processed
      See Also:

    • setBinlogServerId

      public void setBinlogServerId(long serverId)

    • setBinlogThread

      public void setBinlogThread(long threadId)

    • setTransactionId

      private void setTransactionId()

    • resetTransactionId

      private void resetTransactionId()

    • offsetUsingPosition

      private Map<String,Object> offsetUsingPosition(long rowsToSkip)

    • toString

      public String toString()
      Overrides:
      toString in class Object