Package io.debezium.connector.cassandra

Class FileOffsetWriter

java.lang.Object
io.debezium.connector.cassandra.FileOffsetWriter
All Implemented Interfaces:
OffsetWriter
public class FileOffsetWriter extends Object implements OffsetWriter
A concrete implementation of OffsetWriter which tracks the progress of events being processed by the SnapshotProcessor and respective commit log processor to property files, snapshot_offset.properties and commitlog_offset.properties, respectively. The property key is the table for the offset, and is serialized in the format of . The property value is the offset position, and is serialized in the format of :. For snapshots, a table is either fully processed or not processed at all, so offset is given a default value of ":-1" , where the filename is an empty string, and file position is -1. For commit logs, the file_name represents the commit log file name and file position represents bytes read in the commit log.

  • Field Details

    • LOGGER

      private static final org.slf4j.Logger LOGGER

    • SNAPSHOT_OFFSET_FILE

      public static final String SNAPSHOT_OFFSET_FILE
      See Also:

    • COMMITLOG_OFFSET_FILE

      public static final String COMMITLOG_OFFSET_FILE
      See Also:

    • snapshotProps

      private final Properties snapshotProps

    • commitLogProps

      private final Properties commitLogProps

    • snapshotOffsetFile

      private final File snapshotOffsetFile

    • commitLogOffsetFile

      private final File commitLogOffsetFile

    • snapshotOffsetFileLock

      private final FileLock snapshotOffsetFileLock

    • commitLogOffsetFileLock

      private final FileLock commitLogOffsetFileLock

    • offsetFlushPolicy

      private final OffsetFlushPolicy offsetFlushPolicy

    • executorService

      private final ExecutorService executorService

  • Constructor Details

  • Method Details

    • markOffset

      public Future<?> markOffset(String sourceTable, String sourceOffset, boolean isSnapshot)
      Description copied from interface: OffsetWriter
      Update the offset in memory if the provided offset is greater than the existing offset.
      Specified by:
      markOffset in interface OffsetWriter
      Parameters:
      sourceTable - string in the format of .
.
sourceOffset - string in the format of :
isSnapshot - whether the offset is coming from a snapshot or commit log

  • performMarkOffset

    private void performMarkOffset(String sourceTable, String sourceOffset, boolean isSnapshot)

  • isOffsetProcessed

    public boolean isOffsetProcessed(String sourceTable, String sourceOffset, boolean isSnapshot)
    Description copied from interface: OffsetWriter
    Determine if an offset has been processed based on the table name, offset position, and whether it is from snapshot or not.
    Specified by:
    isOffsetProcessed in interface OffsetWriter
    Parameters:
    sourceTable - string in the format of .
    • .
    sourceOffset - string in the format of :
    isSnapshot - whether the offset is coming from a snapshot or commit log
    Returns:
    true if the offset has been processed, false otherwise.

  • flush

    public Future<?> flush()
    Description copied from interface: OffsetWriter
    Flush latest offsets to disk.
    Specified by:
    flush in interface OffsetWriter

  • performFlush

    private void performFlush()

  • close

    public void close()
    Description copied from interface: OffsetWriter
    Close all resources used by this class.
    Specified by:
    close in interface OffsetWriter

  • saveOffset

    private static void saveOffset(File offsetFile, Properties props) throws IOException
    Throws:
    IOException

  • loadOffset

    private void loadOffset(File offsetFile, Properties props) throws IOException
    Throws:
    IOException

  • init

    private FileLock init(File offsetFile) throws IOException
    Throws:
    IOException

  • initLockPath

    private Path initLockPath(File offsetFile) throws IOException
    Throws:
    IOException