Package io.debezium.connector.mysql.legacy

Class MySqlConnectorTask

java.lang.Object
org.apache.kafka.connect.source.SourceTask
io.debezium.connector.common.BaseSourceTask<MySqlPartition,MySqlOffsetContext>
io.debezium.connector.mysql.legacy.MySqlConnectorTask
All Implemented Interfaces:
org.apache.kafka.connect.connector.Task
@NotThreadSafe public final class MySqlConnectorTask extends BaseSourceTask<MySqlPartition,MySqlOffsetContext>
A Kafka Connect source task reads the MySQL binary log and generate the corresponding data change events.
Author:
Randall Hauch
See Also:

  • Field Details

    • logger

      private final org.slf4j.Logger logger

    • taskContext

      private volatile MySqlTaskContext taskContext

    • connectionContext

      private volatile MySqlJdbcContext connectionContext

    • readers

      private volatile ChainedReader readers

  • Constructor Details

  • Method Details

    • version

      public String version()

    • start

      public ChangeEventSourceCoordinator<MySqlPartition,MySqlOffsetContext> start(Configuration config)
      Specified by:
      start in class BaseSourceTask<MySqlPartition,MySqlOffsetContext>

    • getRestartOffset

      private Map<String,?> getRestartOffset(Map<String,?> storedOffset)
      Get the offset to restart the connector from. Normally, this is just the stored offset. However, if we were doing a parallel load with new tables, it's possible that the last committed offset is from reading the new tables, which could be beyond where we want to restart from (and restarting there could cause skipped events). To fix this, the new tables binlog reader records extra information in its offset to tell the connector where to restart from. If this extra information is present in the stored offset, that is the offset that is returned.
      Parameters:
      storedOffset - the stored offset.
      Returns:
      the offset to restart from.
      See Also:

    • createAndStartTaskContext

      private static MySqlTaskContext createAndStartTaskContext(Configuration config, Filters filters)

    • newTablesInConfig

      private boolean newTablesInConfig()
      Returns:
      true if new tables appear to have been added to the config, and false otherwise.

    • getNewFilters

      private static Filters getNewFilters(Map<String,?> offsets, Configuration config)
      Get the filters representing the tables that have been newly added to the config, but not those that previously existed in the config.
      Returns:
      Filters

    • getOldFilters

      private static Filters getOldFilters(Map<String,?> offsets, Configuration config)
      Get the filters representing those tables that previously existed in the config, but not those newly added to the config.
      Returns:
      Filters

    • getAllFilters

      private static Filters getAllFilters(Configuration config)
      Get the filters representing all tables represented by the config.
      Returns:
      Filters

    • doPoll

      public List<org.apache.kafka.connect.source.SourceRecord> doPoll() throws InterruptedException
      Specified by:
      doPoll in class BaseSourceTask<MySqlPartition,MySqlOffsetContext>
      Throws:
      InterruptedException

    • doStop

      protected void doStop()
      Specified by:
      doStop in class BaseSourceTask<MySqlPartition,MySqlOffsetContext>

    • getAllConfigurationFields

      protected Iterable<Field> getAllConfigurationFields()
      Specified by:
      getAllConfigurationFields in class BaseSourceTask<MySqlPartition,MySqlOffsetContext>

    • completeReaders

      protected void completeReaders(MySqlPartition partition)
      When the task is stopped, the readers may have additional work to perform before they actually stop and before all their records have been consumed via the BaseSourceTask.poll() method. This method signals that all of this has completed.

    • isBinlogAvailable

      protected boolean isBinlogAvailable()
      Determine whether the binlog position as set on the SourceInfo is available in the server.
      Returns:
      true if the server has the binlog coordinates, or false otherwise

    • earliestBinlogFilename

      protected String earliestBinlogFilename()
      Determine the earliest binlog filename that is still available in the server.
      Returns:
      the name of the earliest binlog filename, or null if there are none.

    • isBinlogRowImageFull

      protected boolean isBinlogRowImageFull()
      Determine whether the MySQL server has the binlog_row_image set to 'FULL'.
      Returns:
      true if the server's binlog_row_image is set to FULL, or false otherwise

    • isBinlogFormatRow

      protected boolean isBinlogFormatRow()
      Determine whether the MySQL server has the row-level binlog enabled.
      Returns:
      true if the server's binlog_format is set to ROW, or false otherwise