Package io.debezium.connector.binlog

Class BinlogDatabaseSchema<P extends BinlogPartition,O extends BinlogOffsetContext<?>,V extends ValueConverterProvider,D extends DefaultValueConverter>

java.lang.Object
io.debezium.relational.RelationalDatabaseSchema
io.debezium.relational.HistorizedRelationalDatabaseSchema
io.debezium.connector.binlog.BinlogDatabaseSchema<P,O,V,D>
All Implemented Interfaces:
DatabaseSchema<TableId>, HistorizedDatabaseSchema<TableId>, AutoCloseable
public abstract class BinlogDatabaseSchema<P extends BinlogPartition,O extends BinlogOffsetContext<?>,V extends ValueConverterProvider,D extends DefaultValueConverter> extends HistorizedRelationalDatabaseSchema
Abstract implementation that records schema history for binlog-based databases.

The schema information includes the Tables and the Kafka Connect RelationalDatabaseSchema.schemaFor(TableId) for each table, where the Schema excludes any columns that have been specified in the RelationalDatabaseConnectorConfig.COLUMN_EXCLUDE_LIST configuration.
Author:
Chris Cranford

  • Field Details

    • LOGGER

      private static final org.slf4j.Logger LOGGER

    • ignoredQueryStatements

      private final Set<String> ignoredQueryStatements

    • ddlParser

      private final DdlParser ddlParser

    • filters

      private final RelationalTableFilters filters

    • ddlChanges

      private final DdlChanges ddlChanges

    • tableIdsByTableNumber

      private final Map<Long,TableId> tableIdsByTableNumber

    • excludeTableIdsByTableNumber

      private final Map<Long,TableId> excludeTableIdsByTableNumber

    • connectorConfig

      private final BinlogConnectorConfig connectorConfig

  • Constructor Details

    • BinlogDatabaseSchema

      public BinlogDatabaseSchema(BinlogConnectorConfig connectorConfig, V valueConverter, D defaultValueConverter, io.debezium.spi.topic.TopicNamingStrategy<TableId> topicNamingStrategy, SchemaNameAdjuster schemaNameAdjuster, boolean tableIdCaseInsensitive)
      Creates a binlog-connector based relational schema based on the supplied configuration. The DDL statements passed to the schema are parsed and a logical model is maintained of the database.
      Parameters:
      connectorConfig - the connector configuration, should not be null
      valueConverter - the connector-specific value converter, should not be null
      defaultValueConverter - the connector-specific default value converter, should not be null
      topicNamingStrategy - the topic naming strategy to be used, should not be null
      schemaNameAdjuster - the schema name adjuster, should not be null
      tableIdCaseInsensitive - whether table identifiers are case-insensitive

  • Method Details

    • getDdlParser

      protected DdlParser getDdlParser()
      Specified by:
      getDdlParser in class HistorizedRelationalDatabaseSchema

    • skipSchemaChangeEvent

      public boolean skipSchemaChangeEvent(SchemaChangeEvent event)
      Overrides:
      skipSchemaChangeEvent in class HistorizedRelationalDatabaseSchema

    • applySchemaChange

      public void applySchemaChange(SchemaChangeEvent schemaChange)

    • capturedTablesAsStringArray

      public String[] capturedTablesAsStringArray()
      Get all table names for all databases that are captured.
      Returns:
      array of table names

    • setSystemVariables

      public void setSystemVariables(SystemVariables.Scope scope, Map<String,String> variables)
      Sets system variables with use by the DDL parser.
      Parameters:
      scope - the variable scope
      variables - system variables; may not be null but can be empty

    • systemVariables

      public SystemVariables systemVariables()
      Get all system variables known to the DDL parser.
      Returns:
      the system variables; never null

    • isGlobalSetVariableStatement

      public boolean isGlobalSetVariableStatement(String ddl, String databaseName)
      Checks whether the provided DDL is a global set variable statement.
      Parameters:
      ddl - the DDL to check; may be null or empty
      databaseName - the database name; may be null or empty
      Returns:
      true if the statement is a global set variable statement; false otherwise

    • assignTableNumber

      public boolean assignTableNumber(long tableNumber, TableId id)
      Assign the given table number to the table with the specified TableId.
      Parameters:
      tableNumber - the table number found in binlog events
      id - the identifier for the corresponding table
      Returns:
      true if the assignment was successful, false if the table is excluded via configuration

    • getTableId

      public TableId getTableId(long tableNumber)
      Return the table id associated with connector-specific table number.
      Parameters:
      tableNumber - the table number from binlog
      Returns:
      the table id or null if not known

    • getExcludeTableId

      public TableId getExcludeTableId(long tableNumber)
      Return the excluded table id associated with connector-specific table number.
      Parameters:
      tableNumber - the table number from binlog
      Returns:
      the table id or null if not known

    • clearTableMappings

      public void clearTableMappings()
      Clear all table mappings.

      This should be done when the logs are rotated, since in that a different table, numbering scheme will be used by all subsequent TABLE_MAP binlog events.

    • parseSnapshotDdl

      public List<SchemaChangeEvent> parseSnapshotDdl(P partition, String ddlStatements, String databaseName, O offset, Instant sourceTime)
      Parse DDL from the connector's snapshot phase.
      Parameters:
      partition - the partition; should not be null
      ddlStatements - the DDL statement to be parsed
      databaseName - the database name
      offset - the connector offsets; should not be null
      sourceTime - time the DDL happened, should not be null
      Returns:
      list of parsed schema changes

    • parseStreamingDdl

      public List<SchemaChangeEvent> parseStreamingDdl(P partition, String ddlStatements, String databaseName, O offset, Instant sourceTime)
      Parse DDL from the connector's streaming phase.
      Parameters:
      partition - the partition; should not be null
      ddlStatements - the DDL statement to be parsed
      databaseName - the database name
      offset - the connector offsets; should not be null
      sourceTime - time the DDL happened, should not be null
      Returns:
      list of parsed schema changes

    • createDdlParser

      protected abstract DdlParser createDdlParser(BinlogConnectorConfig connectorConfig, V valueConverter)
      Creates the connector-specific DDL parser instance.
      Parameters:
      connectorConfig - the binlog connector configuration, never null
      valueConverter - the value converter, never null
      Returns:
      the connector's DDL parser instance, never null

    • handleTableEvent

      protected void handleTableEvent(O offset, String databaseName, Set<TableId> tableIds, Instant changeTime)
      Update offsets based on a table-specific event.
      Parameters:
      offset - the offsets to be mutated; should not be null
      databaseName - the database name
      tableIds - set of table identifiers
      changeTime - the time the event happened

    • handleDatabaseEvent

      protected void handleDatabaseEvent(O offset, String databaseName, Instant changeTime)
      Update offsets based on a database-specific event.
      Parameters:
      offset - the offsets to be mutated; should not be null
      databaseName - the database name
      changeTime - the time the event happened

    • refreshSchemas

      protected void refreshSchemas()
      Discards any currently-cached schemas and rebuild using filters

    • parseDdl

      private List<SchemaChangeEvent> parseDdl(P partition, String ddlStatements, String databaseName, O offset, Instant sourceTime, boolean snapshot)

    • emitChangeEvent

      private void emitChangeEvent(P partition, O offset, List<SchemaChangeEvent> schemaChangeEvents, String sanitizedDbName, DdlParserListener.Event event, TableId tableId, SchemaChangeEvent.SchemaChangeEventType type, boolean snapshot)

    • getTable

      private Table getTable(TableId tableId, SchemaChangeEvent.SchemaChangeEventType type)

    • acceptableDatabase

      private boolean acceptableDatabase(String databaseName)

    • getTableId

      private TableId getTableId(DdlParserListener.Event event)