Package io.debezium.connector.mysql.legacy

Class MySqlSchema

java.lang.Object
io.debezium.relational.RelationalDatabaseSchema
io.debezium.connector.mysql.legacy.MySqlSchema
All Implemented Interfaces:
DatabaseSchema<TableId>, AutoCloseable
@NotThreadSafe public class MySqlSchema extends RelationalDatabaseSchema
Component that records the schema history for databases hosted by a MySQL database server. The schema information includes the table definitions and the Kafka Connect Schemas for each table, where the Schema excludes any columns that have been specified in the configuration.

The history is changed by applying DDL statements, and every change is persisted as defined in the supplied MySQL connector configuration. This component can be reconstructed (e.g., on connector restart) and the history loaded from persisted storage.

Note that when applying DDL statements, the caller is able to supply a consumer function that will be called with the DDL statements and the database to which they apply, grouped by database names. However, these will only be called based when the databases are included by the database filters defined in the MySQL connector configuration.

Author:
Randall Hauch

  • Field Details

    • logger

      private static final org.slf4j.Logger logger

    • ignoredQueryStatements

      private final Set<String> ignoredQueryStatements

    • ddlParser

      private final DdlParser ddlParser

    • filters

      private final Filters filters

    • dbHistory

      private final DatabaseHistory dbHistory

    • ddlChanges

      private final DdlChanges ddlChanges

    • historyComparator

      private final HistoryRecordComparator historyComparator

    • skipUnparseableDDL

      private final boolean skipUnparseableDDL

    • storeOnlyCapturedTablesDdl

      private final boolean storeOnlyCapturedTablesDdl

    • recoveredTables

      private boolean recoveredTables

  • Constructor Details

    • MySqlSchema

      public MySqlSchema(MySqlConnectorConfig configuration, Predicate<String> gtidFilter, boolean tableIdCaseInsensitive, TopicSelector<TableId> topicSelector, Filters tableFilters)
      Create a schema component given the supplied MySQL connector configuration.
      Parameters:
      configuration - the connector configuration, which is presumed to be valid
      gtidFilter - the predicate function that should be applied to GTID sets in database history, and which returns true if a GTID source is to be included, or false if a GTID source is to be excluded; may be null if not needed
      tableIdCaseInsensitive - true if table lookup ignores letter case

  • Method Details

    • getValueConverters

      private static MySqlValueConverters getValueConverters(MySqlConnectorConfig configuration)

    • historyComparator

      public HistoryRecordComparator historyComparator()

    • start

      public void start()
      Start by acquiring resources needed to persist the database history

    • shutdown

      public void shutdown()
      Stop recording history and release any resources acquired since start().

    • filters

      public Filters filters()
      Get the database and table filters defined by the configuration.
      Returns:
      the filters; never null

    • capturedTablesAsStringArray

      public String[] capturedTablesAsStringArray()
      Get all table names for all databases that are monitored whose events are captured by Debezium
      Returns:
      the array with the table names

    • isTableCaptured

      public boolean isTableCaptured(TableId id)
      Decide whether events should be captured for a given table
      Parameters:
      id - the fully-qualified table identifier; may be null
      Returns:
      true if events from the table are captured

    • historyLocation

      public String historyLocation()
      Get the information about where the DDL statement history is recorded.
      Returns:
      the history description; never null

    • setSystemVariables

      public void setSystemVariables(Map<String,String> variables)
      Set the system variables on the DDL parser.
      Parameters:
      variables - the system variables; may not be null but may be empty

    • systemVariables

      public SystemVariables systemVariables()
      Get the system variables as known by the DDL parser.
      Returns:
      the system variables; never null

    • appendDropTableStatement

      protected void appendDropTableStatement(StringBuilder sb, TableId tableId)

    • appendCreateTableStatement

      protected void appendCreateTableStatement(StringBuilder sb, Table table)

    • loadHistory

      public void loadHistory(SourceInfo startingPoint)
      Load the database schema information using the previously-recorded history, and stop reading the history when the the history reaches the supplied starting point.
      Parameters:
      startingPoint - the source information with the current SourceInfo.partition() and offset at which the database schemas are to reflect; may not be null

    • historyExists

      public boolean historyExists()
      Return true if the database history entity exists

    • intializeHistoryStorage

      public void intializeHistoryStorage()
      Initialize permanent storage for database history

    • refreshSchemas

      public void refreshSchemas()
      Discard any currently-cached schemas and rebuild them using the filters.

    • applyDdl

      public boolean applyDdl(SourceInfo source, String databaseName, String ddlStatements, DdlChanges.DatabaseStatementStringConsumer statementConsumer)
      Apply the supplied DDL statements to this database schema and record the history. If a statementConsumer is supplied, then call it for each sub-sequence of the DDL statements that all apply to the same database.

      Typically DDL statements are applied using a connection to a single database, and unless the statements use fully-qualified names, the DDL statements apply to this database.

      Parameters:
      source - the current SourceInfo.partition() and offset at which these changes are found; may not be null
      databaseName - the name of the default database under which these statements are applied; may not be null
      ddlStatements - the ;-separated DDL statements; may be null or empty
      statementConsumer - the consumer that should be called with each sub-sequence of DDL statements that apply to a single database; may be null if no action is to be performed with the changes
      Returns:
      true if changes were made to the database schema, or false if the DDL statements had no effect on the database schema

    • isGlobalSetVariableStatement

      public boolean isGlobalSetVariableStatement(String ddl, String databaseName)

    • isStoreOnlyCapturedTablesDdl

      public boolean isStoreOnlyCapturedTablesDdl()
      Returns:
      true if only captured tables should be stored in database history, false if all tables should be stored

    • tableInformationComplete

      public boolean tableInformationComplete()
      Specified by:
      tableInformationComplete in interface DatabaseSchema<TableId>
      Overrides:
      tableInformationComplete in class RelationalDatabaseSchema