Package io.debezium.connector.mysql

Class MySqlDatabaseSchema

java.lang.Object

io.debezium.relational.RelationalDatabaseSchema

io.debezium.relational.HistorizedRelationalDatabaseSchema

io.debezium.connector.mysql.MySqlDatabaseSchema

All Implemented Interfaces:

DatabaseSchema<TableId>, HistorizedDatabaseSchema<TableId>

@NotThreadSafe
public class MySqlDatabaseSchema
extends HistorizedRelationalDatabaseSchema

Component that records the schema history for databases hosted by a MySQL database server. The schema information includes the table definitions and the Kafka Connect Schemas for each table, where the Schema excludes any columns that have been specified in the configuration.

The history is changed by applying DDL statements, and every change is persisted as defined in the supplied MySQL connector configuration. This component can be reconstructed (e.g., on connector restart) and the history loaded from persisted storage.

Note that when applying DDL statements, the caller is able to supply a consumer function that will be called with the DDL statements and the database to which they apply, grouped by database names. However, these will only be called based when the databases are included by the database filters defined in the MySQL connector configuration.

Author:

Randall Hauch

Nested Class Summary

Nested classes/interfaces inherited from interface io.debezium.schema.HistorizedDatabaseSchema

HistorizedDatabaseSchema.SchemaChangeEventConsumer

Field Summary

Fields

Modifier and Type	Field	Description
private MySqlConnectorConfig	connectorConfig
private DdlChanges	ddlChanges
private DdlParser	ddlParser
private RelationalTableFilters	filters
private Set<String>	ignoredQueryStatements
private static org.slf4j.Logger	LOGGER
private boolean	storageInitialiationExecuted
private Map<Long,TableId>	tableIdsByTableNumber

Fields inherited from class io.debezium.relational.HistorizedRelationalDatabaseSchema

databaseHistory

Fields inherited from interface io.debezium.schema.DatabaseSchema

NO_CAPTURED_DATA_COLLECTIONS_WARNING

Constructor Summary

Constructors

Constructor	Description
MySqlDatabaseSchema(MySqlConnectorConfig connectorConfig, MySqlValueConverters valueConverter, TopicSelector<TableId> topicSelector, SchemaNameAdjuster schemaNameAdjuster, boolean tableIdCaseInsensitive)	Create a schema component given the supplied MySQL connector configuration.

Method Summary

Modifier and Type	Method	Description
private boolean	acceptableDatabase(String databaseName)
protected void	appendCreateTableStatement(StringBuilder sb, Table table)
protected void	appendDropTableStatement(StringBuilder sb, TableId tableId)
void	applySchemaChange(SchemaChangeEvent schemaChange)
boolean	assignTableNumber(long tableNumber, TableId id)	Assign the given table number to the table with the specified table ID.
String[]	capturedTablesAsStringArray()	Get all table names for all databases that are monitored whose events are captured by Debezium
void	clearTableMappings()	Clear all of the table mappings.
private void	emitChangeEvent(MySqlPartition partition, MySqlOffsetContext offset, List<SchemaChangeEvent> schemaChangeEvents, String sanitizedDbName, DdlParserListener.Event event, TableId tableId, SchemaChangeEvent.SchemaChangeEventType type, boolean snapshot)
protected DdlParser	getDdlParser()
TableId	getTableId(long tableNumber)	Return the table id associated with MySQL-specific table number.
private TableId	getTableId(DdlParserListener.Event event)
boolean	historyExists()	Return true if the database history entity exists
void	initializeStorage()
boolean	isGlobalSetVariableStatement(String ddl, String databaseName)
boolean	isStorageInitializationExecuted()
private List<SchemaChangeEvent>	parseDdl(MySqlPartition partition, String ddlStatements, String databaseName, MySqlOffsetContext offset, Instant sourceTime, boolean snapshot)
List<SchemaChangeEvent>	parseSnapshotDdl(MySqlPartition partition, String ddlStatements, String databaseName, MySqlOffsetContext offset, Instant sourceTime)
List<SchemaChangeEvent>	parseStreamingDdl(MySqlPartition partition, String ddlStatements, String databaseName, MySqlOffsetContext offset, Instant sourceTime)
protected void	refreshSchemas()	Discard any currently-cached schemas and rebuild them using the filters.
void	setSystemVariables(Map<String,String> variables)	Set the system variables on the DDL parser.
boolean	skipSchemaChangeEvent(SchemaChangeEvent event)
boolean	storeOnlyCapturedTables()
SystemVariables	systemVariables()	Get the system variables as known by the DDL parser.

Methods inherited from class io.debezium.relational.HistorizedRelationalDatabaseSchema

close, isHistorized, record, recover, tableInformationComplete

Methods inherited from class io.debezium.relational.RelationalDatabaseSchema

assureNonEmptySchema, buildAndRegisterSchema, clearSchemas, getTableFilter, refresh, refreshSchema, removeSchema, schemaFor, tableFor, tableIds, tables

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface io.debezium.schema.DatabaseSchema

assureNonEmptySchema, schemaFor

Field Detail

LOGGER

private static final org.slf4j.Logger LOGGER

ignoredQueryStatements

private final Set<String> ignoredQueryStatements

ddlParser

private final DdlParser ddlParser

filters

private final RelationalTableFilters filters

ddlChanges

private final DdlChanges ddlChanges

tableIdsByTableNumber

private final Map<Long,TableId> tableIdsByTableNumber

storageInitialiationExecuted

private boolean storageInitialiationExecuted

connectorConfig

private final MySqlConnectorConfig connectorConfig

Constructor Detail

MySqlDatabaseSchema

public MySqlDatabaseSchema(MySqlConnectorConfig connectorConfig,
                           MySqlValueConverters valueConverter,
                           TopicSelector<TableId> topicSelector,
                           SchemaNameAdjuster schemaNameAdjuster,
                           boolean tableIdCaseInsensitive)

Create a schema component given the supplied MySQL connector configuration. The DDL statements passed to the schema are parsed and a logical model of the database schema is created.

Method Detail

capturedTablesAsStringArray

public String[] capturedTablesAsStringArray()

Get all table names for all databases that are monitored whose events are captured by Debezium

Returns:

the array with the table names

setSystemVariables

public void setSystemVariables(Map<String,String> variables)

Set the system variables on the DDL parser.

Parameters:

variables - the system variables; may not be null but may be empty

systemVariables

public SystemVariables systemVariables()

Get the system variables as known by the DDL parser.

Returns:

the system variables; never null

appendDropTableStatement

protected void appendDropTableStatement(StringBuilder sb,
                                        TableId tableId)

appendCreateTableStatement

protected void appendCreateTableStatement(StringBuilder sb,
                                          Table table)

refreshSchemas

protected void refreshSchemas()

Discard any currently-cached schemas and rebuild them using the filters.

isGlobalSetVariableStatement

public boolean isGlobalSetVariableStatement(String ddl,
                                            String databaseName)

applySchemaChange

public void applySchemaChange(SchemaChangeEvent schemaChange)

parseSnapshotDdl

public List<SchemaChangeEvent> parseSnapshotDdl(MySqlPartition partition,
                                                String ddlStatements,
                                                String databaseName,
                                                MySqlOffsetContext offset,
                                                Instant sourceTime)

parseStreamingDdl

public List<SchemaChangeEvent> parseStreamingDdl(MySqlPartition partition,
                                                 String ddlStatements,
                                                 String databaseName,
                                                 MySqlOffsetContext offset,
                                                 Instant sourceTime)

parseDdl

private List<SchemaChangeEvent> parseDdl(MySqlPartition partition,
                                         String ddlStatements,
                                         String databaseName,
                                         MySqlOffsetContext offset,
                                         Instant sourceTime,
                                         boolean snapshot)

emitChangeEvent

private void emitChangeEvent(MySqlPartition partition,
                             MySqlOffsetContext offset,
                             List<SchemaChangeEvent> schemaChangeEvents,
                             String sanitizedDbName,
                             DdlParserListener.Event event,
                             TableId tableId,
                             SchemaChangeEvent.SchemaChangeEventType type,
                             boolean snapshot)

acceptableDatabase

private boolean acceptableDatabase(String databaseName)

getTableId

private TableId getTableId(DdlParserListener.Event event)

getDdlParser

protected DdlParser getDdlParser()

Specified by:

getDdlParser in class HistorizedRelationalDatabaseSchema

historyExists

public boolean historyExists()

Return true if the database history entity exists

storeOnlyCapturedTables

public boolean storeOnlyCapturedTables()

Specified by:

storeOnlyCapturedTables in interface HistorizedDatabaseSchema<TableId>

Overrides:

storeOnlyCapturedTables in class HistorizedRelationalDatabaseSchema

assignTableNumber

public boolean assignTableNumber(long tableNumber,
                                 TableId id)

Assign the given table number to the table with the specified table ID.

Parameters:

tableNumber - the table number found in binlog events

id - the identifier for the corresponding table

Returns:

true if the assignment was successful, or false if the table is currently excluded in the connector's configuration

getTableId

public TableId getTableId(long tableNumber)

Return the table id associated with MySQL-specific table number.

Parameters:

tableNumber -

Returns:

the table id or null if not known

clearTableMappings

public void clearTableMappings()

Clear all of the table mappings. This should be done when the logs are rotated, since in that a different table numbering scheme will be used by all subsequent TABLE_MAP binlog events.

initializeStorage

public void initializeStorage()

Specified by:

initializeStorage in interface HistorizedDatabaseSchema<TableId>

Overrides:

initializeStorage in class HistorizedRelationalDatabaseSchema

isStorageInitializationExecuted

public boolean isStorageInitializationExecuted()

skipSchemaChangeEvent

public boolean skipSchemaChangeEvent(SchemaChangeEvent event)