io.debezium.relational

Class TableSchemaBuilder

java.lang.Object

io.debezium.relational.TableSchemaBuilder

@ThreadSafe
 @Immutable
public class TableSchemaBuilder
extends Object

Builder that constructs TableSchema instances for Table definitions.

This builder is responsible for mapping table columns to fields in Kafka Connect Schemas, and this is necessarily dependent upon the database's supported types. Although mappings are defined for standard types, this class may need to be subclassed for each DBMS to add support for DBMS-specific types by overriding any of the "add*Field" methods.

See the Java SE Mapping SQL and Java Types for details about how JDBC types map to Java value types.

Author:

Randall Hauch

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static interface	TableSchemaBuilder.ValueConverter A function that converts from a column data value into a Kafka Connect object that is compliant with the Kafka Connect Schema's corresponding Field.

Field Summary

Fields

Modifier and Type	Field and Description
private static LocalDate	EPOCH_DAY
private static org.slf4j.Logger	LOGGER

Constructor Summary

Constructors

Constructor and Description
TableSchemaBuilder() Create a new instance of the builder.

Method Summary

Modifier and Type	Method and Description
protected void	addField(org.apache.kafka.connect.data.SchemaBuilder builder, Column column) Add to the supplied SchemaBuilder a field for the column with the given information.
protected void	addField(org.apache.kafka.connect.data.SchemaBuilder builder, String columnName, int jdbcType, String typeName, int columnLength, int columnScale, boolean optional) Add to the supplied SchemaBuilder a field for the column with the given information.
protected void	addOtherField(org.apache.kafka.connect.data.SchemaBuilder builder, String columnName, int jdbcType, String typeName, int columnLength, int columnScale, boolean optional) Add to the supplied SchemaBuilder a field for the column with the given information.
protected Object	convertDate(org.apache.kafka.connect.data.Field fieldDefn, Object data) Converts a value object for an expected JDBC type of Types.DATE.
protected TableSchemaBuilder.ValueConverter[]	convertersForColumns(org.apache.kafka.connect.data.Schema schema, List<Column> columns)
protected Object	convertTime(org.apache.kafka.connect.data.Field fieldDefn, Object data) Converts a value object for an expected JDBC type of Types.TIME.
protected Object	convertTimestamp(org.apache.kafka.connect.data.Field fieldDefn, Object data) Converts a value object for an expected JDBC type of Types.TIMESTAMP.
protected Object	convertTimestampWithZone(org.apache.kafka.connect.data.Field fieldDefn, Object data) Converts a value object for an expected JDBC type of Types.TIMESTAMP_WITH_TIMEZONE.
protected Object	convertTimeWithZone(org.apache.kafka.connect.data.Field fieldDefn, Object data) Converts a value object for an expected JDBC type of Types.TIME_WITH_TIMEZONE.
TableSchema	create(ResultSet resultSet, String name) Create a TableSchema from the given JDBC ResultSet.
TableSchema	create(Table table) Create a TableSchema from the given table definition.
protected Function<Object[],Object>	createKeyGenerator(org.apache.kafka.connect.data.Schema schema, String columnSetName, List<Column> columns) Creates the function that produces a Kafka Connect key object for a row of data.
protected TableSchemaBuilder.ValueConverter	createValueConverterFor(Column column, org.apache.kafka.connect.data.Field fieldDefn) Create a TableSchemaBuilder.ValueConverter that can be used to convert row values for the given column into the Kafka Connect value object described by the field definition.
protected Function<Object[],org.apache.kafka.connect.data.Struct>	createValueGenerator(org.apache.kafka.connect.data.Schema schema, String columnSetName, List<Column> columns) Creates the function that produces a Kafka Connect value object for a row of data.
protected org.apache.kafka.connect.data.Field[]	fieldsForColumns(org.apache.kafka.connect.data.Schema schema, List<Column> columns)
protected Object	handleUnknownData(Column column, org.apache.kafka.connect.data.Field fieldDefn, Object data) Convert a value object from the specified column to the value type described by the specified Kafka Connect field.
protected int[]	indexesForColumns(List<Column> columns)
protected Date	unexpectedDate(Object value, org.apache.kafka.connect.data.Field fieldDefn) Handle the unexpected value from a row with a column type of Types.DATE.
protected Date	unexpectedTime(Object value, org.apache.kafka.connect.data.Field fieldDefn) Handle the unexpected value from a row with a column type of Types.TIME.
protected Date	unexpectedTimestamp(Object value, org.apache.kafka.connect.data.Field fieldDefn) Handle the unexpected value from a row with a column type of Types.TIMESTAMP.
protected OffsetDateTime	unexpectedTimestampWithZone(Object value, org.apache.kafka.connect.data.Field fieldDefn) Handle the unexpected value from a row with a column type of Types.TIMESTAMP_WITH_TIMEZONE.
protected OffsetTime	unexpectedTimeWithZone(Object value, org.apache.kafka.connect.data.Field fieldDefn) Handle the unexpected value from a row with a column type of Types.TIME_WITH_TIMEZONE.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOGGER

private static final org.slf4j.Logger LOGGER

EPOCH_DAY

private static final LocalDate EPOCH_DAY

Constructor Detail

TableSchemaBuilder

public TableSchemaBuilder()

Create a new instance of the builder.

Method Detail

create

public TableSchema create(ResultSet resultSet,
                          String name)
                   throws SQLException

Create a TableSchema from the given JDBC ResultSet. The resulting TableSchema will have no primary key, and its TableSchema.valueSchema() will contain fields for each column in the result set.

Parameters:

resultSet - the result set for a query; may not be null

name - the name of the value schema; may not be null

Returns:

the table schema that can be used for sending rows of data for this table to Kafka Connect; never null

Throws:

SQLException - if an error occurs while using the result set's metadata

create

public TableSchema create(Table table)

Create a TableSchema from the given table definition. The resulting TableSchema will have a key schema that contains all of the columns that make up the table's primary key, and a value schema that contains only those columns that are not in the table's primary key.

This is equivalent to calling create(table,false).

Parameters:

table - the table definition; may not be null

Returns:

the table schema that can be used for sending rows of data for this table to Kafka Connect; never null

createKeyGenerator

protected Function<Object[],Object> createKeyGenerator(org.apache.kafka.connect.data.Schema schema,
                                                       String columnSetName,
                                                       List<Column> columns)

Creates the function that produces a Kafka Connect key object for a row of data.

Parameters:

schema - the Kafka Connect schema for the key; may be null if there is no known schema, in which case the generator will be null

columnSetName - the name for the set of columns, used in error messages; may not be null

columns - the column definitions for the table that defines the row; may not be null

Returns:

the key-generating function, or null if there is no key schema

createValueGenerator

protected Function<Object[],org.apache.kafka.connect.data.Struct> createValueGenerator(org.apache.kafka.connect.data.Schema schema,
                                                                                       String columnSetName,
                                                                                       List<Column> columns)

Creates the function that produces a Kafka Connect value object for a row of data.

Parameters:

schema - the Kafka Connect schema for the value; may be null if there is no known schema, in which case the generator will be null

columnSetName - the name for the set of columns, used in error messages; may not be null

columns - the column definitions for the table that defines the row; may not be null

Returns:

the value-generating function, or null if there is no value schema

indexesForColumns

protected int[] indexesForColumns(List<Column> columns)

fieldsForColumns

protected org.apache.kafka.connect.data.Field[] fieldsForColumns(org.apache.kafka.connect.data.Schema schema,
                                                                 List<Column> columns)

convertersForColumns

protected TableSchemaBuilder.ValueConverter[] convertersForColumns(org.apache.kafka.connect.data.Schema schema,
                                                                   List<Column> columns)

addField

protected void addField(org.apache.kafka.connect.data.SchemaBuilder builder,
                        Column column)

Add to the supplied SchemaBuilder a field for the column with the given information.

Parameters:

builder - the schema builder; never null

column - the column definition

addField

protected void addField(org.apache.kafka.connect.data.SchemaBuilder builder,
                        String columnName,
                        int jdbcType,
                        String typeName,
                        int columnLength,
                        int columnScale,
                        boolean optional)

Add to the supplied SchemaBuilder a field for the column with the given information.

Subclasses that wish to override or extend the mappings of JDBC/DBMS types to Kafka Connect value types can override this method and delegate to this method before and/or after the custom logic. Similar behavior should be addressed in a specialized createValueConverterFor(Column, Field) as well.

Parameters:

builder - the schema builder; never null

columnName - the name of the column

jdbcType - the column's JDBC type

typeName - the column's DBMS-specific type name

columnLength - the length of the column

columnScale - the scale of the column values, or 0 if not a decimal value

optional - true if the column is optional, or false if the column is known to always have a value

addOtherField

protected void addOtherField(org.apache.kafka.connect.data.SchemaBuilder builder,
                             String columnName,
                             int jdbcType,
                             String typeName,
                             int columnLength,
                             int columnScale,
                             boolean optional)

Add to the supplied SchemaBuilder a field for the column with the given information.

Subclasses that wish to override or extend the mappings of JDBC/DBMS types to Kafka Connect value types can override this method and delegate to this method before and/or after the custom logic. Similar behavior should be addressed in a specialized addField(SchemaBuilder, String, int, String, int, int, boolean) as well.

Parameters:

builder - the schema builder; never null

columnName - the name of the column

jdbcType - the column's JDBC type

typeName - the column's DBMS-specific type name

columnLength - the length of the column

columnScale - the scale of the column values, or 0 if not a decimal value

optional - true if the column is optional, or false if the column is known to always have a value

createValueConverterFor

protected TableSchemaBuilder.ValueConverter createValueConverterFor(Column column,
                                                                    org.apache.kafka.connect.data.Field fieldDefn)

Create a TableSchemaBuilder.ValueConverter that can be used to convert row values for the given column into the Kafka Connect value object described by the field definition.

Parameters:

column - the column describing the input values; never null

fieldDefn - the definition for the field in a Kafka Connect Schema describing the output of the function; never null

Returns:

the value conversion function; may not be null

handleUnknownData

protected Object handleUnknownData(Column column,
                                   org.apache.kafka.connect.data.Field fieldDefn,
                                   Object data)

Convert a value object from the specified column to the value type described by the specified Kafka Connect field.

Parameters:

column - the column definition describing the data value; never null

fieldDefn - the field definition; never null

data - the data object to be converted into a Kafka Connect date type; never null

Returns:

the converted value, or null if the conversion could not be made

convertTimestampWithZone

protected Object convertTimestampWithZone(org.apache.kafka.connect.data.Field fieldDefn,
                                          Object data)

Converts a value object for an expected JDBC type of Types.TIMESTAMP_WITH_TIMEZONE. The standard ANSI to Java 8 type mappings specify that the preferred mapping (when using JDBC's getObject(...) methods) in Java 8 is to return OffsetDateTime for these values.

This method handles several types of objects, including OffsetDateTime, Timestamp, Date, LocalTime, and LocalDateTime.

Parameters:

fieldDefn - the field definition; never null

data - the data object to be converted into a Kafka Connect date type; never null

Returns:

the converted value, or null if the conversion could not be made

unexpectedTimestampWithZone

protected OffsetDateTime unexpectedTimestampWithZone(Object value,
                                                     org.apache.kafka.connect.data.Field fieldDefn)

Handle the unexpected value from a row with a column type of Types.TIMESTAMP_WITH_TIMEZONE.

Parameters:

value - the timestamp value for which no conversion was found; never null

fieldDefn - the field definition in the Kafka Connect schema; never null

Returns:

the converted value, or null

convertTimeWithZone

protected Object convertTimeWithZone(org.apache.kafka.connect.data.Field fieldDefn,
                                     Object data)

Converts a value object for an expected JDBC type of Types.TIME_WITH_TIMEZONE. The standard ANSI to Java 8 type mappings specify that the preferred mapping (when using JDBC's getObject(...) methods) in Java 8 is to return OffsetTime for these values.

This method handles several types of objects, including OffsetTime, Time, Date, LocalTime, and LocalDateTime. If any of the types have date components, those date components are ignored.

Parameters:

fieldDefn - the field definition; never null

data - the data object to be converted into a Kafka Connect date type; never null

Returns:

the converted value, or null if the conversion could not be made

unexpectedTimeWithZone

protected OffsetTime unexpectedTimeWithZone(Object value,
                                            org.apache.kafka.connect.data.Field fieldDefn)

Handle the unexpected value from a row with a column type of Types.TIME_WITH_TIMEZONE.

Parameters:

value - the timestamp value for which no conversion was found; never null

fieldDefn - the field definition in the Kafka Connect schema; never null

Returns:

the converted value, or null

convertTimestamp

protected Object convertTimestamp(org.apache.kafka.connect.data.Field fieldDefn,
                                  Object data)

Converts a value object for an expected JDBC type of Types.TIMESTAMP.

Per the JDBC specification, databases should return Timestamp instances, which have date and time info but no time zone info. This method handles Date objects plus any other standard date-related objects such as Date, LocalTime, and LocalDateTime.

Parameters:

fieldDefn - the field definition; never null

data - the data object to be converted into a Kafka Connect date type; never null

Returns:

the converted value, or null if the conversion could not be made

unexpectedTimestamp

protected Date unexpectedTimestamp(Object value,
                                   org.apache.kafka.connect.data.Field fieldDefn)

Handle the unexpected value from a row with a column type of Types.TIMESTAMP.

Parameters:

value - the timestamp value for which no conversion was found; never null

fieldDefn - the field definition in the Kafka Connect schema; never null

Returns:

the converted value, or null

convertTime

protected Object convertTime(org.apache.kafka.connect.data.Field fieldDefn,
                             Object data)

Converts a value object for an expected JDBC type of Types.TIME.

Per the JDBC specification, databases should return Time instances that have no notion of date or time zones. This method handles Date objects plus any other standard date-related objects such as Date, LocalTime, and LocalDateTime. If any of the types might have date components, those date components are ignored.

Parameters:

fieldDefn - the field definition; never null

data - the data object to be converted into a Kafka Connect date type; never null

Returns:

the converted value, or null if the conversion could not be made

unexpectedTime

protected Date unexpectedTime(Object value,
                              org.apache.kafka.connect.data.Field fieldDefn)

Handle the unexpected value from a row with a column type of Types.TIME.

Parameters:

value - the timestamp value for which no conversion was found; never null

fieldDefn - the field definition in the Kafka Connect schema; never null

Returns:

the converted value, or null

convertDate

protected Object convertDate(org.apache.kafka.connect.data.Field fieldDefn,
                             Object data)

Converts a value object for an expected JDBC type of Types.DATE.

Per the JDBC specification, databases should return Date instances that have no notion of time or time zones. This method handles Date objects plus any other standard date-related objects such as Date, LocalDate, and LocalDateTime. If any of the types might have time components, those time components are ignored.

Parameters:

fieldDefn - the field definition; never null

data - the data object to be converted into a Kafka Connect date type; never null

Returns:

the converted value, or null if the conversion could not be made

unexpectedDate

protected Date unexpectedDate(Object value,
                              org.apache.kafka.connect.data.Field fieldDefn)

Handle the unexpected value from a row with a column type of Types.DATE.

Parameters:

value - the timestamp value for which no conversion was found; never null

fieldDefn - the field definition in the Kafka Connect schema; never null

Returns:

the converted value, or null