io.debezium.relational.ddl

Class LegacyDdlParser

java.lang.Object

io.debezium.relational.ddl.AbstractDdlParser

io.debezium.relational.ddl.LegacyDdlParser

All Implemented Interfaces:

DdlParser

Direct Known Subclasses:

DdlParserSql2003

@NotThreadSafe
public class LegacyDdlParser
extends AbstractDdlParser
implements DdlParser

A parser for DDL statements.

Author:

Randall Hauch, Horia Chiorean, Barry LaFond

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static interface	LegacyDdlParser.TokenSet

Field Summary

Fields

Modifier and Type	Field and Description
protected Tables	databaseTables
protected DataTypeParser	dataTypeParser
private Set<String>	keywords
private List<DdlParserListener>	listeners
protected org.slf4j.Logger	logger
private Set<String>	statementStarts
protected TokenStream	tokens

Fields inherited from class io.debezium.relational.ddl.AbstractDdlParser

ddlChanges, skipViews, systemVariables

Constructor Summary

Constructors

Constructor and Description
LegacyDdlParser(String terminator) Create a new parser that uses the supplied DataTypeParser, but that does not include view definitions.
LegacyDdlParser(String terminator, boolean includeViews) Create a new parser that uses the supplied DataTypeParser.

Method Summary

Modifier and Type	Method and Description
void	addListener(DdlParserListener listener)
protected boolean	canConsumeJoin(TokenStream.Marker start)
protected boolean	canConsumeJoinCondition(TokenStream.Marker start)
protected void	consumeBeginStatement(TokenStream.Marker start) Consume the entire BEGIN...END block that appears next in the token stream.
protected String	consumeDoubleQuotedString() Consume the next token that is a double-quoted string.
protected String	consumeQuotedString() Consume the next token that is either a single-quoted string or a double-quoted string.
protected void	consumeRemainingStatement(TokenStream.Marker start) Consume all tokens from the current position until and including either the end-of-statement terminator token or one of the tokens that is registered as the start of a statement.
protected String	consumeSingleQuotedString() Consume the next token that is a single-quoted string.
protected void	consumeStatement() Consume all tokens from the current position that is a starting-statement token until either the end-of-statement terminator token or before the next starting-statement token.
protected SystemVariables	createNewSystemVariablesInstance()
protected void	debugParsed(TokenStream.Marker statementStart)
protected void	debugSkipped(TokenStream.Marker statementStart)
protected int	determineTokenType(int type, String token)
protected String	determineTypeNameForConstant(BigDecimal value)
protected String	determineTypeNameForConstant(double value)
protected String	determineTypeNameForConstant(float value)
protected String	determineTypeNameForConstant(long value)
protected void	initializeDataTypes(DataTypeParser dataTypeParser)
protected void	initializeKeywords(LegacyDdlParser.TokenSet keywords)
protected void	initializeStatementStarts(LegacyDdlParser.TokenSet statementStartTokens)
protected boolean	isNextTokenQuotedIdentifier() Determine if the next token is a single- or double-quoted string.
void	parse(String ddlContent, Tables databaseTables) Examine the supplied string containing DDL statements, and apply those statements to the specified database table definitions.
void	parse(TokenStream ddlContent, Tables databaseTables) Examine the stream starting at its current position for DDL statements, and apply those statements to the specified database table definitions.
private void	parseAliasedTableInFrom(TokenStream.Marker start, Map<String,Table> tablesByAlias) Parse a potentially qualified table name along with an optional alias.
protected void	parseAlter(TokenStream.Marker marker) Parse the a DDL "ALTER" statement.
protected String	parseBitFieldLiteral(TokenStream.Marker start)
protected String	parseCharacterLiteral(TokenStream.Marker start)
protected String	parseCharacterSetName(TokenStream.Marker start)
protected void	parseColumnName(TokenStream.Marker start, Map<String,String> tableAliasByColumnAliases, Map<String,String> columnNameByAliases) Parse the potentially qualified and aliased column information, and add the information to the supplied maps.
protected Map<String,Column>	parseColumnsInSelectClause(TokenStream.Marker start) Parse the column information in the SELECT clause.
protected void	parseComment(TokenStream.Marker marker) Parse the a DDL line comment.
protected void	parseCreate(TokenStream.Marker marker) Parse the a DDL "CREATE" statement.
protected String	parseDateLiteral(TokenStream.Marker start)
protected void	parseDrop(TokenStream.Marker marker) Parse the a DDL "DROP" statement.
protected Object	parseLiteral(TokenStream.Marker start)
protected void	parseNextStatement(TokenStream.Marker marker) Parse the next DDL statement.
protected Object	parseNumericLiteral(TokenStream.Marker start, boolean signed)
protected TableId	parseQualifiedTableName(TokenStream.Marker start) Parse the next tokens for a possibly qualified table name.
protected List<TableId>	parseQualifiedTableNames(TokenStream.Marker start) Parse the next tokens for one or more comma-separated qualified table names.
protected String	parseSchemaQualifiedName(TokenStream.Marker start) Parse the next tokens for a possibly qualified name.
protected Map<String,Table>	parseSelectFromClause(TokenStream.Marker start) Returns the tables keyed by their aliases that appear in a SELECT clause's "FROM" list.
protected String	parseTimeLiteral(TokenStream.Marker start)
protected String	parseTimestampLiteral(TokenStream.Marker start)
protected void	parseUnknownStatement(TokenStream.Marker marker) Parse a DDL statement that is not known by the parseNextStatement(io.debezium.text.TokenStream.Marker).
protected void	parsingFailed(Position position, Collection<ParsingException> errors, String msg) Generate a ParsingException or MultipleParsingExceptions with the supplied error or errors and the supplied message, which is appended by this method with additional information about the position's line and column.
protected void	parsingFailed(Position position, String msg) Generate a ParsingException with the supplied message, which is appended by this method with additional information about the position's line and column.
protected void	parsingFailed(Position position, String msg, Throwable t) Generate a ParsingException with the supplied message, which is appended by this method with additional information about the position's line and column.
boolean	removeListener(DdlParserListener listener)
void	removeListeners()
protected void	signalAlterDatabase(String databaseName, String previousDatabaseName, TokenStream.Marker statementStart) Signal an alter database event to all listeners.
protected void	signalAlterTable(TableId id, TableId previousId, TokenStream.Marker statementStart) Signal an alter table event to all listeners.
protected void	signalAlterView(TableId id, TableId previousId, TokenStream.Marker statementStart) Signal an alter view event to all listeners.
protected void	signalChangeEvent(DdlParserListener.Event event) Signal an event to all listeners.
protected void	signalCreateDatabase(String databaseName, TokenStream.Marker statementStart) Signal a create database event to all listeners.
protected void	signalCreateIndex(String indexName, TableId id, TokenStream.Marker statementStart) Signal a create index event to all listeners.
protected void	signalCreateTable(TableId id, TokenStream.Marker statementStart) Signal a create table event to all listeners.
protected void	signalCreateView(TableId id, TokenStream.Marker statementStart) Signal a create view event to all listeners.
protected void	signalDropDatabase(String databaseName, TokenStream.Marker statementStart) Signal a drop database event to all listeners.
protected void	signalDropIndex(String indexName, TableId id, TokenStream.Marker statementStart) Signal a drop index event to all listeners.
protected void	signalDropTable(TableId id, TokenStream.Marker statementStart) Signal a drop table event to all listeners.
protected void	signalDropView(TableId id, TokenStream.Marker statementStart) Signal a drop view event to all listeners.
protected String	statement(TokenStream.Marker statementStart)

Methods inherited from class io.debezium.relational.ddl.AbstractDdlParser

accumulateParsingFailure, accumulateParsingFailure, commentParsed, createColumnFromConstant, currentSchema, debugParsed, debugSkipped, getDdlChanges, removeLineFeeds, resolveTableId, setCurrentDatabase, setCurrentSchema, setTypeInfoForConstant, signalAlterDatabase, signalAlterTable, signalAlterView, signalCreateDatabase, signalCreateIndex, signalCreateTable, signalCreateView, signalDropDatabase, signalDropIndex, signalDropTable, signalDropView, signalSetVariable, signalTruncateTable, signalUseDatabase, skipComments, systemVariables, terminator, withoutQuotes

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface io.debezium.relational.ddl.DdlParser

getDdlChanges, setCurrentDatabase, setCurrentSchema, systemVariables, terminator

Field Detail

keywords

private final Set<String> keywords

statementStarts

private final Set<String> statementStarts

logger

protected final org.slf4j.Logger logger

dataTypeParser

protected final DataTypeParser dataTypeParser

databaseTables

protected Tables databaseTables

tokens

protected TokenStream tokens

listeners

private final List<DdlParserListener> listeners

Constructor Detail

LegacyDdlParser

public LegacyDdlParser(String terminator)

Create a new parser that uses the supplied DataTypeParser, but that does not include view definitions.

Parameters:

terminator - the terminator character sequence; may be null if the default terminator (;) should be used

LegacyDdlParser

public LegacyDdlParser(String terminator,
                       boolean includeViews)

Create a new parser that uses the supplied DataTypeParser.

Parameters:

terminator - the terminator character sequence; may be null if the default terminator (;) should be used

includeViews - true if view definitions should be included, or false if they should be skipped

Method Detail

createNewSystemVariablesInstance

protected SystemVariables createNewSystemVariablesInstance()

Specified by:

createNewSystemVariablesInstance in class AbstractDdlParser

initializeDataTypes

protected void initializeDataTypes(DataTypeParser dataTypeParser)

initializeKeywords

protected void initializeKeywords(LegacyDdlParser.TokenSet keywords)

initializeStatementStarts

protected void initializeStatementStarts(LegacyDdlParser.TokenSet statementStartTokens)

addListener

public void addListener(DdlParserListener listener)

removeListener

public boolean removeListener(DdlParserListener listener)

removeListeners

public void removeListeners()

isNextTokenQuotedIdentifier

protected boolean isNextTokenQuotedIdentifier()

Determine if the next token is a single- or double-quoted string.

Returns:

true if the next token is a single-quoted string or double-quoted string, or false otherwise

determineTokenType

protected int determineTokenType(int type,
                                 String token)

parseSchemaQualifiedName

protected String parseSchemaQualifiedName(TokenStream.Marker start)

Parse the next tokens for a possibly qualified name. This method builds up a string containing the schema name (or if none is found the the current schema), a '.' delimiter, and the object name. If no schema name is found, just the object name is returned.

Parameters:

start - the start of the statement

Returns:

the qualified schema name.

parseQualifiedTableName

protected TableId parseQualifiedTableName(TokenStream.Marker start)

Parse the next tokens for a possibly qualified table name. This method uses the schema name that appears in the token stream, or if none is found the AbstractDdlParser.currentSchema(), and then calls AbstractDdlParser.resolveTableId(String, String) with the values.

Parameters:

start - the start of the statement

Returns:

the resolved TableId

parseQualifiedTableNames

protected List<TableId> parseQualifiedTableNames(TokenStream.Marker start)

Parse the next tokens for one or more comma-separated qualified table names. This method uses the schema name that appears in the token stream, or if none is found the AbstractDdlParser.currentSchema(), and then calls AbstractDdlParser.resolveTableId(String, String) with the values.

Parameters:

start - the start of the statement

Returns:

the resolved TableId

parse

public final void parse(String ddlContent,
                        Tables databaseTables)

Examine the supplied string containing DDL statements, and apply those statements to the specified database table definitions.

Specified by:

parse in interface DdlParser

Parameters:

ddlContent - the stream of tokens containing the DDL statements; may not be null

databaseTables - the database's table definitions, which should be used by this method to create, change, or remove tables as defined in the DDL content; may not be null

Throws:

ParsingException - if there is a problem parsing the supplied content

parse

public final void parse(TokenStream ddlContent,
                        Tables databaseTables)
                 throws ParsingException,
                        IllegalStateException

Examine the stream starting at its current position for DDL statements, and apply those statements to the specified database table definitions.

Parameters:

ddlContent - the stream of tokens containing the DDL statements; may not be null

databaseTables - the database's table definitions, which should be used by this method to create, change, or remove tables as defined in the DDL content; may not be null

Throws:

ParsingException - if there is a problem parsing the supplied content

IllegalStateException - if the supplied token stream is in an invalid state

parseNextStatement

protected void parseNextStatement(TokenStream.Marker marker)

Parse the next DDL statement. This is the primary entry point for subclasses.

Parameters:

marker - the start of the statement; never null

Throws:

ParsingException - if there is an error parsing the statement

parseComment

protected void parseComment(TokenStream.Marker marker)

Parse the a DDL line comment. This is generally called by parseNextStatement(io.debezium.text.TokenStream.Marker) for line comments that appear between other DDL statements, and is not typically called for comments that appear within DDL statements.

Parameters:

marker - the start of the statement; never null

Throws:

ParsingException - if there is an error parsing the statement

parseCreate

protected void parseCreate(TokenStream.Marker marker)

Parse the a DDL "CREATE" statement. This method is intended to be overridden by subclasses.

By default this method simply consumes the complete statement.

Parameters:

marker - the start of the statement; never null

Throws:

ParsingException - if there is an error parsing the statement

parseAlter

protected void parseAlter(TokenStream.Marker marker)

Parse the a DDL "ALTER" statement. This method is intended to be overridden by subclasses.

By default this method simply consumes the complete statement.

Parameters:

marker - the start of the statement; never null

Throws:

ParsingException - if there is an error parsing the statement

parseDrop

protected void parseDrop(TokenStream.Marker marker)

Parse the a DDL "DROP" statement. This method is intended to be overridden by subclasses.

By default this method simply consumes the complete statement.

Parameters:

marker - the start of the statement; never null

Throws:

ParsingException - if there is an error parsing the statement

parseUnknownStatement

protected void parseUnknownStatement(TokenStream.Marker marker)

Parse a DDL statement that is not known by the parseNextStatement(io.debezium.text.TokenStream.Marker). This method can be overridden by subclasses, although it will be more common for subclasses to override parseNextStatement(io.debezium.text.TokenStream.Marker).

By default this method simply consumes the complete statement.

Parameters:

marker - the start of the statement; never null

Throws:

ParsingException - if there is an error parsing the statement

signalCreateDatabase

protected void signalCreateDatabase(String databaseName,
                                    TokenStream.Marker statementStart)

Signal a create database event to all listeners.

Parameters:

databaseName - the database name; may not be null

statementStart - the start of the statement; may not be null

signalChangeEvent

protected void signalChangeEvent(DdlParserListener.Event event)

Signal an event to all listeners.

Overrides:

signalChangeEvent in class AbstractDdlParser

Parameters:

event - the event; may not be null

signalAlterDatabase

protected void signalAlterDatabase(String databaseName,
                                   String previousDatabaseName,
                                   TokenStream.Marker statementStart)

Signal an alter database event to all listeners.

Parameters:

databaseName - the database name; may not be null

previousDatabaseName - the previous name of the database if it was renamed, or null if it was not renamed

statementStart - the start of the statement; may not be null

signalDropDatabase

protected void signalDropDatabase(String databaseName,
                                  TokenStream.Marker statementStart)

Signal a drop database event to all listeners.

Parameters:

databaseName - the database name; may not be null

statementStart - the start of the statement; may not be null

signalCreateTable

protected void signalCreateTable(TableId id,
                                 TokenStream.Marker statementStart)

Signal a create table event to all listeners.

Parameters:

id - the table identifier; may not be null

statementStart - the start of the statement; may not be null

signalAlterTable

protected void signalAlterTable(TableId id,
                                TableId previousId,
                                TokenStream.Marker statementStart)

Signal an alter table event to all listeners.

Parameters:

id - the table identifier; may not be null

previousId - the previous name of the view if it was renamed, or null if it was not renamed

statementStart - the start of the statement; may not be null

signalDropTable

protected void signalDropTable(TableId id,
                               TokenStream.Marker statementStart)

Signal a drop table event to all listeners.

Parameters:

id - the table identifier; may not be null

statementStart - the start of the statement; may not be null

signalCreateView

protected void signalCreateView(TableId id,
                                TokenStream.Marker statementStart)

Signal a create view event to all listeners.

Parameters:

id - the table identifier; may not be null

statementStart - the start of the statement; may not be null

signalAlterView

protected void signalAlterView(TableId id,
                               TableId previousId,
                               TokenStream.Marker statementStart)

Signal an alter view event to all listeners.

Parameters:

id - the table identifier; may not be null

previousId - the previous name of the view if it was renamed, or null if it was not renamed

statementStart - the start of the statement; may not be null

signalDropView

protected void signalDropView(TableId id,
                              TokenStream.Marker statementStart)

Signal a drop view event to all listeners.

Parameters:

id - the table identifier; may not be null

statementStart - the start of the statement; may not be null

signalCreateIndex

protected void signalCreateIndex(String indexName,
                                 TableId id,
                                 TokenStream.Marker statementStart)

Signal a create index event to all listeners.

Parameters:

indexName - the name of the index; may not be null

id - the table identifier; may be null if the index does not apply to a single table

statementStart - the start of the statement; may not be null

signalDropIndex

protected void signalDropIndex(String indexName,
                               TableId id,
                               TokenStream.Marker statementStart)

Signal a drop index event to all listeners.

Parameters:

indexName - the name of the index; may not be null

id - the table identifier; may not be null

statementStart - the start of the statement; may not be null

debugParsed

protected void debugParsed(TokenStream.Marker statementStart)

debugSkipped

protected void debugSkipped(TokenStream.Marker statementStart)

statement

protected String statement(TokenStream.Marker statementStart)

consumeStatement

protected void consumeStatement()
                         throws ParsingException

Consume all tokens from the current position that is a starting-statement token until either the end-of-statement terminator token or before the next starting-statement token.

Throws:

ParsingException - if the next token is not a starting-statement token

consumeRemainingStatement

protected void consumeRemainingStatement(TokenStream.Marker start)

Consume all tokens from the current position until and including either the end-of-statement terminator token or one of the tokens that is registered as the start of a statement.

Parameters:

start - the marker at which the statement was begun

consumeBeginStatement

protected void consumeBeginStatement(TokenStream.Marker start)

Consume the entire BEGIN...END block that appears next in the token stream. This method may need to be specialized for a specific DDL grammar.

Parameters:

start - the marker at which the statement was begun

consumeSingleQuotedString

protected String consumeSingleQuotedString()

Consume the next token that is a single-quoted string.

Returns:

the quoted string; never null

Throws:

ParsingException - if there is no single-quoted string at the current position

consumeDoubleQuotedString

protected String consumeDoubleQuotedString()

Consume the next token that is a double-quoted string.

Returns:

the quoted string; never null

Throws:

ParsingException - if there is no double-quoted string at the current position

consumeQuotedString

protected String consumeQuotedString()

Consume the next token that is either a single-quoted string or a double-quoted string.

Returns:

the quoted string; never null

Throws:

ParsingException - if there is no single- or double-quoted string at the current position

parsingFailed

protected void parsingFailed(Position position,
                             String msg)

Generate a ParsingException with the supplied message, which is appended by this method with additional information about the position's line and column.

Parameters:

position - the position at which the error occurred; may not be null

msg - the leading portion of the message; may not be null

parsingFailed

protected void parsingFailed(Position position,
                             String msg,
                             Throwable t)

Generate a ParsingException with the supplied message, which is appended by this method with additional information about the position's line and column.

Parameters:

position - the position at which the error occurred

msg - the leading portion of the message; may not be null

t - the exception that occurred; may be null

parsingFailed

protected void parsingFailed(Position position,
                             Collection<ParsingException> errors,
                             String msg)

Generate a ParsingException or MultipleParsingExceptions with the supplied error or errors and the supplied message, which is appended by this method with additional information about the position's line and column.

Parameters:

position - the position at which the error occurred; may not be null

errors - the multiple parsing exception errors; may not be null

msg - the leading portion of the message; may not be null

parseLiteral

protected Object parseLiteral(TokenStream.Marker start)

parseNumericLiteral

protected Object parseNumericLiteral(TokenStream.Marker start,
                                     boolean signed)

parseCharacterLiteral

protected String parseCharacterLiteral(TokenStream.Marker start)

parseCharacterSetName

protected String parseCharacterSetName(TokenStream.Marker start)

parseBitFieldLiteral

protected String parseBitFieldLiteral(TokenStream.Marker start)

parseDateLiteral

protected String parseDateLiteral(TokenStream.Marker start)

parseTimeLiteral

protected String parseTimeLiteral(TokenStream.Marker start)

parseTimestampLiteral

protected String parseTimestampLiteral(TokenStream.Marker start)

parseColumnsInSelectClause

protected Map<String,Column> parseColumnsInSelectClause(TokenStream.Marker start)

Parse the column information in the SELECT clause. This statement stops before consuming the FROM clause.

Parameters:

start - the start of the statement

Returns:

the map of resolved Columns keyed by the column alias (or name) used in the SELECT statement; never null but possibly empty if we couldn't parse the SELECT clause correctly

determineTypeNameForConstant

protected String determineTypeNameForConstant(long value)

determineTypeNameForConstant

protected String determineTypeNameForConstant(float value)

determineTypeNameForConstant

protected String determineTypeNameForConstant(double value)

determineTypeNameForConstant

protected String determineTypeNameForConstant(BigDecimal value)

parseColumnName

protected void parseColumnName(TokenStream.Marker start,
                               Map<String,String> tableAliasByColumnAliases,
                               Map<String,String> columnNameByAliases)

Parse the potentially qualified and aliased column information, and add the information to the supplied maps.

Parameters:

start - the start of the statement

tableAliasByColumnAliases - the map to which is added the column's alias (or name) keyed by the alias of the table in which the column should appear; may not be null

columnNameByAliases - the map to which is added the column's name keyed by the its alias (or itself if there is no alias); may not be null

parseSelectFromClause

protected Map<String,Table> parseSelectFromClause(TokenStream.Marker start)

Returns the tables keyed by their aliases that appear in a SELECT clause's "FROM" list. This method handles the various standard joins.

Parameters:

start - the start of the statement

Returns:

the map of resolved tables keyed by the alias (or table name) used in the SELECT statement; never null but possibly empty if we couldn't parse the from clause correctly

canConsumeJoin

protected boolean canConsumeJoin(TokenStream.Marker start)

canConsumeJoinCondition

protected boolean canConsumeJoinCondition(TokenStream.Marker start)

parseAliasedTableInFrom

private void parseAliasedTableInFrom(TokenStream.Marker start,
                                     Map<String,Table> tablesByAlias)

Parse a potentially qualified table name along with an optional alias.

Parameters:

start - the start of the statement

tablesByAlias - the map to which this method should add the table keyed by its alias (or name if there is no alias); may not be null