io.debezium.embedded

Class ConnectorOutputTest

java.lang.Object

io.debezium.embedded.ConnectorOutputTest

Direct Known Subclasses:

SimpleSourceConnectorOutputTest

public abstract class ConnectorOutputTest
extends Object

A base class for classes the define integration tests for connectors that verify the output of a connector matches an expected sequence of change event records. This base class provides a simple framework for running a connector given a configuration and either recording or comparing the connector's output to expected results.

Overview

One of the common and easiest ways to test a connector is to simply run it and compare the changes events output by the connector with a sequence of expected change events. Manually writing tests to compare the results is tedious and error prone, so an alternative is to run the test once and record the change events output by the connector so that these change events can be used as expected results in subsequent runs of the same test.

This framework makes it possible to define tests that start a connector, record the change events output by the connector, and compare these change events to expected results. Connectors are defined via properties files, and the expected results are recorded to or read from files containing the sequence of change events serialized as an array of JSON documents. The test can also define an optional environment properties file in which test-specific parameters can be specified.

Variations in results

Some fields in the output will change every time the connector is run. For example, a change event may include the timestamp at which the connector processed/generated the event, or it might include a transaction identifier that is different with each run. The framework works around these issues by allowing the test to specify in the environment parameters the fields in the change events that should be ignored when comparing actual change events to expected change events.

Other fields may vary every time the database is started. For example, a MySQL server that is configured to use GTIDs will generate a new server_uuid value when it is initialized, and this value is then used in all GTIDs generated by that server instance. Since multiple builds will start up and initialize a new MySQL server (in a Docker container), the GTID will vary with each build. The framework can handle this by allowing the expected results to contain variables in the form of ${variableName1}, and the framework will use values supplied during test setup to replace these variables prior to parsing the JSON document for each expected change event. The framework allows the test setup to supply a function that provides the replacement variable values given the connector configuration. So the MySQL connector tests, for example, can connect to the MySQL server and return all of the system variables as the variable values; the framework does all the rest.

Stopping the connector

When the test framework has compared all of the generated change events to the expected change events and found no discrepancies, the framework knows that the connector should generate no additional change events. If the connector does generate another event, the test will fail. Otherwise, the framework will stop the connector since it is no longer needed.

Connector restarts

Another requirement is to verify that a connector will resume where it left off when it is restarted following a failure or graceful shutdown. This is normally a bit harder to do, but this framework makes it very easy. Simply insert in the sequence of expected change events a restart command. The framework consumes each change event generated by the connector one at a time, and for each it compares it to the expected change event. As the framework comes across a restart command in the expected results, it knows to stop the connector (persisting the offsets in the last change event processed by the connector) and to then restart it given those persisted offsets. A properly-written connector will resume exactly where it left off, and the framework can once again begin comparing the resulting change events to the next events in the expected results.

Example

A simple test runs a connector and records/compares the connector's output to expected results:

 @Test
 public void shouldCaptureChangesFromMySqlWithNoGtids() {
     runConnector(usingSpec("test1", "src/test/resources/expected/test1"));
 }
 

The framework uses a test specification that encapsulates the connector configuration, the environment properties, how to obtain the variable values, and where to find the results. This example creates a test specification with the name test1 and the connector properties file, the environment properties file, and the expected results file all defined in a directory named src/test/resources/expected/test1. Once again, the test will write out the expected results file if none exists; otherwise, it will read in the expected results and compare them to the events output by the connector.

The framework supports other variations, and the specWith method is a simple convenience method that replaces explicitly building up the test specification by using lots of defaults. Consequently, the framework offers a great deal of flexibility.

To see more detail about the actual and expected records, turn on Debezium's test debug mode by using this line before the runConnector call:

 Testing.Debug.enable();
 

Author:

Randall Hauch

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
private static class	ConnectorOutputTest.ConsumerCompletion
private static class	ConnectorOutputTest.ExecutionResult
protected static interface	ConnectorOutputTest.InputStreamSupplier
protected static interface	ConnectorOutputTest.OutputStreamSupplier
private static class	ConnectorOutputTest.SchemaAndValueConverter
static interface	ConnectorOutputTest.TestData
static class	ConnectorOutputTest.TestSpecification
protected static interface	ConnectorOutputTest.VariableSupplier

Field Summary

Fields

Modifier and Type	Field and Description
static Path	CONNECTOR_OUTPUT_PATH Before each test this class will delete the "target/connector-output" directory under the project in which the test is run.
static String	CONTROL_END
static String	CONTROL_KEY
static String	CONTROL_RESTART
static String	CONTROL_STOP
static String	DEFAULT_CONNECTOR_PROPERTIES_FILENAME
static String	DEFAULT_ENV_PROPERTIES_FILENAME
static String	DEFAULT_EXPECTED_RECORDS_FILENAME
static String	ENV_CONNECTOR_TIMEOUT_IN_SECONDS
static String	ENV_IGNORE_FIELDS
protected static Path	OFFSET_STORE_PATH

Constructor Summary

Constructors

Constructor and Description
ConnectorOutputTest()

Method Summary

Modifier and Type	Method and Description
protected void	addValueComparatorsByFieldPath(BiConsumer<String,VerifyRecord.RecordValueComparator> comparatorsByPath)
protected void	addValueComparatorsBySchemaName(BiConsumer<String,VerifyRecord.RecordValueComparator> comparatorsByPath)
void	afterEachTestMethod()
private void	applyCommand(Document record, ConnectorOutputTest.ConsumerCompletion result)
private void	assertSourceRecordMatch(org.apache.kafka.connect.source.SourceRecord actual, org.apache.kafka.connect.source.SourceRecord expected, Predicate<String> ignoreFields, Map<String,VerifyRecord.RecordValueComparator> comparatorsByName, Map<String,VerifyRecord.RecordValueComparator> comparatorsBySchemaName)
void	cleanOffsetStorage()
private <T> Queue<T>	fixedSizeQueue(int size)
private String	getCommand(Document record)
protected String[]	globallyIgnorableFieldNames() Return the names of the fields that should always be ignored for all tests.
private boolean	isCommand(Document record)
private boolean	isEndCommand(Document record)
private org.apache.kafka.connect.source.SourceRecord	rehydrateSourceRecord(Document record, ConnectorOutputTest.SchemaAndValueConverter keyConverter, ConnectorOutputTest.SchemaAndValueConverter valueConverter)
protected static File	replaceVariables(InputStream stream, AvailableVariables variables) Read the contents of the supplied InputStream, replace all variables found in the content, and write the result to a temporary file.
protected void	runConnector(ConnectorOutputTest.TestSpecification spec) Run the connector described by the supplied test specification.
protected void	runConnector(ConnectorOutputTest.TestSpecification spec, io.debezium.embedded.EmbeddedEngine.CompletionCallback callback) Run the connector described by the supplied test specification.
protected void	runConnector(String testName, Path directory) Run the connector that uses the files in the given directory for the connector configuration, environment configuration, and expected results.
protected void	runConnector(String testName, Path configFile, Path expectedRecordsFile, Path envFile) Run the connector described by the supplied test specification.
protected void	runConnector(String testName, String directory) Run the connector that uses the files in the given directory for the connector configuration, environment configuration, and expected results.
protected void	runConnector(String testName, String configFile, String expectedRecordsFile, String envFile) Run the connector described by the supplied test specification.
private Document	serializeSourceRecord(org.apache.kafka.connect.source.SourceRecord record, ConnectorOutputTest.SchemaAndValueConverter keyConverter, ConnectorOutputTest.SchemaAndValueConverter valueConverter) Serialize the source record to document form.
private Map<String,?>	toMap(Document doc)
protected ConnectorOutputTest.TestSpecification	usingSpec(String name) Create a new test specification with the given name.
protected ConnectorOutputTest.TestSpecification	usingSpec(String name, Path directory) Create a new test specification that uses the files in the given directory for the connector configuration, environment configuration, and expected results.
protected ConnectorOutputTest.TestSpecification	usingSpec(String name, Path configFile, Path expectedRecordsFile, Path envFile) Create a new test specification that uses the given files for the configuration, environment, and expected results.
protected ConnectorOutputTest.TestSpecification	usingSpec(String name, String directory) Create a new test specification that uses the files in the given directory for the connector configuration, environment configuration, and expected results.
protected ConnectorOutputTest.TestSpecification	usingSpec(String name, String configFile, String expectedRecordsFile, String envFile) Create a new test specification that uses the given files for the configuration, environment, and expected results.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_CONNECTOR_PROPERTIES_FILENAME

public static final String DEFAULT_CONNECTOR_PROPERTIES_FILENAME

See Also:

Constant Field Values

DEFAULT_ENV_PROPERTIES_FILENAME

public static final String DEFAULT_ENV_PROPERTIES_FILENAME

See Also:

Constant Field Values

DEFAULT_EXPECTED_RECORDS_FILENAME

public static final String DEFAULT_EXPECTED_RECORDS_FILENAME

See Also:

Constant Field Values

ENV_CONNECTOR_TIMEOUT_IN_SECONDS

public static final String ENV_CONNECTOR_TIMEOUT_IN_SECONDS

See Also:

Constant Field Values

ENV_IGNORE_FIELDS

public static final String ENV_IGNORE_FIELDS

See Also:

Constant Field Values

CONNECTOR_OUTPUT_PATH

public static final Path CONNECTOR_OUTPUT_PATH

Before each test this class will delete the "target/connector-output" directory under the project in which the test is run.

OFFSET_STORE_PATH

protected static final Path OFFSET_STORE_PATH

CONTROL_KEY

public static final String CONTROL_KEY

See Also:

Constant Field Values

CONTROL_RESTART

public static final String CONTROL_RESTART

See Also:

Constant Field Values

CONTROL_STOP

public static final String CONTROL_STOP

See Also:

Constant Field Values

CONTROL_END

public static final String CONTROL_END

See Also:

Constant Field Values

Constructor Detail

ConnectorOutputTest

public ConnectorOutputTest()

Method Detail

addValueComparatorsByFieldPath

protected void addValueComparatorsByFieldPath(BiConsumer<String,VerifyRecord.RecordValueComparator> comparatorsByPath)

addValueComparatorsBySchemaName

protected void addValueComparatorsBySchemaName(BiConsumer<String,VerifyRecord.RecordValueComparator> comparatorsByPath)

usingSpec

protected ConnectorOutputTest.TestSpecification usingSpec(String name)

Create a new test specification with the given name.

Parameters:

name - the name of the test

Returns:

the test specification; never null

usingSpec

protected ConnectorOutputTest.TestSpecification usingSpec(String name,
                                                          String directory)

Create a new test specification that uses the files in the given directory for the connector configuration, environment configuration, and expected results. These names of these files are as follows:

• "connector.properties" for the connector configuration file and is required;
• "connector.properties" for the environment configuration file and is optional; and
• "expected-records.json" for the expected results file and will be generated if missing.

Parameters:

name - the name of the test

directory - the path to the directory that contains the test files; may not be null

Returns:

the test specification; never null

usingSpec

protected ConnectorOutputTest.TestSpecification usingSpec(String name,
                                                          Path directory)

Create a new test specification that uses the files in the given directory for the connector configuration, environment configuration, and expected results. These names of these files are as follows:

• "connector.properties" for the connector configuration file and is required;
• "connector.properties" for the environment configuration file and is optional; and
• "expected-records.json" for the expected results file and will be generated if missing.

Parameters:

name - the name of the test

directory - the path to the directory that contains the test files; may not be null

Returns:

the test specification; never null

usingSpec

protected ConnectorOutputTest.TestSpecification usingSpec(String name,
                                                          String configFile,
                                                          String expectedRecordsFile,
                                                          String envFile)

Create a new test specification that uses the given files for the configuration, environment, and expected results.

Parameters:

name - the name of the test or test run; may not be null

configFile - the path to the configuration file; may not be null

expectedRecordsFile - the path to the file where the expected records can be read or where they are to be written; may not be null

envFile - the path to the file containing the environment properties; may be null if there is no such file

Returns:

the test specification; never null

usingSpec

protected ConnectorOutputTest.TestSpecification usingSpec(String name,
                                                          Path configFile,
                                                          Path expectedRecordsFile,
                                                          Path envFile)

Create a new test specification that uses the given files for the configuration, environment, and expected results.

Parameters:

name - the name of the test or test run; may not be null

configFile - the path to the configuration file; may not be null

expectedRecordsFile - the path to the file where the expected records can be read or where they are to be written; may not be null

envFile - the path to the file containing the environment properties; may be null if there is no such file

Returns:

the test specification; never null

cleanOffsetStorage

public void cleanOffsetStorage()

afterEachTestMethod

public void afterEachTestMethod()

globallyIgnorableFieldNames

protected String[] globallyIgnorableFieldNames()

Return the names of the fields that should always be ignored for all tests. By default this method returns null.

Returns:

the array of field names that should always be ignored, or empty or null if there are no such fields

runConnector

protected void runConnector(String testName,
                            String directory)

Run the connector that uses the files in the given directory for the connector configuration, environment configuration, and expected results. These names of these files are as follows:

• "connector.properties" for the connector configuration file and is required;
• "connector.properties" for the environment configuration file and is optional; and
• "expected-records.json" for the expected results file and will be generated if missing.

Parameters:

testName - the name of the test or test run; may not be null

directory - the path to the directory that contains the test files; may not be null

runConnector

protected void runConnector(String testName,
                            Path directory)

Run the connector that uses the files in the given directory for the connector configuration, environment configuration, and expected results. These names of these files are as follows:

• "connector.properties" for the connector configuration file and is required;
• "connector.properties" for the environment configuration file and is optional; and
• "expected-records.json" for the expected results file and will be generated if missing.

Parameters:

testName - the name of the test or test run; may not be null

directory - the path to the directory that contains the test files; may not be null

runConnector

protected void runConnector(String testName,
                            String configFile,
                            String expectedRecordsFile,
                            String envFile)

Run the connector described by the supplied test specification.

Parameters:

testName - the name of the test or test run; may not be null

configFile - the path to the configuration file; may not be null

expectedRecordsFile - the path to the file where the expected records can be read or where they are to be written; may not be null

envFile - the path to the file containing the environment properties; may be null if there is no such file

runConnector

protected void runConnector(String testName,
                            Path configFile,
                            Path expectedRecordsFile,
                            Path envFile)

Run the connector described by the supplied test specification.

Parameters:

testName - the name of the test or test run; may not be null

configFile - the path to the configuration file; may not be null

expectedRecordsFile - the path to the file where the expected records can be read or where they are to be written; may not be null

envFile - the path to the file containing the environment properties; may be null if there is no such file

runConnector

protected void runConnector(ConnectorOutputTest.TestSpecification spec)

Run the connector described by the supplied test specification.

Parameters:

spec - the test specification

runConnector

protected void runConnector(ConnectorOutputTest.TestSpecification spec,
                            io.debezium.embedded.EmbeddedEngine.CompletionCallback callback)

Run the connector described by the supplied test specification.

Parameters:

spec - the test specification

callback - the function that should be called when the connector is stopped

applyCommand

private void applyCommand(Document record,
                          ConnectorOutputTest.ConsumerCompletion result)

isCommand

private boolean isCommand(Document record)

isEndCommand

private boolean isEndCommand(Document record)

getCommand

private String getCommand(Document record)

rehydrateSourceRecord

private org.apache.kafka.connect.source.SourceRecord rehydrateSourceRecord(Document record,
                                                                           ConnectorOutputTest.SchemaAndValueConverter keyConverter,
                                                                           ConnectorOutputTest.SchemaAndValueConverter valueConverter)
                                                                    throws IOException

Throws:

IOException

serializeSourceRecord

private Document serializeSourceRecord(org.apache.kafka.connect.source.SourceRecord record,
                                       ConnectorOutputTest.SchemaAndValueConverter keyConverter,
                                       ConnectorOutputTest.SchemaAndValueConverter valueConverter)
                                throws IOException

Serialize the source record to document form.

Parameters:

record - the record; may not be null

keyConverter - the converter for the record key's schema and payload

valueConverter - the converter for the record value's schema and payload

Returns:

the document form of the source record; never null

Throws:

IOException - if there is an error converting the key or value

assertSourceRecordMatch

private void assertSourceRecordMatch(org.apache.kafka.connect.source.SourceRecord actual,
                                     org.apache.kafka.connect.source.SourceRecord expected,
                                     Predicate<String> ignoreFields,
                                     Map<String,VerifyRecord.RecordValueComparator> comparatorsByName,
                                     Map<String,VerifyRecord.RecordValueComparator> comparatorsBySchemaName)

toMap

private Map<String,?> toMap(Document doc)

fixedSizeQueue

private <T> Queue<T> fixedSizeQueue(int size)

replaceVariables

protected static File replaceVariables(InputStream stream,
                                       AvailableVariables variables)
                                throws IOException

Read the contents of the supplied InputStream, replace all variables found in the content, and write the result to a temporary file.

Parameters:

stream - the input stream containing zero or more variable expressions

variables - the variables

Returns:

the temporary file that exists in the data directory

Throws:

IOException - if there is a problem reading the input stream or writing to the temporary file