io.debezium.embedded

Class AbstractConnectorTest

java.lang.Object

io.debezium.embedded.AbstractConnectorTest

All Implemented Interfaces:

Testing

Direct Known Subclasses:

EmbeddedEngineTest

public abstract class AbstractConnectorTest
extends Object
implements Testing

An abstract base class for unit testing SourceConnector implementations using the Debezium EmbeddedEngine with local file storage.

To use this abstract class, simply create a test class that extends it, and add one or more test methods that starts the connector using your connector's custom configuration. Then, your test methods can call consumeRecords(int, Consumer) to consume the specified number of records (the supplied function gives you a chance to do something with the record).

Author:

Randall Hauch

Nested Class Summary

Nested classes/interfaces inherited from interface io.debezium.util.Testing

Testing.Debug, Testing.Files, Testing.InterruptableFunction, Testing.Network, Testing.Print, Testing.Timer

Field Summary

Fields

Modifier and Type	Field and Description
private BlockingQueue<org.apache.kafka.connect.source.SourceRecord>	consumedLines
private io.debezium.embedded.EmbeddedEngine	engine
private ExecutorService	executor
private CountDownLatch	latch
protected org.slf4j.Logger	logger
protected static Path	OFFSET_STORE_PATH
protected long	pollTimeoutInMs

Constructor Summary

Constructors

Constructor and Description
AbstractConnectorTest()

Method Summary

Modifier and Type	Method and Description
protected void	append(Object obj, StringBuilder sb)
protected void	assertConnectorIsRunning() Assert that the connector is currently running.
protected void	assertConnectorNotRunning() Assert that the connector is NOT currently running.
protected void	assertNoRecordsToConsume() Assert that there are no records to consume.
protected int	consumeAvailableRecords(Consumer<org.apache.kafka.connect.source.SourceRecord> recordConsumer) Try to consume all of the messages that have already been returned by the connector.
protected org.apache.kafka.connect.source.SourceRecord	consumeRecord() Consume a single record from the connector.
protected int	consumeRecords(int numberOfRecords) Try to consume the specified number of records from the connector, and return the actual number of records that were consumed.
protected int	consumeRecords(int numberOfRecords, Consumer<org.apache.kafka.connect.source.SourceRecord> recordConsumer) Try to consume the specified number of records from the connector, calling the given function for each, and return the actual number of records that were consumed.
protected int	getMaximumEnqueuedRecordCount() Get the maximum number of messages that can be obtained from the connector and held in-memory before they are consumed by test methods using consumeRecord(), consumeRecords(int), or consumeRecords(int, Consumer).
void	initializeConnectorTestFramework()
protected void	print(org.apache.kafka.connect.source.SourceRecord record)
protected void	setConsumeTimeout(long timeout, TimeUnit unit) Set the maximum amount of time that the consumeRecord(), consumeRecords(int), and consumeRecords(int, Consumer) methods block while waiting for each record before returning null.
protected void	start(Class<? extends org.apache.kafka.connect.source.SourceConnector> connectorClass, io.debezium.config.Configuration connectorConfig) Start the connector using the supplied connector configuration, where upon completion the status of the connector is logged.
protected void	start(Class<? extends org.apache.kafka.connect.source.SourceConnector> connectorClass, io.debezium.config.Configuration connectorConfig, io.debezium.embedded.EmbeddedEngine.CompletionCallback callback) Start the connector using the supplied connector configuration.
void	stopConnector() Stop the connector and block until the connector has completely stopped.
void	stopConnector(BooleanConsumer callback) Stop the connector, and return whether the connector was successfully stopped.
protected boolean	waitForAvailableRecords(long timeout, TimeUnit unit) Wait for a maximum amount of time until the first record is available.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface io.debezium.util.Testing

debug, once, once, print, print, printError, printError, printError, resetBeforeEachTest, time, time

Field Detail

OFFSET_STORE_PATH

protected static final Path OFFSET_STORE_PATH

executor

private ExecutorService executor

engine

private io.debezium.embedded.EmbeddedEngine engine

consumedLines

private BlockingQueue<org.apache.kafka.connect.source.SourceRecord> consumedLines

pollTimeoutInMs

protected long pollTimeoutInMs

logger

protected final org.slf4j.Logger logger

latch

private CountDownLatch latch

Constructor Detail

AbstractConnectorTest

public AbstractConnectorTest()

Method Detail

initializeConnectorTestFramework

public final void initializeConnectorTestFramework()
                                            throws Exception

Throws:

Exception

stopConnector

public final void stopConnector()

Stop the connector and block until the connector has completely stopped.

stopConnector

public void stopConnector(BooleanConsumer callback)

Stop the connector, and return whether the connector was successfully stopped.

Parameters:

callback - the function that should be called with whether the connector was successfully stopped; may be null

getMaximumEnqueuedRecordCount

protected int getMaximumEnqueuedRecordCount()

Get the maximum number of messages that can be obtained from the connector and held in-memory before they are consumed by test methods using consumeRecord(), consumeRecords(int), or consumeRecords(int, Consumer).

By default this method return 100.

Returns:

the maximum number of records that can be enqueued

start

protected void start(Class<? extends org.apache.kafka.connect.source.SourceConnector> connectorClass,
                     io.debezium.config.Configuration connectorConfig)

Start the connector using the supplied connector configuration, where upon completion the status of the connector is logged.

Parameters:

connectorClass - the connector class; may not be null

connectorConfig - the configuration for the connector; may not be null

start

protected void start(Class<? extends org.apache.kafka.connect.source.SourceConnector> connectorClass,
                     io.debezium.config.Configuration connectorConfig,
                     io.debezium.embedded.EmbeddedEngine.CompletionCallback callback)

Start the connector using the supplied connector configuration.

Parameters:

connectorClass - the connector class; may not be null

connectorConfig - the configuration for the connector; may not be null

callback - the function that will be called when the engine fails to start the connector or when the connector stops running after completing successfully or due to an error; may be null

setConsumeTimeout

protected void setConsumeTimeout(long timeout,
                                 TimeUnit unit)

Set the maximum amount of time that the consumeRecord(), consumeRecords(int), and consumeRecords(int, Consumer) methods block while waiting for each record before returning null.

Parameters:

timeout - the timeout; must be positive

unit - the time unit; may not be null

consumeRecord

protected org.apache.kafka.connect.source.SourceRecord consumeRecord()
                                                              throws InterruptedException

Consume a single record from the connector.

Returns:

the next record that was returned from the connector, or null if no such record has been produced by the connector

Throws:

InterruptedException - if the thread was interrupted while waiting for a record to be returned

consumeRecords

protected int consumeRecords(int numberOfRecords)
                      throws InterruptedException

Try to consume the specified number of records from the connector, and return the actual number of records that were consumed. Use this method when your test does not care what the records might contain.

Parameters:

numberOfRecords - the number of records that should be consumed

Returns:

the actual number of records that were consumed

Throws:

InterruptedException - if the thread was interrupted while waiting for a record to be returned

consumeRecords

protected int consumeRecords(int numberOfRecords,
                             Consumer<org.apache.kafka.connect.source.SourceRecord> recordConsumer)
                      throws InterruptedException

Try to consume the specified number of records from the connector, calling the given function for each, and return the actual number of records that were consumed.

Parameters:

numberOfRecords - the number of records that should be consumed

recordConsumer - the function that should be called with each consumed record

Returns:

the actual number of records that were consumed

Throws:

InterruptedException - if the thread was interrupted while waiting for a record to be returned

consumeAvailableRecords

protected int consumeAvailableRecords(Consumer<org.apache.kafka.connect.source.SourceRecord> recordConsumer)

Try to consume all of the messages that have already been returned by the connector.

Parameters:

recordConsumer - the function that should be called with each consumed record

Returns:

the number of records that were consumed

waitForAvailableRecords

protected boolean waitForAvailableRecords(long timeout,
                                          TimeUnit unit)

Wait for a maximum amount of time until the first record is available.

Parameters:

timeout - the maximum amount of time to wait; must not be negative

unit - the time unit for timeout

Returns:

true if records are available, or false if the timeout occurred and no records are available

assertConnectorIsRunning

protected void assertConnectorIsRunning()

Assert that the connector is currently running.

assertConnectorNotRunning

protected void assertConnectorNotRunning()

Assert that the connector is NOT currently running.

assertNoRecordsToConsume

protected void assertNoRecordsToConsume()

Assert that there are no records to consume.

print

protected void print(org.apache.kafka.connect.source.SourceRecord record)

append

protected void append(Object obj,
                      StringBuilder sb)