Package io.debezium.engine

Interface DebeziumEngine<R>

All Superinterfaces:

AutoCloseable, Closeable, Runnable

public interface DebeziumEngine<R>
extends Runnable, Closeable

A mechanism for running a single Kafka Connect SourceConnector within an application's process. The engine is entirely standalone and only talks with the source system; no Kafka, Kafka Connect, or Zookeeper processes are needed. Applications using the engine simply set one up and supply a consumer function to which the engine will pass all records containing database change events.

With the engine, the application that runs the connector assumes all responsibility for fault tolerance, scalability, and durability. Additionally, applications must specify how the engine can store its relational database schema history and offsets. By default, this information will be stored in memory and will thus be lost upon application restart.

Engine Is designed to be submitted to an Executor or ExecutorService for execution by a single thread, and a running connector can be stopped either by calling #stop() from another thread or by interrupting the running thread (e.g., as is the case with ExecutorService.shutdownNow()).

Author:

Randall Hauch

Nested Class Summary

Nested Classes

Modifier and Type	Interface	Description
static interface	DebeziumEngine.Builder<R>	A builder to set up and create DebeziumEngine instances.
static interface	DebeziumEngine.BuilderFactory	Internal contract between the API and implementation, for bootstrapping the latter.
static interface	DebeziumEngine.ChangeConsumer<R>	A contract invoked by the embedded engine when it has received a batch of change records to be processed.
static interface	DebeziumEngine.CompletionCallback	A callback function to be notified when the connector completes.
static interface	DebeziumEngine.ConnectorCallback	Callback function which informs users about the various stages a connector goes through during startup
static interface	DebeziumEngine.Offsets	Contract that should be passed to DebeziumEngine.RecordCommitter.markProcessed(Object, Offsets) for marking a record as processed with updated offsets.
static interface	DebeziumEngine.RecordCommitter<R>	Contract passed to DebeziumEngine.ChangeConsumers, allowing them to commit single records as they have been processed and to signal that offsets may be flushed eventually.

Field Summary

Fields

Modifier and Type	Field	Description
static final String	OFFSET_FLUSH_INTERVAL_MS_PROP

Method Summary

Static Methods

Modifier and Type	Method	Description
static <T, V extends SerializationFormat<T>> DebeziumEngine.Builder<RecordChangeEvent<T>>	create(ChangeEventFormat<V> format)	Obtain a new DebeziumEngine.Builder instance that can be used to construct runnable DebeziumEngine instances.
static <S, T, K extends SerializationFormat<S>, V extends SerializationFormat<T>> DebeziumEngine.Builder<ChangeEvent<S,T>>	create(KeyValueChangeEventFormat<K,V> format)
static <S, T, U, K extends SerializationFormat<S>, V extends SerializationFormat<T>, H extends SerializationFormat<U>> DebeziumEngine.Builder<ChangeEvent<S,T>>	create(KeyValueHeaderChangeEventFormat<K,V,H> format)
static <K, V> DebeziumEngine.Builder<ChangeEvent<K,V>>	create(Class<? extends SerializationFormat<K>> keyFormat, Class<? extends SerializationFormat<V>> valueFormat)	Obtain a new DebeziumEngine.Builder instance that can be used to construct runnable DebeziumEngine instances.
static <K, V, H> DebeziumEngine.Builder<ChangeEvent<K,V>>	create(Class<? extends SerializationFormat<K>> keyFormat, Class<? extends SerializationFormat<V>> valueFormat, Class<? extends SerializationFormat<H>> headerFormat)	Obtain a new DebeziumEngine.Builder instance that can be used to construct runnable DebeziumEngine instances.
static <T> DebeziumEngine.Builder<ChangeEvent<T,T>>	create(Class<? extends SerializationFormat<T>> format)	Obtain a new DebeziumEngine.Builder instance that can be used to construct runnable DebeziumEngine instances.
private static DebeziumEngine.BuilderFactory	determineBuilderFactory()

Methods inherited from interface java.io.Closeable

close

Methods inherited from interface java.lang.Runnable

run

Field Details

OFFSET_FLUSH_INTERVAL_MS_PROP

static final String OFFSET_FLUSH_INTERVAL_MS_PROP

See Also:

• Constant Field Values

Method Details

create

static <T> DebeziumEngine.Builder<ChangeEvent<T,T>> create(Class<? extends SerializationFormat<T>> format)

Obtain a new DebeziumEngine.Builder instance that can be used to construct runnable DebeziumEngine instances. The same format is used for key and the value of emitted change events.

Convenience method, equivalent to calling create(KeyValueChangeEventFormat.of(MyFormat.class, MyFormat.class).

Returns:

the new builder; never null

create

static <K,
V>
DebeziumEngine.Builder<ChangeEvent<K,V>> create(Class<? extends SerializationFormat<K>> keyFormat,
 Class<? extends SerializationFormat<V>> valueFormat)

Obtain a new DebeziumEngine.Builder instance that can be used to construct runnable DebeziumEngine instances. Different formats are used for key and the value of emitted change events.

Convenience method, equivalent to calling create(KeyValueChangeEventFormat.of(MyKeyFormat.class, MyValueFormat.class).

Returns:

the new builder; never null

create

static <K,
V,
H>
DebeziumEngine.Builder<ChangeEvent<K,V>> create(Class<? extends SerializationFormat<K>> keyFormat,
 Class<? extends SerializationFormat<V>> valueFormat,
 Class<? extends SerializationFormat<H>> headerFormat)

Obtain a new DebeziumEngine.Builder instance that can be used to construct runnable DebeziumEngine instances. Different formats are used for key, value, and headers of emitted change events.

Convenience method, equivalent to calling create(KeyValueChangeEventFormat.of(MyKeyFormat.class, MyValueFormat.class, MyHeaderFormat.class).

Returns:

the new builder; never null

create

static <S,
T,
K extends SerializationFormat<S>,
V extends SerializationFormat<T>>
DebeziumEngine.Builder<ChangeEvent<S,T>> create(KeyValueChangeEventFormat<K,V> format)

create

static <S,
T,
U,
K extends SerializationFormat<S>,
V extends SerializationFormat<T>,
H extends SerializationFormat<U>>
DebeziumEngine.Builder<ChangeEvent<S,T>> create(KeyValueHeaderChangeEventFormat<K,V,H> format)

create

static <T,
V extends SerializationFormat<T>>
DebeziumEngine.Builder<RecordChangeEvent<T>> create(ChangeEventFormat<V> format)

Obtain a new DebeziumEngine.Builder instance that can be used to construct runnable DebeziumEngine instances. Emitted change events encapsulate both key and value.

Returns:

the new builder; never null

determineBuilderFactory

private static DebeziumEngine.BuilderFactory determineBuilderFactory()