io.debezium.connector.common

Class BaseSourceTask

java.lang.Object

org.apache.kafka.connect.source.SourceTask

io.debezium.connector.common.BaseSourceTask

All Implemented Interfaces:

org.apache.kafka.connect.connector.Task

public abstract class BaseSourceTask
extends org.apache.kafka.connect.source.SourceTask

Base class for Debezium's CDC SourceTask implementations. Provides functionality common to all connectors, such as validation of the configuration.

Author:

Gunnar Morling

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	BaseSourceTask.State

Field Summary

Fields

Modifier and Type	Field and Description
private ChangeEventSourceCoordinator	coordinator The change event source coordinator for those connectors adhering to the new framework structure, null for legacy-style connectors.
private Map<String,?>	lastOffset The latest offset that has been acknowledged by the Kafka producer.
private static org.slf4j.Logger	LOGGER
private Map<String,String>	props Raw connector properties, kept here so they can be passed again in case of a restart.
private ElapsedTimeStrategy	restartDelay
private Duration	retriableRestartWait
private AtomicReference<BaseSourceTask.State>	state
private ReentrantLock	stateLock Used to ensure that start(), stop() and commitRecord() calls are serialized.

Fields inherited from class org.apache.kafka.connect.source.SourceTask

context

Constructor Summary

Constructors

Constructor and Description
BaseSourceTask()

Method Summary

Modifier and Type	Method and Description
void	commit()
void	commitRecord(org.apache.kafka.connect.source.SourceRecord record)
protected abstract List<org.apache.kafka.connect.source.SourceRecord>	doPoll() Returns the next batch of source records, if any are available.
protected abstract void	doStop()
protected abstract Iterable<Field>	getAllConfigurationFields() Returns all configuration Field supported by this source task.
protected OffsetContext	getPreviousOffset(OffsetContext.Loader loader) Loads the connector's persistent offset (if present) via the given loader.
List<org.apache.kafka.connect.source.SourceRecord>	poll()
protected abstract ChangeEventSourceCoordinator	start(Configuration config) Called once when starting this source task.
void	start(Map<String,String> props)
private boolean	startIfNeededAndPossible() Starts this connector in case it has been stopped after a retriable error, and the backoff period has passed.
void	stop()
private void	stop(boolean restart)

Methods inherited from class org.apache.kafka.connect.source.SourceTask

commitRecord, initialize

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.kafka.connect.connector.Task

version

Field Detail

LOGGER

private static final org.slf4j.Logger LOGGER

state

private final AtomicReference<BaseSourceTask.State> state

stateLock

private final ReentrantLock stateLock

Used to ensure that start(), stop() and commitRecord() calls are serialized.

restartDelay

private volatile ElapsedTimeStrategy restartDelay

props

private volatile Map<String,String> props

Raw connector properties, kept here so they can be passed again in case of a restart.

coordinator

private ChangeEventSourceCoordinator coordinator

The change event source coordinator for those connectors adhering to the new framework structure, null for legacy-style connectors.

lastOffset

private volatile Map<String,?> lastOffset

The latest offset that has been acknowledged by the Kafka producer. Will be acknowledged with the source database in commit() (which may be a no-op depending on the connector).

retriableRestartWait

private Duration retriableRestartWait

Constructor Detail

BaseSourceTask

public BaseSourceTask()

Method Detail

start

public final void start(Map<String,String> props)

Specified by:

start in interface org.apache.kafka.connect.connector.Task

Specified by:

start in class org.apache.kafka.connect.source.SourceTask

start

protected abstract ChangeEventSourceCoordinator start(Configuration config)

Called once when starting this source task.

Parameters:

config - the task configuration; implementations should wrap it in a dedicated implementation of CommonConnectorConfig and work with typed access to configuration properties that way

poll

public final List<org.apache.kafka.connect.source.SourceRecord> poll()
                                                              throws InterruptedException

Specified by:

poll in class org.apache.kafka.connect.source.SourceTask

Throws:

InterruptedException

doPoll

protected abstract List<org.apache.kafka.connect.source.SourceRecord> doPoll()
                                                                      throws InterruptedException

Returns the next batch of source records, if any are available.

Throws:

InterruptedException

startIfNeededAndPossible

private boolean startIfNeededAndPossible()

Starts this connector in case it has been stopped after a retriable error, and the backoff period has passed.

stop

public final void stop()

Specified by:

stop in interface org.apache.kafka.connect.connector.Task

Specified by:

stop in class org.apache.kafka.connect.source.SourceTask

stop

private void stop(boolean restart)

doStop

protected abstract void doStop()

commitRecord

public void commitRecord(org.apache.kafka.connect.source.SourceRecord record)
                  throws InterruptedException

Overrides:

commitRecord in class org.apache.kafka.connect.source.SourceTask

Throws:

InterruptedException

commit

public void commit()
            throws InterruptedException

Overrides:

commit in class org.apache.kafka.connect.source.SourceTask

Throws:

InterruptedException

getAllConfigurationFields

protected abstract Iterable<Field> getAllConfigurationFields()

Returns all configuration Field supported by this source task.

getPreviousOffset

protected OffsetContext getPreviousOffset(OffsetContext.Loader loader)

Loads the connector's persistent offset (if present) via the given loader.