Package io.debezium.connector.mysql.legacy

Class AbstractReader

java.lang.Object

io.debezium.connector.mysql.legacy.AbstractReader

All Implemented Interfaces:

Reader

Direct Known Subclasses:

BinlogReader, SnapshotReader

public abstract class AbstractReader
extends Object
implements Reader

A component that performs a snapshot of a MySQL server, and records the schema changes in MySqlSchema.

Author:

Randall Hauch

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	AbstractReader.AcceptAllPredicate	A predicate that returns true for all sourceRecords

Nested classes/interfaces inherited from interface io.debezium.connector.mysql.legacy.Reader

Reader.State

Field Summary

Fields

Modifier and Type	Field	Description
private final HaltingPredicate	acceptAndContinue
protected final ChangeEventQueueMetrics	changeEventQueueMetrics
protected final MySqlJdbcContext	connectionContext
protected final MySqlTaskContext	context
private final AtomicReference<org.apache.kafka.connect.errors.ConnectException>	failure
private org.apache.kafka.connect.errors.ConnectException	failureException
protected final org.slf4j.Logger	logger
private final int	maxBatchSize
private final Metronome	metronome
private final String	name
private final AtomicReference<MySqlPartition>	partition
private final Duration	pollInterval
private final BlockingQueue<org.apache.kafka.connect.source.SourceRecord>	records
private final AtomicBoolean	running
private final AtomicBoolean	success
private final AtomicReference<Consumer<MySqlPartition>>	uponCompletion

Constructor Summary

Constructors

Constructor	Description
AbstractReader(String name, MySqlTaskContext context, HaltingPredicate acceptAndContinue)	Create a snapshot reader.

Method Summary

Modifier and Type	Method	Description
protected void	cleanupResources(MySqlPartition partition)	This method is normally called by poll() when there this reader finishes normally and all generated records are consumed prior to being stopped.
protected void	completeSuccessfully()	Call this method only when the reader has successfully completed all of its work, signaling that subsequent calls to poll() should forever return null and that this reader should transition from Reader.State.STOPPING to Reader.State.STOPPED.
final void	destroy()	After the reader has stopped, there may still be some resources we want left available until the connector task is destroyed.
protected abstract void	doCleanup()	The reader has completed all processing and all enqueued records have been consumed, so this reader should clean up any resources that might remain.
protected void	doDestroy()	The reader has been requested to de-initialize resources after stopping.
protected void	doInitialize()	The reader has been requested to initialize resources prior to starting.
protected abstract void	doStart(MySqlPartition partition)	The reader has been requested to start, so initialize any un-initialized resources required by the reader.
protected abstract void	doStop(MySqlPartition partition)	The reader has been requested to stop, so perform any work required to stop the reader's resources that were previously started.
protected void	enqueueRecord(org.apache.kafka.connect.source.SourceRecord record)	Enqueue a record so that it can be obtained when this reader is polled.
protected void	failed(Throwable error)	Call this method only when the reader has failed, that a subsequent call to poll() should throw this error, and that doCleanup() can be called at any time.
protected void	failed(Throwable error, String msg)	Call this method only when the reader has failed, that a subsequent call to poll() should throw this error, and that doCleanup() can be called at any time.
final void	initialize()	Perform any initialization of the reader before being started.
protected boolean	isRunning()
String	name()	Get the name of this reader.
List<org.apache.kafka.connect.source.SourceRecord>	poll()	Poll for the next batch of source records.
protected void	pollComplete(List<org.apache.kafka.connect.source.SourceRecord> batch)	Method called when poll() completes sending a non-zero-sized batch of records.
void	start(MySqlPartition partition)	Start the reader and return immediately.
Reader.State	state()	Get the current state of this reader.
void	stop()	Stop the reader from running and transition to the Reader.State.STOPPING state until all remaining records are consumed, at which point its state transitions to Reader.State.STOPPED.
String	toString()
void	uponCompletion(Consumer<MySqlPartition> handler)	Set the function that should be called when this reader transitions from the Reader.State.STOPPING to Reader.State.STOPPED state, which is after all generated records have been consumed via the poll method.
protected org.apache.kafka.connect.errors.ConnectException	wrap(Throwable error)	Wraps the specified exception in a ConnectException, ensuring that all useful state is captured inside the new exception's message.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Details

logger

protected final org.slf4j.Logger logger

name

private final String name

context

protected final MySqlTaskContext context

connectionContext

protected final MySqlJdbcContext connectionContext

records

private final BlockingQueue<org.apache.kafka.connect.source.SourceRecord> records

running

private final AtomicBoolean running

partition

private final AtomicReference<MySqlPartition> partition

success

private final AtomicBoolean success

failure

private final AtomicReference<org.apache.kafka.connect.errors.ConnectException> failure

failureException

private org.apache.kafka.connect.errors.ConnectException failureException

maxBatchSize

private final int maxBatchSize

metronome

private final Metronome metronome

uponCompletion

private final AtomicReference<Consumer<MySqlPartition>> uponCompletion

pollInterval

private final Duration pollInterval

changeEventQueueMetrics

protected final ChangeEventQueueMetrics changeEventQueueMetrics

acceptAndContinue

private final HaltingPredicate acceptAndContinue

Constructor Details

AbstractReader

public AbstractReader(String name,
 MySqlTaskContext context,
 HaltingPredicate acceptAndContinue)

Create a snapshot reader.

Parameters:

name - the name of the reader

context - the task context in which this reader is running; may not be null

acceptAndContinue - a predicate that returns true if the tested SourceRecord should be accepted and false if the record and all subsequent records should be ignored. The reader will stop accepting records once enqueueRecord(SourceRecord) is called with a record that tests as false. Can be null. If null, all records will be accepted.

Method Details

name

public String name()

Description copied from interface: Reader

Get the name of this reader.

Specified by:

name in interface Reader

Returns:

the reader's name; never null

uponCompletion

public void uponCompletion(Consumer<MySqlPartition> handler)

Description copied from interface: Reader

Set the function that should be called when this reader transitions from the Reader.State.STOPPING to Reader.State.STOPPED state, which is after all generated records have been consumed via the poll method.

This method should only be called while the reader is in the Reader.State.STOPPED state.

Specified by:

uponCompletion in interface Reader

Parameters:

handler - the function; may not be null

initialize

public final void initialize()

Description copied from interface: Reader

Perform any initialization of the reader before being started. This method should be called exactly once before Reader.start(MySqlPartition) ()} is called, and it should block until all initialization is completed.

Specified by:

initialize in interface Reader

destroy

public final void destroy()

Description copied from interface: Reader

After the reader has stopped, there may still be some resources we want left available until the connector task is destroyed. This method is used to clean up those remaining resources upon shutdown. This method is effectively the opposite of Reader.initialize(), performing any de-initialization of the reader entity before shutdown. This method should be called exactly once after Reader.stop() is called, and it should block until all de-initialization is completed.

Specified by:

destroy in interface Reader

start

public void start(MySqlPartition partition)

Description copied from interface: Reader

Start the reader and return immediately. Once started, the SourceRecords generated by the reader can be obtained by periodically calling Reader.poll() until that method returns null.

This method does nothing if it is already running.

Specified by:

start in interface Reader

stop

public void stop()

Description copied from interface: Reader

Stop the reader from running and transition to the Reader.State.STOPPING state until all remaining records are consumed, at which point its state transitions to Reader.State.STOPPED.

Specified by:

stop in interface Reader

doInitialize

protected void doInitialize()

The reader has been requested to initialize resources prior to starting. This should only be called once before doStart(MySqlPartition).

doDestroy

protected void doDestroy()

The reader has been requested to de-initialize resources after stopping. This should only be called once after doStop(MySqlPartition).

doStart

protected abstract void doStart(MySqlPartition partition)

The reader has been requested to start, so initialize any un-initialized resources required by the reader.

doStop

protected abstract void doStop(MySqlPartition partition)

The reader has been requested to stop, so perform any work required to stop the reader's resources that were previously started.

This method is always called when Reader.stop() is called, and the first time isRunning() will return true the first time and false for any subsequent calls.

doCleanup

protected abstract void doCleanup()

The reader has completed all processing and all enqueued records have been consumed, so this reader should clean up any resources that might remain.

completeSuccessfully

protected void completeSuccessfully()

Call this method only when the reader has successfully completed all of its work, signaling that subsequent calls to poll() should forever return null and that this reader should transition from Reader.State.STOPPING to Reader.State.STOPPED.

failed

protected void failed(Throwable error)

Call this method only when the reader has failed, that a subsequent call to poll() should throw this error, and that doCleanup() can be called at any time.

Parameters:

error - the error that resulted in the failure; should not be null

failed

protected void failed(Throwable error,
 String msg)

Call this method only when the reader has failed, that a subsequent call to poll() should throw this error, and that doCleanup() can be called at any time.

Parameters:

error - the error that resulted in the failure; should not be null

msg - the error message; may not be null

wrap

protected org.apache.kafka.connect.errors.ConnectException wrap(Throwable error)

Wraps the specified exception in a ConnectException, ensuring that all useful state is captured inside the new exception's message.

Parameters:

error - the exception; may not be null

Returns:

the wrapped Kafka Connect exception

state

public Reader.State state()

Description copied from interface: Reader

Get the current state of this reader.

Specified by:

state in interface Reader

Returns:

the state; never null

isRunning

protected boolean isRunning()

poll

public List<org.apache.kafka.connect.source.SourceRecord> poll()
                                                        throws InterruptedException

Description copied from interface: Reader

Poll for the next batch of source records. This method returns null only when all records generated by this reader have been processed, following the natural or explicit stopping of this reader. Note that this method may block if no additional records are available but the reader may produce more, thus callers should call this method continually until this method returns null.

Specified by:

poll in interface Reader

Returns:

the list of source records that may or may not be empty; or null when there will be no more records because the reader has completely Reader.State.STOPPED.

Throws:

InterruptedException - if this thread is interrupted while waiting for more records

cleanupResources

protected void cleanupResources(MySqlPartition partition)

This method is normally called by poll() when there this reader finishes normally and all generated records are consumed prior to being stopped. However, if this reader is explicitly stopped while still working, then subclasses should call this method when they have completed all of their shutdown work.

pollComplete

protected void pollComplete(List<org.apache.kafka.connect.source.SourceRecord> batch)

Method called when poll() completes sending a non-zero-sized batch of records.

Parameters:

batch - the batch of records being recorded

enqueueRecord

protected void enqueueRecord(org.apache.kafka.connect.source.SourceRecord record)
                      throws InterruptedException

Enqueue a record so that it can be obtained when this reader is polled. This method will block if the queue is full.

Parameters:

record - the record to be enqueued

Throws:

InterruptedException - if interrupted while waiting for the queue to have room for this record

toString

public String toString()

Overrides:

toString in class Object