Package io.debezium.connector.mysql

Class EventBuffer

java.lang.Object

io.debezium.connector.mysql.EventBuffer

class EventBuffer
extends Object

This class represents a look-ahead buffer that allows Debezium to accumulate binlog events and decide if the last event in transaction is either ROLLBACK or COMMIT. The incoming events are either supposed to be in transaction or out-of-transaction. When out-of-transaction they are sent directly into the destination handler. When in transaction the event goes through the buffering.

The reason for the buffering is that the binlog contains rolled back transactions in some cases. E.g. that's the case when a temporary table is dropped (see DBZ-390). For rolled back transactions we may not propagate any of the contained events, hence the buffering is applied.

The transaction start is identified by a BEGIN event. Transaction is ended either by COMMIT event or by XID an event.

If there are more events that can fit to the buffer then:

• Binlog position is recorded for the first event not fitting into the buffer
• Binlog position is recorded for the commit event
• Buffer content is sent to the final handler
• Binlog position is rewound and all events between the above recorded positions are sent to the final handler

Author:

Jiri Pechanec

Field Summary

Fields

Modifier and Type	Field	Description
private Queue<com.github.shyiko.mysql.binlog.event.Event>	buffer
private int	capacity
private ChangeEventSource.ChangeEventSourceContext	changeEventSourceContext
private MySqlStreamingChangeEventSource.BinlogPosition	forwardTillPosition	Contains the position of the last event belonging to the transaction that has not fit into the buffer.
private MySqlStreamingChangeEventSource.BinlogPosition	largeTxNotBufferedPosition	Contains the position of the first event that has not fit into the buffer.
private static org.slf4j.Logger	LOGGER
private MySqlStreamingChangeEventSource	streamingChangeEventSource
private boolean	txStarted

Constructor Summary

Constructors

Constructor	Description
EventBuffer(int capacity, MySqlStreamingChangeEventSource streamingChangeEventSource, ChangeEventSource.ChangeEventSourceContext changeEventSourceContext)

Method Summary

Modifier and Type	Method	Description
void	add(MySqlPartition partition, MySqlOffsetContext offsetContext, com.github.shyiko.mysql.binlog.event.Event event)	An entry point to the buffer that should be used by BinlogReader to push events.
private void	addToBuffer(com.github.shyiko.mysql.binlog.event.Event event)	Adds an event to the buffer if there is a space available.
private void	beginTransaction(MySqlPartition partition, MySqlOffsetContext offsetContext, com.github.shyiko.mysql.binlog.event.Event event)
private void	clear()	Cleans-up the buffer after the transaction is either thrown away or streamed into a Kafka topic
private void	completeTransaction(MySqlPartition partition, MySqlOffsetContext offsetContext, boolean wellFormed, com.github.shyiko.mysql.binlog.event.Event event)	Sends all events from the buffer int a final handler.
private void	consumeEvent(MySqlPartition partition, MySqlOffsetContext offsetContext, com.github.shyiko.mysql.binlog.event.Event event)
private boolean	isInBufferFullMode()
private boolean	isReplayingEventsBeyondBufferCapacity()	Whether we are replaying TX events from binlog that have not fit into the buffer before
private void	rollbackTransaction()
private void	switchToBufferFullMode()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOGGER

private static final org.slf4j.Logger LOGGER

capacity

private final int capacity

buffer

private final Queue<com.github.shyiko.mysql.binlog.event.Event> buffer

streamingChangeEventSource

private final MySqlStreamingChangeEventSource streamingChangeEventSource

txStarted

private boolean txStarted

changeEventSourceContext

private final ChangeEventSource.ChangeEventSourceContext changeEventSourceContext

largeTxNotBufferedPosition

private MySqlStreamingChangeEventSource.BinlogPosition largeTxNotBufferedPosition

Contains the position of the first event that has not fit into the buffer.

forwardTillPosition

private MySqlStreamingChangeEventSource.BinlogPosition forwardTillPosition

Contains the position of the last event belonging to the transaction that has not fit into the buffer.

Constructor Detail

EventBuffer

public EventBuffer(int capacity,
                   MySqlStreamingChangeEventSource streamingChangeEventSource,
                   ChangeEventSource.ChangeEventSourceContext changeEventSourceContext)

Method Detail

add

public void add(MySqlPartition partition,
                MySqlOffsetContext offsetContext,
                com.github.shyiko.mysql.binlog.event.Event event)

An entry point to the buffer that should be used by BinlogReader to push events.

Parameters:

partition - the partition to which the event belongs

event - to be stored in the buffer

isReplayingEventsBeyondBufferCapacity

private boolean isReplayingEventsBeyondBufferCapacity()

Whether we are replaying TX events from binlog that have not fit into the buffer before

addToBuffer

private void addToBuffer(com.github.shyiko.mysql.binlog.event.Event event)

Adds an event to the buffer if there is a space available. Records binlog position for the first event that does not fit for later replay.

Parameters:

event -

switchToBufferFullMode

private void switchToBufferFullMode()

isInBufferFullMode

private boolean isInBufferFullMode()

consumeEvent

private void consumeEvent(MySqlPartition partition,
                          MySqlOffsetContext offsetContext,
                          com.github.shyiko.mysql.binlog.event.Event event)

beginTransaction

private void beginTransaction(MySqlPartition partition,
                              MySqlOffsetContext offsetContext,
                              com.github.shyiko.mysql.binlog.event.Event event)

completeTransaction

private void completeTransaction(MySqlPartition partition,
                                 MySqlOffsetContext offsetContext,
                                 boolean wellFormed,
                                 com.github.shyiko.mysql.binlog.event.Event event)

Sends all events from the buffer int a final handler. For large transactions it executes rewind of binlog reader back to the first event that was not stored in the buffer.

Parameters:

partition - the partition where the transaction was committed

wellFormed -

event -

rollbackTransaction

private void rollbackTransaction()

clear

private void clear()

Cleans-up the buffer after the transaction is either thrown away or streamed into a Kafka topic