io.debezium.connector.oracle.logminer

Class TransactionalBuffer

java.lang.Object

io.debezium.connector.oracle.logminer.TransactionalBuffer

@NotThreadSafe
public final class TransactionalBuffer
extends Object

Author:

Andrey Pustovetov

Transactional buffer is designed to register callbacks, to execute them when transaction commits and to clear them when transaction rollbacks.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	TransactionalBuffer.CommitCallback Callback is designed to execute when transaction commits.
private static class	TransactionalBuffer.Transaction

Field Summary

Fields

Modifier and Type	Field and Description
private Set<String>	abandonedTransactionIds
private Supplier<Integer>	commitQueueCapacity
private io.debezium.pipeline.ErrorHandler	errorHandler
private ExecutorService	executor
private BigDecimal	largestScn
private BigDecimal	lastCommittedScn
private static org.slf4j.Logger	LOGGER
private TransactionalBufferMetrics	metrics
private Set<String>	rolledBackTransactionIds
private AtomicInteger	taskCounter
private Map<String,TransactionalBuffer.Transaction>	transactions

Constructor Summary

Constructors

Constructor and Description
TransactionalBuffer(String logicalName, io.debezium.pipeline.ErrorHandler errorHandler, TransactionalBufferMetrics metrics, int inCommitQueueCapacity) Constructor to create a new instance.

Method Summary

Modifier and Type	Method and Description
(package private) void	abandonLongTransactions(Long thresholdScn) If for some reason the connector got restarted, the offset will point to the beginning of the oldest captured transaction.
private void	calculateLargestScn()
private BigDecimal	calculateSmallestScn()
(package private) void	close() Closes buffer.
(package private) boolean	commit(String transactionId, BigDecimal scn, OracleOffsetContext offsetContext, Timestamp timestamp, io.debezium.pipeline.source.spi.ChangeEventSource.ChangeEventSourceContext context, String debugMessage) If the commit executor queue is full, back-pressure will be applied by letting execution of the callback be performed by the calling thread.
(package private) BigDecimal	getLargestScn()
(package private) Set<String>	getRolledBackTransactionIds()
(package private) boolean	isEmpty() Returns true if buffer is empty, otherwise false.
(package private) void	registerCommitCallback(String transactionId, BigDecimal scn, Instant changeTime, String redoSql, TransactionalBuffer.CommitCallback callback) Registers callback to execute when transaction commits.
(package private) void	resetLargestScn(Long value) Reset Largest SCN
(package private) boolean	rollback(String transactionId, String debugMessage) Clears registered callbacks for given transaction identifier.
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

LOGGER

private static final org.slf4j.Logger LOGGER

transactions

private final Map<String,TransactionalBuffer.Transaction> transactions

executor

private final ExecutorService executor

taskCounter

private final AtomicInteger taskCounter

errorHandler

private final io.debezium.pipeline.ErrorHandler errorHandler

commitQueueCapacity

private final Supplier<Integer> commitQueueCapacity

metrics

private TransactionalBufferMetrics metrics

abandonedTransactionIds

private final Set<String> abandonedTransactionIds

rolledBackTransactionIds

private final Set<String> rolledBackTransactionIds

largestScn

private BigDecimal largestScn

lastCommittedScn

private BigDecimal lastCommittedScn

Constructor Detail

TransactionalBuffer

TransactionalBuffer(String logicalName,
                    io.debezium.pipeline.ErrorHandler errorHandler,
                    TransactionalBufferMetrics metrics,
                    int inCommitQueueCapacity)

Constructor to create a new instance.

Parameters:

logicalName - logical name

errorHandler - logError handler

metrics - metrics MBean

inCommitQueueCapacity - commit queue capacity. On overflow, caller runs task

Method Detail

getLargestScn

BigDecimal getLargestScn()

Returns:

largest last SCN in the buffer among all transactions

getRolledBackTransactionIds

Set<String> getRolledBackTransactionIds()

Returns:

rolled back transactions

resetLargestScn

void resetLargestScn(Long value)

Reset Largest SCN

registerCommitCallback

void registerCommitCallback(String transactionId,
                            BigDecimal scn,
                            Instant changeTime,
                            String redoSql,
                            TransactionalBuffer.CommitCallback callback)

Registers callback to execute when transaction commits.

Parameters:

transactionId - transaction identifier

scn - SCN

changeTime - time of DML parsing completion

redoSql - statement from redo

callback - callback to execute when transaction commits

commit

boolean commit(String transactionId,
               BigDecimal scn,
               OracleOffsetContext offsetContext,
               Timestamp timestamp,
               io.debezium.pipeline.source.spi.ChangeEventSource.ChangeEventSourceContext context,
               String debugMessage)

If the commit executor queue is full, back-pressure will be applied by letting execution of the callback be performed by the calling thread.

Parameters:

transactionId - transaction identifier

scn - SCN of the commit.

offsetContext - Oracle offset

timestamp - commit timestamp

context - context to check that source is running

debugMessage - message

Returns:

true if committed transaction is in the buffer, was not processed yet and processed now

rollback

boolean rollback(String transactionId,
                 String debugMessage)

Clears registered callbacks for given transaction identifier.

Parameters:

transactionId - transaction id

debugMessage - message

Returns:

true if the rollback is for a transaction in the buffer

abandonLongTransactions

void abandonLongTransactions(Long thresholdScn)

If for some reason the connector got restarted, the offset will point to the beginning of the oldest captured transaction. Taking in consideration offset flush interval, the offset could be even older. If that transaction was lasted for a long time, let say > 30 minutes, the offset will be not accessible after restart, because we don't mine archived logs, neither rely on continuous_mine configuration option. Hence we have to address these cases manually.

It is limited by following condition: allOnlineRedoLogFiles.size() - currentlyMinedLogFiles.size() <= 1

If each redo lasts for 10 minutes and 7 redo group have been configured, any transaction cannot lasts longer than 1 hour.

In case of an abandonment, all DMLs/Commits/Rollbacs for this transaction will be ignored

In other words connector will not send any part of this transaction to Kafka

Parameters:

thresholdScn - the smallest SVN of any transaction to keep in the buffer. All others will be removed.

calculateSmallestScn

private BigDecimal calculateSmallestScn()

calculateLargestScn

private void calculateLargestScn()

isEmpty

boolean isEmpty()

Returns true if buffer is empty, otherwise false.

Returns:

true if buffer is empty, otherwise false

toString

public String toString()

Overrides:

toString in class Object

close

void close()

Closes buffer.