Package io.debezium.connector.cassandra

Class KafkaRecordEmitter

java.lang.Object

io.debezium.connector.cassandra.KafkaRecordEmitter

All Implemented Interfaces:

Emitter, AutoCloseable

public class KafkaRecordEmitter
extends Object
implements Emitter

This emitter is responsible for emitting records to Kafka broker and managing offsets post send.

Field Summary

Fields

Modifier and Type	Field	Description
private final CommitLogTransfer	commitLogTransfer
private long	emitCount
private final Set<String>	erroneousCommitLogs
private final Map<Record,Future<org.apache.kafka.clients.producer.RecordMetadata>>	futures
private final org.apache.kafka.connect.storage.Converter	keyConverter
private final Object	lock
private static final org.slf4j.Logger	LOGGER
private final OffsetFlushPolicy	offsetFlushPolicy
private final OffsetWriter	offsetWriter
private final org.apache.kafka.clients.producer.KafkaProducer<byte[],byte[]>	producer
private long	timeOfLastFlush
private final io.debezium.schema.TopicSelector<KeyspaceTable>	topicSelector
private final org.apache.kafka.connect.storage.Converter	valueConverter

Constructor Summary

Constructors

Constructor

Description

KafkaRecordEmitter(String kafkaTopicPrefix, String heartbeatPrefix, org.apache.kafka.clients.producer.KafkaProducer kafkaProducer, OffsetWriter offsetWriter, Duration offsetFlushIntervalMs, long maxOffsetFlushSize, org.apache.kafka.connect.storage.Converter keyConverter, org.apache.kafka.connect.storage.Converter valueConverter, Set<String> erroneousCommitLogs, CommitLogTransfer commitLogTransfer)

Method Summary

Modifier and Type	Method	Description
void	close()
void	emit(Record record)
private boolean	flush(Map.Entry<Record,Future<org.apache.kafka.clients.producer.RecordMetadata>> recordEntry)
private void	flushAndMarkOffset()
private boolean	hasOffset(Map.Entry<Record,Future<org.apache.kafka.clients.producer.RecordMetadata>> recordEntry)
private void	markOffset(Map.Entry<Record,Future<org.apache.kafka.clients.producer.RecordMetadata>> recordEntry)
private void	maybeFlushAndMarkOffset()
protected org.apache.kafka.clients.producer.ProducerRecord<byte[],byte[]>	toProducerRecord(Record record)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

LOGGER

private static final org.slf4j.Logger LOGGER

producer

private final org.apache.kafka.clients.producer.KafkaProducer<byte[],byte[]> producer

topicSelector

private final io.debezium.schema.TopicSelector<KeyspaceTable> topicSelector

offsetWriter

private final OffsetWriter offsetWriter

offsetFlushPolicy

private final OffsetFlushPolicy offsetFlushPolicy

erroneousCommitLogs

private final Set<String> erroneousCommitLogs

commitLogTransfer

private final CommitLogTransfer commitLogTransfer

futures

private final Map<Record,Future<org.apache.kafka.clients.producer.RecordMetadata>> futures

lock

private final Object lock

keyConverter

private final org.apache.kafka.connect.storage.Converter keyConverter

valueConverter

private final org.apache.kafka.connect.storage.Converter valueConverter

timeOfLastFlush

private long timeOfLastFlush

emitCount

private long emitCount

Constructor Details

KafkaRecordEmitter

public KafkaRecordEmitter(String kafkaTopicPrefix,
 String heartbeatPrefix,
 org.apache.kafka.clients.producer.KafkaProducer kafkaProducer,
 OffsetWriter offsetWriter,
 Duration offsetFlushIntervalMs,
 long maxOffsetFlushSize,
 org.apache.kafka.connect.storage.Converter keyConverter,
 org.apache.kafka.connect.storage.Converter valueConverter,
 Set<String> erroneousCommitLogs,
 CommitLogTransfer commitLogTransfer)

Method Details

emit

public void emit(Record record)

Specified by:

emit in interface Emitter

toProducerRecord

protected org.apache.kafka.clients.producer.ProducerRecord<byte[],byte[]> toProducerRecord(Record record)

maybeFlushAndMarkOffset

private void maybeFlushAndMarkOffset()

flushAndMarkOffset

private void flushAndMarkOffset()

flush

private boolean flush(Map.Entry<Record,Future<org.apache.kafka.clients.producer.RecordMetadata>> recordEntry)

hasOffset

private boolean hasOffset(Map.Entry<Record,Future<org.apache.kafka.clients.producer.RecordMetadata>> recordEntry)

markOffset

private void markOffset(Map.Entry<Record,Future<org.apache.kafka.clients.producer.RecordMetadata>> recordEntry)

close

public void close()
           throws Exception

Specified by:

close in interface AutoCloseable

Throws:

Exception