io.debezium.pipeline

Class EventDispatcher<T extends DataCollectionId>

java.lang.Object

io.debezium.pipeline.EventDispatcher<T>

public class EventDispatcher<T extends DataCollectionId>
extends Object

Central dispatcher for data change and schema change events. The former will be routed to the change event queue, the latter will be routed to the DatabaseSchema. But based on the applying whitelist/blacklist configuration, events may be not be dispatched at all.

This router is also in charge of emitting heartbeat messages, exposing of metrics via JMX etc.

Author:

Gunnar Morling

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
private class	EventDispatcher.BufferingSnapshotChangeRecordReceiver
static interface	EventDispatcher.InconsistentSchemaHandler<T extends DataCollectionId> Reaction to an incoming change event for which schema is not found
private class	EventDispatcher.SchemaChangeEventReceiver
static interface	EventDispatcher.SnapshotReceiver Change record receiver used during snapshotting.
private class	EventDispatcher.StreamingChangeRecordReceiver

Field Summary

Fields

Modifier and Type	Field and Description
private ChangeEventCreator	changeEventCreator
private CommonConnectorConfig	connectorConfig
private boolean	emitTombstonesOnDelete
private DataChangeEventListener	eventListener
private DataCollectionFilters.DataCollectionFilter<T>	filter
private Heartbeat	heartbeat
private HistorizedDatabaseSchema<T>	historizedSchema
private EventDispatcher.InconsistentSchemaHandler<T>	inconsistentSchemaHandler
private static org.slf4j.Logger	LOGGER
private ChangeEventQueue<DataChangeEvent>	queue
private DatabaseSchema<T>	schema
private EventDispatcher.StreamingChangeRecordReceiver	streamingReceiver Change event receiver for events dispatched from a streaming change event source.
private TopicSelector<T>	topicSelector

Constructor Summary

Constructors

Constructor and Description
EventDispatcher(CommonConnectorConfig connectorConfig, TopicSelector<T> topicSelector, DatabaseSchema<T> schema, ChangeEventQueue<DataChangeEvent> queue, DataCollectionFilters.DataCollectionFilter<T> filter, ChangeEventCreator changeEventCreator)
EventDispatcher(CommonConnectorConfig connectorConfig, TopicSelector<T> topicSelector, DatabaseSchema<T> schema, ChangeEventQueue<DataChangeEvent> queue, DataCollectionFilters.DataCollectionFilter<T> filter, ChangeEventCreator changeEventCreator, EventDispatcher.InconsistentSchemaHandler<T> inconsistentSchemaHandler)

Method Summary

Modifier and Type	Method and Description
void	alwaysDispatchHeartbeatEvent(OffsetContext offset)
boolean	dispatchDataChangeEvent(T dataCollectionId, ChangeRecordEmitter changeRecordEmitter) Dispatches one or more DataChangeEvents.
void	dispatchHeartbeatEvent(OffsetContext offset)
void	dispatchSchemaChangeEvent(T dataCollectionId, SchemaChangeEventEmitter schemaChangeEventEmitter)
void	dispatchSnapshotEvent(T dataCollectionId, ChangeRecordEmitter changeRecordEmitter, EventDispatcher.SnapshotReceiver receiver)
private void	enqueueHeartbeat(org.apache.kafka.connect.source.SourceRecord record)
Optional<DataCollectionSchema>	errorOnMissingSchema(T dataCollectionId, ChangeRecordEmitter changeRecordEmitter)
EventDispatcher.SnapshotReceiver	getSnapshotChangeEventReceiver()
boolean	heartbeatsEnabled()
Optional<DataCollectionSchema>	ignoreMissingSchema(T dataCollectionId, ChangeRecordEmitter changeRecordEmitter)
void	setEventListener(DataChangeEventListener eventListener) Provide a listener that is invoked for every incoming event to be processed.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOGGER

private static final org.slf4j.Logger LOGGER

topicSelector

private final TopicSelector<T extends DataCollectionId> topicSelector

schema

private final DatabaseSchema<T extends DataCollectionId> schema

historizedSchema

private final HistorizedDatabaseSchema<T extends DataCollectionId> historizedSchema

queue

private final ChangeEventQueue<DataChangeEvent> queue

filter

private final DataCollectionFilters.DataCollectionFilter<T extends DataCollectionId> filter

changeEventCreator

private final ChangeEventCreator changeEventCreator

heartbeat

private final Heartbeat heartbeat

eventListener

private DataChangeEventListener eventListener

emitTombstonesOnDelete

private final boolean emitTombstonesOnDelete

inconsistentSchemaHandler

private final EventDispatcher.InconsistentSchemaHandler<T extends DataCollectionId> inconsistentSchemaHandler

connectorConfig

private final CommonConnectorConfig connectorConfig

streamingReceiver

private final EventDispatcher.StreamingChangeRecordReceiver streamingReceiver

Change event receiver for events dispatched from a streaming change event source.

Constructor Detail

EventDispatcher

public EventDispatcher(CommonConnectorConfig connectorConfig,
                       TopicSelector<T> topicSelector,
                       DatabaseSchema<T> schema,
                       ChangeEventQueue<DataChangeEvent> queue,
                       DataCollectionFilters.DataCollectionFilter<T> filter,
                       ChangeEventCreator changeEventCreator)

EventDispatcher

public EventDispatcher(CommonConnectorConfig connectorConfig,
                       TopicSelector<T> topicSelector,
                       DatabaseSchema<T> schema,
                       ChangeEventQueue<DataChangeEvent> queue,
                       DataCollectionFilters.DataCollectionFilter<T> filter,
                       ChangeEventCreator changeEventCreator,
                       EventDispatcher.InconsistentSchemaHandler<T> inconsistentSchemaHandler)

Method Detail

dispatchSnapshotEvent

public void dispatchSnapshotEvent(T dataCollectionId,
                                  ChangeRecordEmitter changeRecordEmitter,
                                  EventDispatcher.SnapshotReceiver receiver)
                           throws InterruptedException

Throws:

InterruptedException

getSnapshotChangeEventReceiver

public EventDispatcher.SnapshotReceiver getSnapshotChangeEventReceiver()

dispatchDataChangeEvent

public boolean dispatchDataChangeEvent(T dataCollectionId,
                                       ChangeRecordEmitter changeRecordEmitter)
                                throws InterruptedException

Dispatches one or more DataChangeEvents. If the given data collection is included in the currently captured set of collections, the given emitter will be invoked, so it can emit one or more events (in the common case, one event will be emitted, but e.g. in case of PK updates, it may be a deletion and a creation event). The receiving coordinator creates SourceRecords for all emitted events and passes them to this dispatcher's ChangeEventCreator for converting them into data change events.

Returns:

true if an event was dispatched (i.e. sent to the message broker), false otherwise.

Throws:

InterruptedException

errorOnMissingSchema

public Optional<DataCollectionSchema> errorOnMissingSchema(T dataCollectionId,
                                                           ChangeRecordEmitter changeRecordEmitter)

ignoreMissingSchema

public Optional<DataCollectionSchema> ignoreMissingSchema(T dataCollectionId,
                                                          ChangeRecordEmitter changeRecordEmitter)

dispatchSchemaChangeEvent

public void dispatchSchemaChangeEvent(T dataCollectionId,
                                      SchemaChangeEventEmitter schemaChangeEventEmitter)
                               throws InterruptedException

Throws:

InterruptedException

alwaysDispatchHeartbeatEvent

public void alwaysDispatchHeartbeatEvent(OffsetContext offset)
                                  throws InterruptedException

Throws:

InterruptedException

dispatchHeartbeatEvent

public void dispatchHeartbeatEvent(OffsetContext offset)
                            throws InterruptedException

Throws:

InterruptedException

heartbeatsEnabled

public boolean heartbeatsEnabled()

enqueueHeartbeat

private void enqueueHeartbeat(org.apache.kafka.connect.source.SourceRecord record)
                       throws InterruptedException

Throws:

InterruptedException

setEventListener

public void setEventListener(DataChangeEventListener eventListener)

Provide a listener that is invoked for every incoming event to be processed.

Parameters:

eventListener -