Package io.debezium.connector.spanner

Class SpannerStreamingChangeEventSource

java.lang.Object

io.debezium.connector.spanner.SpannerStreamingChangeEventSource

All Implemented Interfaces:

CommittingRecordsStreamingChangeEventSource<SpannerPartition,SpannerOffsetContext>, io.debezium.pipeline.source.spi.ChangeEventSource, io.debezium.pipeline.source.spi.StreamingChangeEventSource<SpannerPartition,SpannerOffsetContext>

public class SpannerStreamingChangeEventSource
extends Object
implements CommittingRecordsStreamingChangeEventSource<SpannerPartition,SpannerOffsetContext>

Processes all types of Spanner Stream Events

Nested Class Summary

Nested classes/interfaces inherited from interface io.debezium.pipeline.source.spi.ChangeEventSource

io.debezium.pipeline.source.spi.ChangeEventSource.ChangeEventSourceContext

Field Summary

Fields

Modifier and Type	Field	Description
private final SpannerConnectorConfig	connectorConfig
private final io.debezium.pipeline.ErrorHandler	errorHandler
private final StreamEventQueue	eventQueue
private static final Duration	FINISHING_PARTITION_TIMEOUT
private final FinishingPartitionManager	finishingPartitionManager
private final FinishPartitionStrategy	finishPartitionStrategy
private final FinishPartitionWatchDog	finishPartitionWatchDog
private static final org.slf4j.Logger	LOGGER
private final MetricsEventPublisher	metricsEventPublisher
private final SpannerOffsetContextFactory	offsetContextFactory
private final PartitionManager	partitionManager
private final SchemaRegistry	schemaRegistry
private final SpannerEventDispatcher	spannerEventDispatcher
private final ChangeStream	stream
private static final StuckPartitionStrategy	STUCK_PARTITION_STRATEGY
private Thread	thread

Constructor Summary

Constructors

Constructor

Description

SpannerStreamingChangeEventSource(SpannerConnectorConfig connectorConfig, io.debezium.pipeline.ErrorHandler errorHandler, ChangeStream stream, StreamEventQueue eventQueue, MetricsEventPublisher metricsEventPublisher, PartitionManager partitionManager, SchemaRegistry schemaRegistry, SpannerEventDispatcher spannerEventDispatcher, boolean finishingAfterCommit, SpannerOffsetContextFactory offsetContextFactory)

Method Summary

Modifier and Type	Method	Description
void	commitRecords(List<org.apache.kafka.connect.source.SourceRecord> records)
void	execute(io.debezium.pipeline.source.spi.ChangeEventSource.ChangeEventSourceContext context, SpannerPartition partition, SpannerOffsetContext offsetContext)
private void	processChildPartitionsEvent(ChildPartitionsEvent event)
(package private) void	processDataChangeEvent(DataChangeEvent event)
private void	processFailure(Exception ex)
private void	processHeartBeatEvent(HeartbeatEvent event)
private void	startProcessing(io.debezium.pipeline.source.spi.ChangeEventSource.ChangeEventSourceContext context)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface io.debezium.pipeline.source.spi.StreamingChangeEventSource

commitOffset, executeIteration, init

Field Details

LOGGER

private static final org.slf4j.Logger LOGGER

STUCK_PARTITION_STRATEGY

private static final StuckPartitionStrategy STUCK_PARTITION_STRATEGY

FINISHING_PARTITION_TIMEOUT

private static final Duration FINISHING_PARTITION_TIMEOUT

finishPartitionStrategy

private final FinishPartitionStrategy finishPartitionStrategy

errorHandler

private final io.debezium.pipeline.ErrorHandler errorHandler

eventQueue

private final StreamEventQueue eventQueue

metricsEventPublisher

private final MetricsEventPublisher metricsEventPublisher

stream

private final ChangeStream stream

partitionManager

private final PartitionManager partitionManager

schemaRegistry

private final SchemaRegistry schemaRegistry

spannerEventDispatcher

private final SpannerEventDispatcher spannerEventDispatcher

finishingPartitionManager

private final FinishingPartitionManager finishingPartitionManager

finishPartitionWatchDog

private final FinishPartitionWatchDog finishPartitionWatchDog

offsetContextFactory

private final SpannerOffsetContextFactory offsetContextFactory

connectorConfig

private final SpannerConnectorConfig connectorConfig

thread

private volatile Thread thread

Constructor Details

SpannerStreamingChangeEventSource

public SpannerStreamingChangeEventSource(SpannerConnectorConfig connectorConfig,
 io.debezium.pipeline.ErrorHandler errorHandler,
 ChangeStream stream,
 StreamEventQueue eventQueue,
 MetricsEventPublisher metricsEventPublisher,
 PartitionManager partitionManager,
 SchemaRegistry schemaRegistry,
 SpannerEventDispatcher spannerEventDispatcher,
 boolean finishingAfterCommit,
 SpannerOffsetContextFactory offsetContextFactory)

Method Details

execute

public void execute(io.debezium.pipeline.source.spi.ChangeEventSource.ChangeEventSourceContext context,
 SpannerPartition partition,
 SpannerOffsetContext offsetContext)
             throws InterruptedException

Specified by:

execute in interface io.debezium.pipeline.source.spi.StreamingChangeEventSource<SpannerPartition,SpannerOffsetContext>

Throws:

InterruptedException

startProcessing

private void startProcessing(io.debezium.pipeline.source.spi.ChangeEventSource.ChangeEventSourceContext context)

processDataChangeEvent

void processDataChangeEvent(DataChangeEvent event)
                     throws InterruptedException

Throws:

InterruptedException

processHeartBeatEvent

private void processHeartBeatEvent(HeartbeatEvent event)
                            throws InterruptedException

Throws:

InterruptedException

processChildPartitionsEvent

private void processChildPartitionsEvent(ChildPartitionsEvent event)
                                  throws InterruptedException

Throws:

InterruptedException

processFailure

private void processFailure(Exception ex)

commitRecords

public void commitRecords(List<org.apache.kafka.connect.source.SourceRecord> records)
                   throws InterruptedException

Specified by:

commitRecords in interface CommittingRecordsStreamingChangeEventSource<SpannerPartition,SpannerOffsetContext>

Throws:

InterruptedException