io.debezium.transforms

Class ExtractNewRecordState<R extends org.apache.kafka.connect.connector.ConnectRecord<R>>

java.lang.Object

io.debezium.transforms.ExtractNewRecordState<R>

Type Parameters:

R - the subtype of ConnectRecord on which this transformation will operate

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.kafka.common.Configurable, org.apache.kafka.connect.transforms.Transformation<R>

Direct Known Subclasses:

UnwrapFromEnvelope

public class ExtractNewRecordState<R extends org.apache.kafka.connect.connector.ConnectRecord<R>>
extends Object
implements org.apache.kafka.connect.transforms.Transformation<R>

Debezium generates CDC (Envelope) records that are struct of values containing values before and after change. Sink connectors usually are not able to work with a complex structure so a user use this SMT to extract after value and send it down unwrapped in Envelope.

The functionality is similar to ExtractField SMT but has a special semantics for handling delete events; when delete event is emitted by database then Debezium emits two messages: a delete message and a tombstone message that serves as a signal to Kafka compaction process.

The SMT by default drops the tombstone message created by Debezium and converts the delete message into a tombstone message that can be dropped, too, if required. *

The SMT also has the option to insert fields from the original record's 'source' struct into the new unwrapped record prefixed with "__" (for example __lsn in Postgres, or __file in MySQL)

Author:

Jiri Pechanec

Field Summary

Fields

Modifier and Type	Field and Description
private boolean	addOperationHeader
private String[]	addSourceFields
private org.apache.kafka.connect.transforms.ExtractField<R>	afterDelegate
private org.apache.kafka.connect.transforms.ExtractField<R>	beforeDelegate
private boolean	dropTombstones
private ExtractNewRecordStateConfigDefinition.DeleteHandling	handleDeletes
private static org.slf4j.Logger	LOGGER
private static String	PURPOSE
private org.apache.kafka.connect.transforms.InsertField<R>	removedDelegate
private static int	SCHEMA_CACHE_SIZE
private BoundedConcurrentHashMap<org.apache.kafka.connect.data.Schema,org.apache.kafka.connect.data.Schema>	schemaUpdateCache
private SmtManager<R>	smtManager
private org.apache.kafka.connect.transforms.InsertField<R>	updatedDelegate

Constructor Summary

Constructors

Constructor and Description
ExtractNewRecordState()

Method Summary

Modifier and Type	Method and Description
private R	addSourceFields(String[] addSourceFields, R originalRecord, R unwrappedRecord)
R	apply(R record)
void	close()
org.apache.kafka.common.config.ConfigDef	config()
void	configure(Map<String,?> configs)
private org.apache.kafka.connect.data.Schema	makeUpdatedSchema(org.apache.kafka.connect.data.Schema schema, org.apache.kafka.connect.data.Schema sourceSchema, String[] addSourceFields)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

PURPOSE

private static final String PURPOSE

See Also:

Constant Field Values

SCHEMA_CACHE_SIZE

private static final int SCHEMA_CACHE_SIZE

See Also:

Constant Field Values

LOGGER

private static final org.slf4j.Logger LOGGER

dropTombstones

private boolean dropTombstones

handleDeletes

private ExtractNewRecordStateConfigDefinition.DeleteHandling handleDeletes

addOperationHeader

private boolean addOperationHeader

addSourceFields

private String[] addSourceFields

afterDelegate

private final org.apache.kafka.connect.transforms.ExtractField<R extends org.apache.kafka.connect.connector.ConnectRecord<R>> afterDelegate

beforeDelegate

private final org.apache.kafka.connect.transforms.ExtractField<R extends org.apache.kafka.connect.connector.ConnectRecord<R>> beforeDelegate

removedDelegate

private final org.apache.kafka.connect.transforms.InsertField<R extends org.apache.kafka.connect.connector.ConnectRecord<R>> removedDelegate

updatedDelegate

private final org.apache.kafka.connect.transforms.InsertField<R extends org.apache.kafka.connect.connector.ConnectRecord<R>> updatedDelegate

schemaUpdateCache

private BoundedConcurrentHashMap<org.apache.kafka.connect.data.Schema,org.apache.kafka.connect.data.Schema> schemaUpdateCache

smtManager

private SmtManager<R extends org.apache.kafka.connect.connector.ConnectRecord<R>> smtManager

Constructor Detail

ExtractNewRecordState

public ExtractNewRecordState()

Method Detail

configure

public void configure(Map<String,?> configs)

Specified by:

configure in interface org.apache.kafka.common.Configurable

apply

public R apply(R record)

Specified by:

apply in interface org.apache.kafka.connect.transforms.Transformation<R extends org.apache.kafka.connect.connector.ConnectRecord<R>>

addSourceFields

private R addSourceFields(String[] addSourceFields,
                          R originalRecord,
                          R unwrappedRecord)

makeUpdatedSchema

private org.apache.kafka.connect.data.Schema makeUpdatedSchema(org.apache.kafka.connect.data.Schema schema,
                                                               org.apache.kafka.connect.data.Schema sourceSchema,
                                                               String[] addSourceFields)

config

public org.apache.kafka.common.config.ConfigDef config()

Specified by:

config in interface org.apache.kafka.connect.transforms.Transformation<R extends org.apache.kafka.connect.connector.ConnectRecord<R>>

close

public void close()

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Specified by:

close in interface org.apache.kafka.connect.transforms.Transformation<R extends org.apache.kafka.connect.connector.ConnectRecord<R>>