Package io.debezium.connector.mongodb.transforms

Class ExtractNewDocumentState<R extends org.apache.kafka.connect.connector.ConnectRecord<R>>

java.lang.Object

io.debezium.connector.mongodb.transforms.ExtractNewDocumentState<R>

Type Parameters:

R - the subtype of ConnectRecord on which this transformation will operate

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.kafka.common.Configurable, org.apache.kafka.connect.transforms.Transformation<R>

public class ExtractNewDocumentState<R extends org.apache.kafka.connect.connector.ConnectRecord<R>>
extends Object
implements org.apache.kafka.connect.transforms.Transformation<R>

Debezium Mongo Connector generates the CDC records in String format. Sink connectors usually are not able to parse the string and insert the document as it is represented in the Source. so a user use this SMT to parse the String and insert the MongoDB document in the JSON format.

Author:

Sairam Polavarapu, Renato mefi

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static enum	ExtractNewDocumentState.ArrayEncoding
private static class	ExtractNewDocumentState.FieldReference	Represents a field that should be added to the outgoing record as a header attribute or struct field.

Field Summary

Fields

Modifier and Type	Field	Description
private String	addFieldsPrefix
private List<ExtractNewDocumentState.FieldReference>	additionalFields
private List<ExtractNewDocumentState.FieldReference>	additionalHeaders
private final org.apache.kafka.connect.transforms.ExtractField<R>	afterExtractor
private static final Field	ARRAY_ENCODING
private MongoDataConverter	converter
private String	delimiter
private static final Field	DELIMITER
private boolean	dropTombstones
private static final Pattern	FIELD_SEPARATOR
private static final Field	FLATTEN_STRUCT
private boolean	flattenStruct
private ExtractNewRecordStateConfigDefinition.DeleteHandling	handleDeletes
private final org.apache.kafka.connect.transforms.ExtractField<R>	keyExtractor
private static final org.slf4j.Logger	LOGGER
private final org.apache.kafka.connect.transforms.ExtractField<R>	patchExtractor
private final org.apache.kafka.connect.transforms.Flatten<R>	recordFlattener
static final Field	SANITIZE_FIELD_NAMES
private SmtManager<R>	smtManager

Constructor Summary

Constructors

Constructor	Description
ExtractNewDocumentState()

Method Summary

Modifier and Type	Method	Description
private void	addAdditionalFieldsSchema(List<ExtractNewDocumentState.FieldReference> additionalFields, R originalRecord, org.apache.kafka.connect.data.SchemaBuilder valueSchemaBuilder)
private void	addFields(List<ExtractNewDocumentState.FieldReference> additionalFields, R originalRecord, org.apache.kafka.connect.data.Struct value)
R	apply(R record)
void	close()
org.apache.kafka.common.config.ConfigDef	config()
void	configure(Map<String,?> map)
private static List<String>	determineAdditionalSourceField(String addSourceFieldsConfig)
private org.bson.BsonDocument	getInsertDocument(R record, org.bson.BsonDocument key)
private org.bson.BsonDocument	getUpdateDocument(R patchRecord, org.bson.BsonDocument keyDocument)
private org.apache.kafka.connect.header.Headers	makeHeaders(List<ExtractNewDocumentState.FieldReference> additionalHeaders, org.apache.kafka.connect.data.Struct originalRecordValue)
private R	newRecord(R record, org.bson.BsonDocument keyDocument, org.bson.BsonDocument valueDocument)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

addFieldsPrefix

private String addFieldsPrefix

LOGGER

private static final org.slf4j.Logger LOGGER

FIELD_SEPARATOR

private static final Pattern FIELD_SEPARATOR

ARRAY_ENCODING

private static final Field ARRAY_ENCODING

FLATTEN_STRUCT

private static final Field FLATTEN_STRUCT

DELIMITER

private static final Field DELIMITER

SANITIZE_FIELD_NAMES

public static final Field SANITIZE_FIELD_NAMES

afterExtractor

private final org.apache.kafka.connect.transforms.ExtractField<R extends org.apache.kafka.connect.connector.ConnectRecord<R>> afterExtractor

patchExtractor

private final org.apache.kafka.connect.transforms.ExtractField<R extends org.apache.kafka.connect.connector.ConnectRecord<R>> patchExtractor

keyExtractor

private final org.apache.kafka.connect.transforms.ExtractField<R extends org.apache.kafka.connect.connector.ConnectRecord<R>> keyExtractor

converter

private MongoDataConverter converter

recordFlattener

private final org.apache.kafka.connect.transforms.Flatten<R extends org.apache.kafka.connect.connector.ConnectRecord<R>> recordFlattener

additionalHeaders

private List<ExtractNewDocumentState.FieldReference> additionalHeaders

additionalFields

private List<ExtractNewDocumentState.FieldReference> additionalFields

flattenStruct

private boolean flattenStruct

delimiter

private String delimiter

dropTombstones

private boolean dropTombstones

handleDeletes

private ExtractNewRecordStateConfigDefinition.DeleteHandling handleDeletes

smtManager

private SmtManager<R extends org.apache.kafka.connect.connector.ConnectRecord<R>> smtManager

Constructor Details

ExtractNewDocumentState

public ExtractNewDocumentState()

Method Details

apply

public R apply(R record)

Specified by:

apply in interface org.apache.kafka.connect.transforms.Transformation<R extends org.apache.kafka.connect.connector.ConnectRecord<R>>

newRecord

private R newRecord(R record,
 org.bson.BsonDocument keyDocument,
 org.bson.BsonDocument valueDocument)

addAdditionalFieldsSchema

private void addAdditionalFieldsSchema(List<ExtractNewDocumentState.FieldReference> additionalFields,
 R originalRecord,
 org.apache.kafka.connect.data.SchemaBuilder valueSchemaBuilder)

addFields

private void addFields(List<ExtractNewDocumentState.FieldReference> additionalFields,
 R originalRecord,
 org.apache.kafka.connect.data.Struct value)

getUpdateDocument

private org.bson.BsonDocument getUpdateDocument(R patchRecord,
 org.bson.BsonDocument keyDocument)

getInsertDocument

private org.bson.BsonDocument getInsertDocument(R record,
 org.bson.BsonDocument key)

makeHeaders

private org.apache.kafka.connect.header.Headers makeHeaders(List<ExtractNewDocumentState.FieldReference> additionalHeaders,
 org.apache.kafka.connect.data.Struct originalRecordValue)

config

public org.apache.kafka.common.config.ConfigDef config()

Specified by:

config in interface org.apache.kafka.connect.transforms.Transformation<R extends org.apache.kafka.connect.connector.ConnectRecord<R>>

close

public void close()

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Specified by:

close in interface org.apache.kafka.connect.transforms.Transformation<R extends org.apache.kafka.connect.connector.ConnectRecord<R>>

configure

public void configure(Map<String,?> map)

Specified by:

configure in interface org.apache.kafka.common.Configurable

determineAdditionalSourceField

private static List<String> determineAdditionalSourceField(String addSourceFieldsConfig)