Package io.debezium.connector.cassandra

Class CassandraConnectorConfig

java.lang.Object
io.debezium.config.CommonConnectorConfig
io.debezium.connector.cassandra.CassandraConnectorConfig
public class CassandraConnectorConfig extends io.debezium.config.CommonConnectorConfig
All configs used by a Cassandra connector agent.

  • Field Details

    • KAFKA_PRODUCER_CONFIG_PREFIX

      public static final String KAFKA_PRODUCER_CONFIG_PREFIX
      The prefix prepended to all Kafka producer configurations, including schema registry
      See Also:

    • KEY_CONVERTER_PREFIX

      public static final String KEY_CONVERTER_PREFIX
      The prefix prepended to all Kafka key converter configurations, including schema registry.
      See Also:

    • VALUE_CONVERTER_PREFIX

      public static final String VALUE_CONVERTER_PREFIX
      The prefix prepended to all Kafka value converter configurations, including schema registry.
      See Also:

    • COMMIT_LOG_TRANSFER_CONFIG_PREFIX

      public static final String COMMIT_LOG_TRANSFER_CONFIG_PREFIX
      The prefix for all CommitLogTransfer configurations.
      See Also:

    • TOPIC_PREFIX

      public static final io.debezium.config.Field TOPIC_PREFIX

    • KEY_CONVERTER_CLASS_CONFIG

      public static final io.debezium.config.Field KEY_CONVERTER_CLASS_CONFIG

    • VALUE_CONVERTER_CLASS_CONFIG

      public static final io.debezium.config.Field VALUE_CONVERTER_CLASS_CONFIG

    • DEFAULT_SNAPSHOT_MODE

      public static final String DEFAULT_SNAPSHOT_MODE
      Must be one of 'INITIAL', 'ALWAYS', or 'NEVER'. The default snapshot mode is 'INITIAL'. See for details.
      See Also:

    • SNAPSHOT_MODE

      public static final io.debezium.config.Field SNAPSHOT_MODE

    • DEFAULT_SNAPSHOT_CONSISTENCY

      public static final String DEFAULT_SNAPSHOT_CONSISTENCY
      Specify the ConsistencyLevel used for the snapshot query.
      See Also:

    • SNAPSHOT_CONSISTENCY

      public static final io.debezium.config.Field SNAPSHOT_CONSISTENCY

    • DEFAULT_HTTP_PORT

      public static final int DEFAULT_HTTP_PORT
      See Also:

    • HTTP_PORT

      public static final io.debezium.config.Field HTTP_PORT

    • CASSANDRA_CONFIG

      public static final io.debezium.config.Field CASSANDRA_CONFIG

    • COMMIT_LOG_RELOCATION_DIR

      public static final io.debezium.config.Field COMMIT_LOG_RELOCATION_DIR

    • DEFAULT_COMMIT_LOG_POST_PROCESSING_ENABLED

      public static final boolean DEFAULT_COMMIT_LOG_POST_PROCESSING_ENABLED
      If disabled, commit logs would not be deleted post-process, and this could lead to disk storage
      See Also:

    • COMMIT_LOG_POST_PROCESSING_ENABLED

      public static final io.debezium.config.Field COMMIT_LOG_POST_PROCESSING_ENABLED

    • DEFAULT_COMMIT_LOG_ERROR_REPROCESSING_ENABLED

      public static final boolean DEFAULT_COMMIT_LOG_ERROR_REPROCESSING_ENABLED
      See Also:

    • COMMIT_LOG_ERROR_REPROCESSING_ENABLED

      public static final io.debezium.config.Field COMMIT_LOG_ERROR_REPROCESSING_ENABLED

    • DEFAULT_COMMIT_LOG_REAL_TIME_PROCESSING_ENABLED

      public static final boolean DEFAULT_COMMIT_LOG_REAL_TIME_PROCESSING_ENABLED
      Only valid for Cassandra 4 and if enabled, commit logs would be read incrementally instead of reading complete log file
      See Also:

    • COMMIT_LOG_REAL_TIME_PROCESSING_ENABLED

      public static final io.debezium.config.Field COMMIT_LOG_REAL_TIME_PROCESSING_ENABLED

    • DEFAULT_COMMIT_LOG_MARKED_COMPLETE_POLL_INTERVAL_IN_MS

      public static final int DEFAULT_COMMIT_LOG_MARKED_COMPLETE_POLL_INTERVAL_IN_MS
      Only valid for Cassandra 4 and defines the polling interval to check for completeness of commit log file
      See Also:

    • COMMIT_LOG_MARKED_COMPLETE_POLL_INTERVAL_IN_MS

      public static final io.debezium.config.Field COMMIT_LOG_MARKED_COMPLETE_POLL_INTERVAL_IN_MS

    • DEFAULT_COMMIT_LOG_TRANSFER_CLASS

      public static final String DEFAULT_COMMIT_LOG_TRANSFER_CLASS
      The fully qualified CommitLogTransfer class used to transfer commit logs. The default option will delete all commit log files after processing (successful or otherwise). You can extend a custom implementation.
      See Also:

    • COMMIT_LOG_TRANSFER_CLASS

      public static final io.debezium.config.Field COMMIT_LOG_TRANSFER_CLASS

    • OFFSET_BACKING_STORE_DIR

      public static final io.debezium.config.Field OFFSET_BACKING_STORE_DIR

    • DEFAULT_OFFSET_FLUSH_INTERVAL_MS

      public static final int DEFAULT_OFFSET_FLUSH_INTERVAL_MS
      The default value of 0 implies the offset will be flushed every time.
      See Also:

    • OFFSET_FLUSH_INTERVAL_MS

      public static final io.debezium.config.Field OFFSET_FLUSH_INTERVAL_MS

    • DEFAULT_MAX_OFFSET_FLUSH_SIZE

      public static final int DEFAULT_MAX_OFFSET_FLUSH_SIZE
      This config is effective only if offset_flush_interval_ms != 0
      See Also:

    • MAX_OFFSET_FLUSH_SIZE

      public static final io.debezium.config.Field MAX_OFFSET_FLUSH_SIZE

    • DEFAULT_SCHEMA_POLL_INTERVAL_MS

      public static final int DEFAULT_SCHEMA_POLL_INTERVAL_MS
      See Also:

    • SCHEMA_POLL_INTERVAL_MS

      public static final io.debezium.config.Field SCHEMA_POLL_INTERVAL_MS

    • DEFAULT_CDC_DIR_POLL_INTERVAL_MS

      public static final int DEFAULT_CDC_DIR_POLL_INTERVAL_MS
      See Also:

    • CDC_DIR_POLL_INTERVAL_MS

      public static final io.debezium.config.Field CDC_DIR_POLL_INTERVAL_MS

    • DEFAULT_SNAPSHOT_POLL_INTERVAL_MS

      public static final int DEFAULT_SNAPSHOT_POLL_INTERVAL_MS
      See Also:

    • SNAPSHOT_POLL_INTERVAL_MS

      public static final io.debezium.config.Field SNAPSHOT_POLL_INTERVAL_MS

    • DEFAULT_COMMIT_LOG_RELOCATION_DIR_POLL_INTERVAL_MS

      public static final int DEFAULT_COMMIT_LOG_RELOCATION_DIR_POLL_INTERVAL_MS
      See Also:

    • COMMIT_LOG_RELOCATION_DIR_POLL_INTERVAL_MS

      public static final io.debezium.config.Field COMMIT_LOG_RELOCATION_DIR_POLL_INTERVAL_MS

    • DEFAULT_NUM_OF_CHANGE_EVENT_QUEUES

      public static final int DEFAULT_NUM_OF_CHANGE_EVENT_QUEUES
      See Also:

    • NUM_OF_CHANGE_EVENT_QUEUES

      public static final io.debezium.config.Field NUM_OF_CHANGE_EVENT_QUEUES

    • FIELD_EXCLUDE_LIST

      public static final io.debezium.config.Field FIELD_EXCLUDE_LIST
      A comma-separated list of fully-qualified names of fields that should be excluded from change event message values. Fully-qualified names for fields are in the form <keyspace_name>.<field_name>.<nested_field_name>.

    • CASSANDRA_DRIVER_CONFIG_FILE

      public static final io.debezium.config.Field CASSANDRA_DRIVER_CONFIG_FILE

    • DEFAULT_LATEST_COMMIT_LOG_ONLY

      public static final boolean DEFAULT_LATEST_COMMIT_LOG_ONLY
      Instead of parsing commit logs from CDC directory, this will look for the commit log with the latest modified timestamp in the commit log directory and attempt to process this file only. Only used for Testing!
      See Also:

    • LATEST_COMMIT_LOG_ONLY

      public static final io.debezium.config.Field LATEST_COMMIT_LOG_ONLY

    • DEFAULT_POLL_INTERVAL_MS

      public static final int DEFAULT_POLL_INTERVAL_MS
      See Also:

    • DEFAULT_TOMBSTONES_ON_DELETE

      public static final boolean DEFAULT_TOMBSTONES_ON_DELETE
      See Also:

    • DEFAULT_SNAPSHOT_FETCH_SIZE

      protected static final int DEFAULT_SNAPSHOT_FETCH_SIZE
      See Also:

    • DECIMAL_HANDLING_MODE

      public static final io.debezium.config.Field DECIMAL_HANDLING_MODE
      Must be one of 'DOUBLE', 'PRECISE', or 'STRING'. The default decimal handling mode is 'DOUBLE'. See for details.

    • VARINT_HANDLING_MODE

      public static final io.debezium.config.Field VARINT_HANDLING_MODE
      Must be one of 'LONG', 'PRECISE', or 'STRING'. The default varint handling mode is 'LONG'. See for details.

    • CASSANDRA_NODE_ID

      static final io.debezium.config.Field CASSANDRA_NODE_ID

    • SOURCE_INFO_STRUCT_MAKER

      public static final io.debezium.config.Field SOURCE_INFO_STRUCT_MAKER

    • validationFieldList

      private static List<io.debezium.config.Field> validationFieldList

  • Constructor Details

    • CassandraConnectorConfig

      public CassandraConnectorConfig(io.debezium.config.Configuration config)

  • Method Details

    • isCommitLogRealTimeProcessingEnabled

      public boolean isCommitLogRealTimeProcessingEnabled()

    • getCommitLogMarkedCompletePollInterval

      public int getCommitLogMarkedCompletePollInterval()

    • getKafkaConfigs

      public Properties getKafkaConfigs()

    • commitLogTransferConfigs

      public Properties commitLogTransferConfigs()

    • latestCommitLogOnly

      public boolean latestCommitLogOnly()

    • snapshotMode

      public CassandraConnectorConfig.SnapshotMode snapshotMode()

    • snapshotConsistencyLevel

      public com.datastax.oss.driver.api.core.ConsistencyLevel snapshotConsistencyLevel()

    • httpPort

      public int httpPort()

    • cassandraConfig

      public String cassandraConfig()

    • commitLogRelocationDir

      public String commitLogRelocationDir()

    • postProcessEnabled

      public boolean postProcessEnabled()

    • errorCommitLogReprocessEnabled

      public boolean errorCommitLogReprocessEnabled()

    • getCommitLogTransfer

      public CommitLogTransfer getCommitLogTransfer()

    • offsetBackingStoreDir

      public String offsetBackingStoreDir()

    • offsetFlushIntervalMs

      public Duration offsetFlushIntervalMs()

    • maxOffsetFlushSize

      public long maxOffsetFlushSize()

    • maxQueueSize

      public int maxQueueSize()

    • maxBatchSize

      public int maxBatchSize()

    • cassandraDriverConfig

      public String cassandraDriverConfig()

    • pollInterval

      public Duration pollInterval()
      Positive integer value that specifies the number of milliseconds the commit log processor should wait during each iteration for new change events to appear in the queue. Defaults to 1000 milliseconds, or 1 second.

    • schemaPollInterval

      public Duration schemaPollInterval()

    • cdcDirPollInterval

      public Duration cdcDirPollInterval()

    • snapshotPollInterval

      public Duration snapshotPollInterval()

    • commitLogRelocationDirPollInterval

      public Duration commitLogRelocationDirPollInterval()

    • numOfChangeEventQueues

      public int numOfChangeEventQueues()

    • fieldExcludeList

      public List<String> fieldExcludeList()

    • tombstonesOnDelete

      public boolean tombstonesOnDelete()
      Whether deletion events should have a subsequent tombstone event (true) or not (false). It's important to note that in Cassandra, two events with the same key may be updating different columns of a given table. So this could potentially result in records being lost during compaction if they haven't been consumed by the consumer yet. In other words, do NOT set this to true if you have kafka compaction turned on.

    • getDecimalMode

      public CassandraTypeDeserializer.DecimalMode getDecimalMode()

    • getVarIntMode

      public CassandraTypeDeserializer.VarIntMode getVarIntMode()

    • getKeyConverter

      public org.apache.kafka.connect.storage.Converter getKeyConverter() throws CassandraConnectorConfigException
      Throws:
      CassandraConnectorConfigException

    • getValueConverter

      public org.apache.kafka.connect.storage.Converter getValueConverter() throws CassandraConnectorConfigException
      Throws:
      CassandraConnectorConfigException

    • keyValueConverterConfigs

      private Map<String,Object> keyValueConverterConfigs(String converterPrefix)

    • toString

      public String toString()
      Overrides:
      toString in class Object

    • getContextName

      public String getContextName()
      Specified by:
      getContextName in class io.debezium.config.CommonConnectorConfig

    • getSourceInfoStructMaker

      protected io.debezium.connector.SourceInfoStructMaker<? extends io.debezium.connector.AbstractSourceInfo> getSourceInfoStructMaker(io.debezium.config.CommonConnectorConfig.Version version)
      Specified by:
      getSourceInfoStructMaker in class io.debezium.config.CommonConnectorConfig

    • getConnectorName

      public String getConnectorName()
      Specified by:
      getConnectorName in class io.debezium.config.CommonConnectorConfig

    • getValidationFieldSet

      public io.debezium.config.Field.Set getValidationFieldSet()

    • setValidationFieldList

      public void setValidationFieldList(List<io.debezium.config.Field> validationFieldList)

    • getNodeId

      public String getNodeId()