Packages

final case class ConsumerSettings(properties: Map[String, AnyRef] = Map.empty, closeTimeout: zio.Duration = 30.seconds, pollTimeout: zio.Duration = 50.millis, commitTimeout: zio.Duration = ..., offsetRetrieval: OffsetRetrieval = OffsetRetrieval.Auto(), rebalanceListener: RebalanceListener = RebalanceListener.noop, restartStreamOnRebalancing: Boolean = false, fetchStrategy: FetchStrategy = QueueSizeBasedFetchStrategy()) extends Product with Serializable

Settings for the consumer.

To stay source compatible with future releases, you are recommended to construct the settings as follows:

ConsumerSettings(bootstrapServers)
  .withGroupId(groupId)
  .withProperties(properties)
  .... etc.
Linear Supertypes
Serializable, Serializable, Product, Equals, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. ConsumerSettings
  2. Serializable
  3. Serializable
  4. Product
  5. Equals
  6. AnyRef
  7. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new ConsumerSettings(properties: Map[String, AnyRef] = Map.empty, closeTimeout: zio.Duration = 30.seconds, pollTimeout: zio.Duration = 50.millis, commitTimeout: zio.Duration = ..., offsetRetrieval: OffsetRetrieval = OffsetRetrieval.Auto(), rebalanceListener: RebalanceListener = RebalanceListener.noop, restartStreamOnRebalancing: Boolean = false, fetchStrategy: FetchStrategy = QueueSizeBasedFetchStrategy())

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native() @IntrinsicCandidate()
  6. val closeTimeout: zio.Duration
  7. val commitTimeout: zio.Duration
  8. def driverSettings: Map[String, AnyRef]
  9. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  10. val fetchStrategy: FetchStrategy
  11. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native() @IntrinsicCandidate()
  12. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  13. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  14. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @IntrinsicCandidate()
  15. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @IntrinsicCandidate()
  16. val offsetRetrieval: OffsetRetrieval
  17. val pollTimeout: zio.Duration
  18. val properties: Map[String, AnyRef]
  19. val rebalanceListener: RebalanceListener
  20. val restartStreamOnRebalancing: Boolean
  21. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  22. def tuneForHighThroughput: ConsumerSettings

    Tunes the consumer for high throughput.

    Tunes the consumer for high throughput.

    Sets poll timeout to 500 ms, max.poll.records to 2000 and partitionPreFetchBufferLimit to 4096.

    See also

    ConsumerSettings.withPollTimeout

    ConsumerSettings.withMaxPollRecords

    zio.kafka.consumer.fetch.QueueSizeBasedFetchStrategy

  23. def tuneForLowLatency: ConsumerSettings

    Tunes the consumer for low latency.

    Tunes the consumer for low latency.

    Sets poll timeout to 50 ms, max.poll.records to 100 and partitionPreFetchBufferLimit to 512.

    See also

    ConsumerSettings.withPollTimeout

    ConsumerSettings.withMaxPollRecords

    zio.kafka.consumer.fetch.QueueSizeBasedFetchStrategy

  24. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  25. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  26. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  27. def withBootstrapServers(servers: List[String]): ConsumerSettings
  28. def withClientId(clientId: String): ConsumerSettings
  29. def withCloseTimeout(timeout: zio.Duration): ConsumerSettings
  30. def withCommitTimeout(timeout: zio.Duration): ConsumerSettings
  31. def withCredentials(credentialsStore: KafkaCredentialStore): ConsumerSettings
  32. def withFetchStrategy(fetchStrategy: FetchStrategy): ConsumerSettings

    WARNING: zio.kafka.consumer.fetch.FetchStrategy is an EXPERIMENTAL API and may change in an incompatible way without notice in any zio-kafka version.

    WARNING: zio.kafka.consumer.fetch.FetchStrategy is an EXPERIMENTAL API and may change in an incompatible way without notice in any zio-kafka version.

    fetchStrategy

    The fetch strategy which selects which partitions can fetch data in the next poll. The default is to use the zio.kafka.consumer.fetch.QueueSizeBasedFetchStrategy with a partitionPreFetchBufferLimit parameter of 1024, which is calculated by taking 2 * the default max.poll.records of 500, rounded to the nearest power of 2.

  33. def withGroupId(groupId: String): ConsumerSettings
  34. def withGroupInstanceId(groupInstanceId: String): ConsumerSettings
  35. def withMaxPollInterval(maxPollInterval: zio.Duration): ConsumerSettings

    Set Kafka's max.poll.interval.ms configuration.

    Set Kafka's max.poll.interval.ms configuration. See https://kafka.apache.org/documentation/#consumerconfigs_max.poll.interval.ms for more information.

    Zio-kafka uses this value also to determine whether a stream stopped processing. If no chunks are pulled from a stream for this interval (while data is available) we consider the stream to be halted. When this happens we interrupt the stream with a failure. In addition the entire consumer is shutdown. In future versions of zio-kafka we may (instead of a shutdown) stop only the affected subscription.

    The default is 5 minutes. Make sure that all records from a single poll can be processed in this interval. The maximum number of records in a single poll is configured with the max.poll.records configuration (see https://kafka.apache.org/documentation/#consumerconfigs_max.poll.records and withMaxPollRecords).

  36. def withMaxPollRecords(maxPollRecords: Int): ConsumerSettings

    Set Kafka's max.poll.records configuration.

    Set Kafka's max.poll.records configuration. See https://kafka.apache.org/documentation/#consumerconfigs_max.poll.records for more information.

  37. def withOffsetRetrieval(retrieval: OffsetRetrieval): ConsumerSettings
  38. def withPartitionPreFetchBufferLimit(partitionPreFetchBufferLimit: Int): ConsumerSettings

    partitionPreFetchBufferLimit

    The queue size below which more records are fetched and buffered (per partition). This buffer improves throughput and supports varying downstream message processing time, while maintaining some backpressure. Large values effectively disable backpressure at the cost of high memory usage, low values will effectively disable prefetching in favour of low memory consumption. The number of records that is fetched on every poll is controlled by the max.poll.records setting, the number of records fetched for every partition is somewhere between 0 and max.poll.records. A value that is a power of 2 offers somewhat better queueing performance. The default value for this parameter is 1024. It is calculated by taking 2 * the default max.poll.records of 500, rounded to the nearest power of 2.

  39. def withPollTimeout(timeout: zio.Duration): ConsumerSettings

    The maximum time to block while polling the Kafka consumer.

    The maximum time to block while polling the Kafka consumer. The Kafka consumer will return earlier when the maximum number of record to poll (see https://kafka.apache.org/documentation/#consumerconfigs_max.poll.records) is collected.

    The default is 50ms which is good for low latency applications. Set this higher, e.g. 500ms for better throughput.

  40. def withProperties(kvs: Map[String, AnyRef]): ConsumerSettings
  41. def withProperties(kvs: (String, AnyRef)*): ConsumerSettings
  42. def withProperty(key: String, value: AnyRef): ConsumerSettings
  43. def withRebalanceListener(listener: RebalanceListener): ConsumerSettings
  44. def withRestartStreamOnRebalancing(value: Boolean): ConsumerSettings

    value

    When true _all_ streams are restarted during a rebalance, including those streams that are not revoked. The default is false.

Deprecated Value Members

  1. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] ) @Deprecated
    Deprecated
  2. def withMaxPartitionQueueSize(partitionPreFetchBufferLimit: Int): ConsumerSettings
    Annotations
    @deprecated
    Deprecated

    (Since version 2.6.0) Use withPartitionPreFetchBufferLimit instead

Inherited from Serializable

Inherited from Serializable

Inherited from Product

Inherited from Equals

Inherited from AnyRef

Inherited from Any

Ungrouped