Packages

o

zio.kafka.consumer

Consumer

object Consumer

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. Consumer
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Type Members

  1. sealed trait AutoOffsetStrategy extends AnyRef
  2. final case class Live[R, K, V](consumer: ConsumerAccess, settings: ConsumerSettings, runloop: Runloop, keyDeserializer: Deserializer[R, K], valueDeserializer: Deserializer[R, V]) extends Service[R, K, V] with Product with Serializable
  3. sealed trait OffsetRetrieval extends AnyRef
  4. trait Service[R, K, V] extends AnyRef

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def assignment[R, K, V](implicit arg0: zio.Tagged[R], arg1: zio.Tagged[K], arg2: zio.Tagged[V]): RIO[R with Blocking with Consumer[R, K, V], Set[TopicPartition]]

    Accessor method for Service.assignment

  6. def beginningOffsets[R, K, V](partitions: Set[TopicPartition], timeout: Duration = Duration.Infinity)(implicit arg0: zio.Tagged[R], arg1: zio.Tagged[K], arg2: zio.Tagged[V]): RIO[R with Blocking with Consumer[R, K, V], Map[TopicPartition, Long]]

    Accessor method for Service.beginningOffsets

  7. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  8. def committed[R, K, V](partitions: Set[TopicPartition], timeout: Duration = Duration.Infinity)(implicit arg0: zio.Tagged[R], arg1: zio.Tagged[K], arg2: zio.Tagged[V]): RIO[R with Blocking with Consumer[R, K, V], Map[TopicPartition, Option[OffsetAndMetadata]]]

    Accessor method for Service.committed

  9. def consumeWith[R, R1, K, V](settings: ConsumerSettings, subscription: Subscription, keyDeserializer: Deserializer[R1, K], valueDeserializer: Deserializer[R1, V], commitRetryPolicy: Schedule[Clock, Any, Any] = ...)(f: (K, V) ⇒ ZIO[R, Nothing, Unit])(implicit arg0: zio.Tagged[R1], arg1: zio.Tagged[K], arg2: zio.Tagged[V]): ZIO[R with R1 with Blocking with Clock, Throwable, Unit]

    Execute an effect for each record and commit the offset after processing

    Execute an effect for each record and commit the offset after processing

    This method is the easiest way of processing messages on a Kafka topic.

    Messages on a single partition are processed sequentially, while the processing of multiple partitions happens in parallel.

    Offsets are committed after execution of the effect. They are batched when a commit action is in progress to avoid backpressuring the stream. When commits fail due to a org.apache.kafka.clients.consumer.RetriableCommitFailedException they are retried according to commitRetryPolicy

    The effect should absorb any failures. Failures should be handled by retries or ignoring the error, which will result in the Kafka message being skipped.

    Messages are processed with 'at least once' consistency: it is not guaranteed that every message that is processed by the effect has a corresponding offset commit before stream termination.

    Usage example:

    val settings: ConsumerSettings = ???
val subscription = Subscription.Topics(Set("my-kafka-topic"))

val consumerIO = Consumer.consumeWith(settings, subscription, Serdes.string, Serdes.string) { case (key, value) =>
  // Process the received record here
  putStrLn(s"Received record: ${key}: ${value}")
}
    R

    Environment for the consuming effect

    R1

    Environment for the deserializers

    K

    Type of keys (an implicit Deserializer should be in scope)

    V

    Type of values (an implicit Deserializer should be in scope)

    settings

    Settings for creating a Consumer

    subscription

    Topic subscription parameters

    keyDeserializer

    Deserializer for the key of the messages

    valueDeserializer

    Deserializer for the value of the messages

    commitRetryPolicy

    Retry commits that failed due to a RetriableCommitFailedException according to this schedule

    f

    Function that returns the effect to execute for each message. It is passed the key and value

    returns

    Effect that completes with a unit value only when interrupted. May fail when the Consumer fails.

  10. def endOffsets[R, K, V](partitions: Set[TopicPartition], timeout: Duration = Duration.Infinity)(implicit arg0: zio.Tagged[R], arg1: zio.Tagged[K], arg2: zio.Tagged[V]): RIO[R with Blocking with Consumer[R, K, V], Map[TopicPartition, Long]]

    Accessor method for Service.endOffsets

  11. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  12. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  13. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  14. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  15. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  16. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  17. def listTopics[R, K, V](timeout: Duration = Duration.Infinity)(implicit arg0: zio.Tagged[R], arg1: zio.Tagged[K], arg2: zio.Tagged[V]): RIO[R with Blocking with Consumer[R, K, V], Map[String, List[PartitionInfo]]]

    Accessor method for Service.listTopics

  18. def live[R, K, V](implicit arg0: zio.Tagged[R], arg1: zio.Tagged[K], arg2: zio.Tagged[V]): ZLayer[Clock with Blocking with Has[Deserializer[R, K]] with Has[Deserializer[R, V]] with Has[ConsumerSettings] with Has[Diagnostics], Throwable, Consumer[R, K, V]]
  19. def make[R, K, V](settings: ConsumerSettings, keyDeserializer: Deserializer[R, K], valueDeserializer: Deserializer[R, V], diagnostics: Diagnostics = Diagnostics.NoOp)(implicit arg0: zio.Tagged[R], arg1: zio.Tagged[K], arg2: zio.Tagged[V]): ZLayer[Clock with Blocking, Throwable, Consumer[R, K, V]]
  20. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  21. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  22. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  23. val offsetBatches: ZSink[Any, Nothing, Nothing, Offset, OffsetBatch]
  24. def offsetsForTimes[R, K, V](timestamps: Map[TopicPartition, Long], timeout: Duration = Duration.Infinity)(implicit arg0: zio.Tagged[R], arg1: zio.Tagged[K], arg2: zio.Tagged[V]): RIO[R with Blocking with Consumer[R, K, V], Map[TopicPartition, OffsetAndTimestamp]]

    Accessor method for Service.offsetsForTimes

  25. def partitionedStream[R, K, V](implicit arg0: zio.Tagged[R], arg1: zio.Tagged[K], arg2: zio.Tagged[V]): ZStream[Consumer[R, K, V] with Clock with Blocking, Throwable, (TopicPartition, ZStreamChunk[R, Throwable, CommittableRecord[K, V]])]

    Accessor method for Service.partitionedStream

  26. def partitionsFor[R, K, V](topic: String, timeout: Duration = Duration.Infinity)(implicit arg0: zio.Tagged[R], arg1: zio.Tagged[K], arg2: zio.Tagged[V]): RIO[R with Blocking with Consumer[R, K, V], List[PartitionInfo]]

    Accessor method for Service.partitionsFor

  27. def plainStream[R, K, V](implicit arg0: zio.Tagged[R], arg1: zio.Tagged[K], arg2: zio.Tagged[V]): ZStreamChunk[R with Consumer[R, K, V] with Clock with Blocking, Throwable, CommittableRecord[K, V]]

    Accessor method for Service.plainStream

  28. def position[R, K, V](partition: TopicPartition, timeout: Duration = Duration.Infinity)(implicit arg0: zio.Tagged[R], arg1: zio.Tagged[K], arg2: zio.Tagged[V]): RIO[R with Blocking with Consumer[R, K, V], Long]

    Accessor method for Service.position

  29. def stopConsumption[R, K, V](implicit arg0: zio.Tagged[R], arg1: zio.Tagged[K], arg2: zio.Tagged[V]): RIO[Consumer[R, K, V], Unit]

    Accessor method for Service.stopConsumption

  30. def subscribe[R, K, V](subscription: Subscription)(implicit arg0: zio.Tagged[R], arg1: zio.Tagged[K], arg2: zio.Tagged[V]): RIO[R with Blocking with Consumer[R, K, V], Unit]

    Accessor method for Service.subscribe

  31. def subscribeAnd[R, K, V](subscription: Subscription)(implicit arg0: zio.Tagged[R], arg1: zio.Tagged[K], arg2: zio.Tagged[V]): SubscribedConsumerFromEnvironment[R, K, V]

    Accessor method for Service.subscribeAnd

  32. def subscription[R, K, V](implicit arg0: zio.Tagged[R], arg1: zio.Tagged[K], arg2: zio.Tagged[V]): RIO[R with Blocking with Consumer[R, K, V], Set[String]]

    Accessor method for Service.subscription

  33. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  34. def toString(): String
    Definition Classes
    AnyRef → Any
  35. def unsubscribe[R, K, V](implicit arg0: zio.Tagged[R], arg1: zio.Tagged[K], arg2: zio.Tagged[V]): RIO[R with Blocking with Consumer[R, K, V], Unit]

    Accessor method for Service.unsubscribe

  36. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  37. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  38. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  39. def withConsumerService[R, RC, K, V, A](r: (Service[R, K, V]) ⇒ RIO[R with RC with Blocking, A])(implicit arg0: zio.Tagged[R], arg1: zio.Tagged[K], arg2: zio.Tagged[V]): RIO[R with RC with Blocking with Consumer[R, K, V], A]
  40. object AutoOffsetStrategy
  41. object OffsetRetrieval

Inherited from AnyRef

Inherited from Any

Ungrouped