Packages

class MapReduce[F[_], K, V, K1, V1] extends Process[F] with StrictLogging

In-memory implementation of MapReduce algorithm:

F

an effect type

K

initial key type

V

initial value type

K1

key type produced by mapper

V1

value type produced by mapper

Linear Supertypes
StrictLogging, Process[F], FlowSyntax[F], EventSyntax[F], WithDsl[F], AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. MapReduce
  2. StrictLogging
  3. Process
  4. FlowSyntax
  5. EventSyntax
  6. WithDsl
  7. AnyRef
  8. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new MapReduce(mapperFunction: (Record[K, V]) ⇒ Seq[Record[K1, V1]], nMapperWorkers: Int, reducerFunction: (K1, Seq[V1]) ⇒ V1, nReducerWorkers: Int)(implicit arg0: Ordering[K1])

    mapperFunction

    a function that takes a V - pair of (key, value) and produces a sequence of V1

    nMapperWorkers

    the number of mapper workers. Tasks aren't assigned to workers, instead workers stealing tasks from the task queue

    reducerFunction

    a function that takes a key and values associated with this key and produces a single value

    nReducerWorkers

    the number of reducer workers. Records assigned to a worker using a partition function: key.hashCode() % nReducerWorkers

Type Members

  1. implicit class EventOps extends AnyRef
    Definition Classes
    EventSyntax
  2. implicit class EventSeqOps extends AnyRef
    Definition Classes
    EventSyntax
  3. implicit class FreeOps[A] extends AnyRef
    Definition Classes
    FlowSyntax
  4. class MapWorker extends Process[F]

    Mapper worker reads chucks from the shared queue.

    Mapper worker reads chucks from the shared queue. Once the queue is empty the worker stops itself.

  5. type Program = Free[[β$0$]FlowOp[F, β$0$], Unit]
    Definition Classes
    Process
  6. type Receive = PartialFunction[Event, DslF[F, Unit]]
    Definition Classes
    Process
  7. class ReduceWorker extends Process[F]

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. def ++[B](that: Process[F]): Process[F]
    Definition Classes
    Process
  4. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  5. def and(that: Process[F]): Process[F]
    Definition Classes
    Process
  6. def apply(e: Event): Program
    Definition Classes
    Process
  7. def apply(caller: ProcessRef, e: Event): Program
    Definition Classes
    Process
  8. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  9. def clone(): AnyRef
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @native() @throws( ... )
  10. val dsl: FlowOps[F, [β$19$]FlowOp[F, β$19$]]
    Attributes
    protected
    Definition Classes
    WithDsl
  11. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  12. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  13. def finalize(): Unit
    Attributes
    protected[java.lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  14. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  15. def handle: Receive
    Definition Classes
    MapReduce → Process
  16. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  17. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  18. val logger: Logger
    Attributes
    protected
    Definition Classes
    StrictLogging
  19. val name: String
    Definition Classes
    Process
  20. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  21. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  22. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  23. def or(that: Process[F]): Process[F]
    Definition Classes
    Process
  24. val ref: ProcessRef
    Definition Classes
    Process
  25. def switch(newHandler: ⇒ Receive): Program
    Definition Classes
    Process
  26. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  27. def toString(): String
    Definition Classes
    Process → AnyRef → Any
  28. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  29. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  30. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @native() @throws( ... )

Inherited from StrictLogging

Inherited from Process[F]

Inherited from FlowSyntax[F]

Inherited from EventSyntax[F]

Inherited from WithDsl[F]

Inherited from AnyRef

Inherited from Any

Ungrouped