eu.stratosphere.arraymodel.functions

Class ReduceFunction

java.lang.Object

eu.stratosphere.api.common.functions.AbstractFunction

eu.stratosphere.arraymodel.functions.AbstractArrayModelFunction

eu.stratosphere.arraymodel.functions.ReduceFunction

All Implemented Interfaces:

Function, GenericReducer<Value[],Value[]>

Direct Known Subclasses:

WordCountArrayTuples.CountWords

public abstract class ReduceFunction
extends AbstractArrayModelFunction
implements GenericReducer<Value[],Value[]>

The ReduceFunction must be extended to provide a reducer implementation which is called by a Reduce PACT. By definition, the Reduce PACT calls the reduce implementation once for each distinct key and all records that come with that key. For details on the Reduce PACT read the documentation of the PACT programming model.

The ReduceFunction extension must be parameterized with the types of its input keys.

For a reduce implementation, the reduce() method must be implemented.

Constructor Summary

Constructors

Constructor and Description
ReduceFunction()

Method Summary

Methods

Modifier and Type	Method and Description
void	combine(Iterator<Value[]> records, Collector<Value[]> out) No default implementation provided.
Method	getUDFMethod()
abstract void	reduce(Iterator<Value[]> records, Collector<Value[]> out) The central function to be implemented for a reducer.

Methods inherited from class eu.stratosphere.arraymodel.functions.AbstractArrayModelFunction

getDataTypes

Methods inherited from class eu.stratosphere.api.common.functions.AbstractFunction

close, getIterationRuntimeContext, getRuntimeContext, open, setRuntimeContext

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface eu.stratosphere.api.common.functions.Function

close, getRuntimeContext, open, setRuntimeContext

Constructor Detail

ReduceFunction

public ReduceFunction()

Method Detail

reduce

public abstract void reduce(Iterator<Value[]> records,
          Collector<Value[]> out)
                     throws Exception

The central function to be implemented for a reducer. The function receives per call one key and all the values that belong to that key. Each key is guaranteed to be processed by exactly one function call across all involved instances across all computing nodes.

Specified by:

reduce in interface GenericReducer<Value[],Value[]>

Parameters:

records - All records that belong to the given input key.

out - The collector to hand results to.

Throws:

Exception - Implementations may forward exceptions, which are caught by the runtime. When the runtime catches an exception, it aborts the reduce task and lets the fail-over logic decide whether to retry the reduce execution.

combine

public void combine(Iterator<Value[]> records,
           Collector<Value[]> out)
             throws Exception

No default implementation provided. This method must be overridden by reduce stubs that want to make use of the combining feature. In addition, the ReduceFunction extending class must be annotated as Combinable. Note that this function must be implemented, if the reducer is annotated as combinable.

The use of the combiner is typically a pre-reduction of the data. It works similar as the reducer, only that is is not guaranteed to see all values with the same key in one call to the combine function. Since it is called prior to the reduce() method, input and output types of the combine method are the input types of the reduce() method.

Specified by:

combine in interface GenericReducer<Value[],Value[]>

Parameters:

records - The records to be combined. Unlike in the reduce method, these are not necessarily all records belonging to the given key.

out - The collector to write the result to.

Throws:

Exception - Implementations may forward exceptions, which are caught by the runtime. When the runtime catches an exception, it aborts the combine task and lets the fail-over logic decide whether to retry the combiner execution.

See Also:

eu.stratosphere.api.java.record.operators.ReduceOperator.Combinable

getUDFMethod

public Method getUDFMethod()

Specified by:

getUDFMethod in class AbstractArrayModelFunction