io.cdap.wrangler.sampling

Class Reservoir<T>

java.lang.Object

io.cdap.wrangler.sampling.Sampler<T>

io.cdap.wrangler.sampling.Reservoir<T>

Type Parameters:

T - The type of the sampler.

public class Reservoir<T>
extends Sampler<T>

A simple in memory implementation of Reservoir Sampling without replacement, and with only one pass through the input iteration whose size is unpredictable. The basic idea behind this sampler implementation is to generate a random number for each input element as its weight, select the top K elements with max weight. As the weights are generated randomly, so are the selected top K elements. In the first phase, we generate random numbers as the weights for each element and select top K elements as the output of each partitions.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Reservoir.IntermediateSample<T> An intermediate sample

Field Summary

Fields inherited from class io.cdap.wrangler.sampling.Sampler

emptyIterable, EPSILON

Constructor Summary

Constructors

Constructor and Description
Reservoir(int numSamples) Create a new sampler with reservoir size and a default random number generator.
Reservoir(int numSamples, long seed) Create a new sampler with reservoir size and the seed for random number generator.
Reservoir(int numSamples, Random random) Create a new sampler with reservoir size and a supplied random number generator.

Method Summary

Modifier and Type	Method and Description
Iterator<T>	sample(Iterator<T> input) Randomly sample the elements from input in sequence, and return the result iterator.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Reservoir

public Reservoir(int numSamples,
                 Random random)

Create a new sampler with reservoir size and a supplied random number generator.

Parameters:

numSamples - Maximum number of samples to retain in reservoir, must be non-negative.

random - Instance of random number generator for sampling.

Reservoir

public Reservoir(int numSamples)

Create a new sampler with reservoir size and a default random number generator.

Parameters:

numSamples - Maximum number of samples to retain in reservoir, must be non-negative.

Reservoir

public Reservoir(int numSamples,
                 long seed)

Create a new sampler with reservoir size and the seed for random number generator.

Parameters:

numSamples - Maximum number of samples to retain in reservoir, must be non-negative.

seed - Random number generator seed.

Method Detail

sample

public Iterator<T> sample(Iterator<T> input)

Description copied from class: Sampler

Randomly sample the elements from input in sequence, and return the result iterator.

Specified by:

sample in class Sampler<T>

Parameters:

input - Scope data

Returns:

The sample result.