org.calrissian.accumulorecipes.commons.iterators

Class ReadAheadIterator

java.lang.Object

org.calrissian.accumulorecipes.commons.iterators.ReadAheadIterator

All Implemented Interfaces:

org.apache.accumulo.core.iterators.OptionDescriber, org.apache.accumulo.core.iterators.SortedKeyValueIterator<org.apache.accumulo.core.data.Key,org.apache.accumulo.core.data.Value>

public class ReadAheadIterator
extends Object
implements org.apache.accumulo.core.iterators.SortedKeyValueIterator<org.apache.accumulo.core.data.Key,org.apache.accumulo.core.data.Value>, org.apache.accumulo.core.iterators.OptionDescriber

This iterator takes the source iterator (the one below it in the iterator stack) and puts it in a background thread. The background thread continues processing and fills a queue with the Keys and Values from the source iterator. When seek() is called on this iterator, it pauses the background thread, clears the queue, calls seek() on the source iterator, then resumes the thread filling the queue.

Users of this iterator can set the queue size, default is five elements. Users must be aware of the potential for OutOfMemory errors when using this iterator with large queue sizes or large objects. This iterator copies the Key and Value from the source iterator and puts them into the queue.

This iterator introduces some parallelism into the server side iterator stack. One use case for this would be when an iterator takes a relatively long time to process each K,V pair and causes the iterators above it to wait. By putting the longer running iterator in a background thread we should be able to achieve greater throughput.

NOTE: Experimental!

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.accumulo.core.iterators.OptionDescriber

org.apache.accumulo.core.iterators.OptionDescriber.IteratorOptions

Field Summary

Fields

Modifier and Type	Field and Description
static String	QUEUE_SIZE
static String	TIMEOUT

Constructor Summary

Constructors

Modifier	Constructor and Description
	ReadAheadIterator()
protected	ReadAheadIterator(ReadAheadIterator other, org.apache.accumulo.core.iterators.IteratorEnvironment env)

Method Summary

Methods

Modifier and Type	Method and Description
org.apache.accumulo.core.iterators.SortedKeyValueIterator<org.apache.accumulo.core.data.Key,org.apache.accumulo.core.data.Value>	deepCopy(org.apache.accumulo.core.iterators.IteratorEnvironment env)
org.apache.accumulo.core.iterators.OptionDescriber.IteratorOptions	describeOptions()
org.apache.accumulo.core.data.Key	getTopKey()
org.apache.accumulo.core.data.Value	getTopValue()
boolean	hasTop()
void	init(org.apache.accumulo.core.iterators.SortedKeyValueIterator<org.apache.accumulo.core.data.Key,org.apache.accumulo.core.data.Value> source, Map<String,String> options, org.apache.accumulo.core.iterators.IteratorEnvironment env)
void	next() Populate the key and value
void	seek(org.apache.accumulo.core.data.Range range, Collection<org.apache.accumulo.core.data.ByteSequence> columnFamilies, boolean inclusive) Seek to the next matching cell and call next to populate the key and value.
boolean	validateOptions(Map<String,String> options)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

QUEUE_SIZE

public static final String QUEUE_SIZE

See Also:

Constant Field Values

TIMEOUT

public static final String TIMEOUT

See Also:

Constant Field Values

Constructor Detail

ReadAheadIterator

protected ReadAheadIterator(ReadAheadIterator other,
                 org.apache.accumulo.core.iterators.IteratorEnvironment env)

ReadAheadIterator

public ReadAheadIterator()

Method Detail

deepCopy

public org.apache.accumulo.core.iterators.SortedKeyValueIterator<org.apache.accumulo.core.data.Key,org.apache.accumulo.core.data.Value> deepCopy(org.apache.accumulo.core.iterators.IteratorEnvironment env)

Specified by:

deepCopy in interface org.apache.accumulo.core.iterators.SortedKeyValueIterator<org.apache.accumulo.core.data.Key,org.apache.accumulo.core.data.Value>

getTopKey

public org.apache.accumulo.core.data.Key getTopKey()

Specified by:

getTopKey in interface org.apache.accumulo.core.iterators.SortedKeyValueIterator<org.apache.accumulo.core.data.Key,org.apache.accumulo.core.data.Value>

getTopValue

public org.apache.accumulo.core.data.Value getTopValue()

Specified by:

getTopValue in interface org.apache.accumulo.core.iterators.SortedKeyValueIterator<org.apache.accumulo.core.data.Key,org.apache.accumulo.core.data.Value>

hasTop

public boolean hasTop()

Specified by:

hasTop in interface org.apache.accumulo.core.iterators.SortedKeyValueIterator<org.apache.accumulo.core.data.Key,org.apache.accumulo.core.data.Value>

init

public void init(org.apache.accumulo.core.iterators.SortedKeyValueIterator<org.apache.accumulo.core.data.Key,org.apache.accumulo.core.data.Value> source,
        Map<String,String> options,
        org.apache.accumulo.core.iterators.IteratorEnvironment env)
          throws IOException

Specified by:

init in interface org.apache.accumulo.core.iterators.SortedKeyValueIterator<org.apache.accumulo.core.data.Key,org.apache.accumulo.core.data.Value>

Throws:

IOException

next

public void next()
          throws IOException

Populate the key and value

Specified by:

next in interface org.apache.accumulo.core.iterators.SortedKeyValueIterator<org.apache.accumulo.core.data.Key,org.apache.accumulo.core.data.Value>

Throws:

IOException

seek

public void seek(org.apache.accumulo.core.data.Range range,
        Collection<org.apache.accumulo.core.data.ByteSequence> columnFamilies,
        boolean inclusive)
          throws IOException

Seek to the next matching cell and call next to populate the key and value.

Specified by:

seek in interface org.apache.accumulo.core.iterators.SortedKeyValueIterator<org.apache.accumulo.core.data.Key,org.apache.accumulo.core.data.Value>

Throws:

IOException

describeOptions

public org.apache.accumulo.core.iterators.OptionDescriber.IteratorOptions describeOptions()

Specified by:

describeOptions in interface org.apache.accumulo.core.iterators.OptionDescriber

validateOptions

public boolean validateOptions(Map<String,String> options)

Specified by:

validateOptions in interface org.apache.accumulo.core.iterators.OptionDescriber