eu.stratosphere.api.java.record.io.avro

Class AvroRecordInputFormat

java.lang.Object

eu.stratosphere.api.common.io.FileInputFormat<eu.stratosphere.types.Record>

eu.stratosphere.api.java.record.io.FileInputFormat

eu.stratosphere.api.java.record.io.avro.AvroRecordInputFormat

All Implemented Interfaces:

eu.stratosphere.api.common.io.InputFormat<eu.stratosphere.types.Record,eu.stratosphere.core.fs.FileInputSplit>, Serializable

public class AvroRecordInputFormat
extends FileInputFormat

Input format to read Avro files. The input format currently supports only flat avro schemas. So there is no support for complex types except for nullable primitve fields, e.g. ["string", null] (See http://avro.apache.org/docs/current/spec.html#schema_complex)

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	AvroRecordInputFormat.BooleanListValue
static class	AvroRecordInputFormat.BooleanMapValue
static class	AvroRecordInputFormat.DoubleListValue
static class	AvroRecordInputFormat.DoubleMapValue
static class	AvroRecordInputFormat.FloatListValue
static class	AvroRecordInputFormat.FloatMapValue
static class	AvroRecordInputFormat.IntListValue
static class	AvroRecordInputFormat.IntMapValue
static class	AvroRecordInputFormat.LongListValue
static class	AvroRecordInputFormat.LongMapValue
static class	AvroRecordInputFormat.StringListValue
static class	AvroRecordInputFormat.StringMapValue

Nested classes/interfaces inherited from class eu.stratosphere.api.common.io.FileInputFormat

eu.stratosphere.api.common.io.FileInputFormat.AbstractConfigBuilder<T>, eu.stratosphere.api.common.io.FileInputFormat.ConfigBuilder, eu.stratosphere.api.common.io.FileInputFormat.FileBaseStatistics, eu.stratosphere.api.common.io.FileInputFormat.InputSplitOpenThread

Field Summary

Fields inherited from class eu.stratosphere.api.common.io.FileInputFormat

DEFLATE_SUFFIX, filePath, minSplitSize, numSplits, openTimeout, READ_WHOLE_SPLIT_FLAG, splitLength, splitStart, stream, unsplittable

Constructor Summary

Constructors

Constructor and Description
AvroRecordInputFormat()

Method Summary

Methods

Modifier and Type	Method and Description
eu.stratosphere.core.fs.FileInputSplit[]	createInputSplits(int minNumSplits) Set minNumSplits to number of files.
eu.stratosphere.types.Record	nextRecord(eu.stratosphere.types.Record record)
void	open(eu.stratosphere.core.fs.FileInputSplit split)
boolean	reachedEnd()

Methods inherited from class eu.stratosphere.api.common.io.FileInputFormat

acceptFile, close, configure, configureFileFormat, getFilePath, getFileStats, getInputSplitType, getMinSplitSize, getNumSplits, getOpenTimeout, getSplitLength, getSplitStart, getStatistics, setFilePath, setFilePath, setMinSplitSize, setNumSplits, setOpenTimeout, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

AvroRecordInputFormat

public AvroRecordInputFormat()

Method Detail

open

public void open(eu.stratosphere.core.fs.FileInputSplit split)
          throws IOException

Specified by:

open in interface eu.stratosphere.api.common.io.InputFormat<eu.stratosphere.types.Record,eu.stratosphere.core.fs.FileInputSplit>

Overrides:

open in class eu.stratosphere.api.common.io.FileInputFormat<eu.stratosphere.types.Record>

Throws:

IOException

reachedEnd

public boolean reachedEnd()
                   throws IOException

Throws:

IOException

nextRecord

public eu.stratosphere.types.Record nextRecord(eu.stratosphere.types.Record record)
                                        throws IOException

Throws:

IOException

createInputSplits

public eu.stratosphere.core.fs.FileInputSplit[] createInputSplits(int minNumSplits)
                                                           throws IOException

Set minNumSplits to number of files.

Specified by:

createInputSplits in interface eu.stratosphere.api.common.io.InputFormat<eu.stratosphere.types.Record,eu.stratosphere.core.fs.FileInputSplit>

Overrides:

createInputSplits in class eu.stratosphere.api.common.io.FileInputFormat<eu.stratosphere.types.Record>

Throws:

IOException