cascading.scheme.hadoop

Class SequenceFile

java.lang.Object

cascading.scheme.Scheme<Configuration,RecordReader,OutputCollector,java.lang.Object[],java.lang.Void>

cascading.scheme.hadoop.SequenceFile

All Implemented Interfaces:

cascading.scheme.FileFormat, cascading.util.Traceable, java.io.Serializable

Direct Known Subclasses:

WritableSequenceFile

public class SequenceFile
extends cascading.scheme.Scheme<Configuration,RecordReader,OutputCollector,java.lang.Object[],java.lang.Void>
implements cascading.scheme.FileFormat

A SequenceFile is a type of Scheme, which is a flat file consisting of binary key/value pairs. This is a space and time efficient means to store data.

See Also:

Serialized Form

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	SequenceFile() Protected for use by TempDfs and other subclasses.
	SequenceFile(cascading.tuple.Fields fields) Creates a new SequenceFile instance that stores the given field names.

Method Summary

Modifier and Type	Method and Description
java.lang.String	getExtension()
void	sink(cascading.flow.FlowProcess<? extends Configuration> flowProcess, cascading.scheme.SinkCall<java.lang.Void,OutputCollector> sinkCall)
void	sinkConfInit(cascading.flow.FlowProcess<? extends Configuration> flowProcess, cascading.tap.Tap<Configuration,RecordReader,OutputCollector> tap, Configuration conf)
boolean	source(cascading.flow.FlowProcess<? extends Configuration> flowProcess, cascading.scheme.SourceCall<java.lang.Object[],RecordReader> sourceCall)
void	sourceCleanup(cascading.flow.FlowProcess<? extends Configuration> flowProcess, cascading.scheme.SourceCall<java.lang.Object[],RecordReader> sourceCall)
void	sourceConfInit(cascading.flow.FlowProcess<? extends Configuration> flowProcess, cascading.tap.Tap<Configuration,RecordReader,OutputCollector> tap, Configuration conf)
void	sourcePrepare(cascading.flow.FlowProcess<? extends Configuration> flowProcess, cascading.scheme.SourceCall<java.lang.Object[],RecordReader> sourceCall)

Methods inherited from class cascading.scheme.Scheme

equals, getNumSinkParts, getSinkFields, getSourceFields, getTrace, hashCode, isSink, isSource, isSymmetrical, presentSinkFields, presentSinkFieldsInternal, presentSourceFields, presentSourceFieldsInternal, retrieveSinkFields, retrieveSourceFields, setNumSinkParts, setSinkFields, setSourceFields, sinkCleanup, sinkPrepare, sinkWrap, sourceRePrepare, sourceWrap, toString

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

SequenceFile

protected SequenceFile()

Protected for use by TempDfs and other subclasses. Not for general consumption.

SequenceFile

@ConstructorProperties(value="fields")
public SequenceFile(cascading.tuple.Fields fields)

Creates a new SequenceFile instance that stores the given field names.

Parameters:

fields -

Method Detail

sourceConfInit

public void sourceConfInit(cascading.flow.FlowProcess<? extends Configuration> flowProcess,
                           cascading.tap.Tap<Configuration,RecordReader,OutputCollector> tap,
                           Configuration conf)

Specified by:

sourceConfInit in class cascading.scheme.Scheme<Configuration,RecordReader,OutputCollector,java.lang.Object[],java.lang.Void>

sinkConfInit

public void sinkConfInit(cascading.flow.FlowProcess<? extends Configuration> flowProcess,
                         cascading.tap.Tap<Configuration,RecordReader,OutputCollector> tap,
                         Configuration conf)

Specified by:

sinkConfInit in class cascading.scheme.Scheme<Configuration,RecordReader,OutputCollector,java.lang.Object[],java.lang.Void>

sourcePrepare

public void sourcePrepare(cascading.flow.FlowProcess<? extends Configuration> flowProcess,
                          cascading.scheme.SourceCall<java.lang.Object[],RecordReader> sourceCall)

Overrides:

sourcePrepare in class cascading.scheme.Scheme<Configuration,RecordReader,OutputCollector,java.lang.Object[],java.lang.Void>

source

public boolean source(cascading.flow.FlowProcess<? extends Configuration> flowProcess,
                      cascading.scheme.SourceCall<java.lang.Object[],RecordReader> sourceCall)
               throws java.io.IOException

Specified by:

source in class cascading.scheme.Scheme<Configuration,RecordReader,OutputCollector,java.lang.Object[],java.lang.Void>

Throws:

java.io.IOException

sourceCleanup

public void sourceCleanup(cascading.flow.FlowProcess<? extends Configuration> flowProcess,
                          cascading.scheme.SourceCall<java.lang.Object[],RecordReader> sourceCall)

Overrides:

sourceCleanup in class cascading.scheme.Scheme<Configuration,RecordReader,OutputCollector,java.lang.Object[],java.lang.Void>

sink

public void sink(cascading.flow.FlowProcess<? extends Configuration> flowProcess,
                 cascading.scheme.SinkCall<java.lang.Void,OutputCollector> sinkCall)
          throws java.io.IOException

Specified by:

sink in class cascading.scheme.Scheme<Configuration,RecordReader,OutputCollector,java.lang.Object[],java.lang.Void>

Throws:

java.io.IOException

getExtension

public java.lang.String getExtension()

Specified by:

getExtension in interface cascading.scheme.FileFormat