cascading.nested.json.hadoop2

Class JSONTextLine

java.lang.Object

cascading.scheme.Scheme<org.apache.hadoop.conf.Configuration,org.apache.hadoop.mapred.RecordReader,org.apache.hadoop.mapred.OutputCollector,java.lang.Object[],java.lang.Object[]>

cascading.scheme.hadoop.TextLine

cascading.nested.json.hadoop2.JSONTextLine

All Implemented Interfaces:

cascading.scheme.FileFormat, cascading.util.Traceable, java.io.Serializable

public class JSONTextLine
extends cascading.scheme.hadoop.TextLine

A JSONTextLine is a type of Scheme for JSON text files. Files are broken into lines, where each line is a JSON object. Either line-feed or carriage-return are used to signal end of line.

By default, this scheme returns a Tuple with one field, "json" with the type JSONCoercibleType.

Any Fields object passed to the constructor will have the JSONCoercibleType.TYPE type applied.

To create a binary JSON file, use the SequenceFile Scheme with one or more fields having the JSONCoercibleType type.

Note, when supplying a custom ObjectMapper, the default JSONCoercibleType.TYPE and ObjectMapper sets the DeserializationFeature.FAIL_ON_READING_DUP_TREE_KEY Jackson property.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class cascading.scheme.hadoop.TextLine

cascading.scheme.hadoop.TextLine.Compress

Field Summary

Fields

Modifier and Type	Field and Description
static cascading.tuple.Fields	DEFAULT_FIELDS

Fields inherited from class cascading.scheme.hadoop.TextLine

DEFAULT_CHARSET, DEFAULT_SOURCE_FIELDS

Constructor Summary

Constructors

Constructor and Description
JSONTextLine() Constructor JSONTextLine creates a new JSONTextLine instance for use with any of the Hadoop based FlowConnector instances returning results with the default field named "json".
JSONTextLine(cascading.tuple.Fields fields) Constructor JSONTextLine creates a new JSONTextLine instance for use with any of the Hadoop based FlowConnector instances.
JSONTextLine(cascading.tuple.Fields fields, java.lang.String charsetName) Constructor JSONTextLine creates a new JSONTextLine instance for use with any of the Hadoop based FlowConnector instances.
JSONTextLine(cascading.tuple.Fields fields, cascading.scheme.hadoop.TextLine.Compress sinkCompression) Constructor JSONTextLine creates a new JSONTextLine instance for use with any of the Hadoop based FlowConnector instances.
JSONTextLine(cascading.tuple.Fields fields, cascading.scheme.hadoop.TextLine.Compress sinkCompression, java.lang.String charsetName) Constructor JSONTextLine creates a new JSONTextLine instance for use with any of the Hadoop based FlowConnector instances.
JSONTextLine(ObjectMapper mapper, cascading.tuple.Fields fields) Constructor JSONTextLine creates a new JSONTextLine instance for use with any of the Hadoop based FlowConnector instances.
JSONTextLine(ObjectMapper mapper, cascading.tuple.Fields fields, java.lang.String charsetName) Constructor JSONTextLine creates a new JSONTextLine instance for use with any of the Hadoop based FlowConnector instances.
JSONTextLine(ObjectMapper mapper, cascading.tuple.Fields fields, cascading.scheme.hadoop.TextLine.Compress sinkCompression) Constructor JSONTextLine creates a new JSONTextLine instance for use with any of the Hadoop based FlowConnector instances.
JSONTextLine(ObjectMapper mapper, cascading.tuple.Fields fields, cascading.scheme.hadoop.TextLine.Compress sinkCompression, java.lang.String charsetName) Constructor JSONTextLine creates a new JSONTextLine instance for use with any of the Hadoop based FlowConnector instances.

Method Summary

Modifier and Type	Method and Description
java.lang.String	getExtension()
void	sink(cascading.flow.FlowProcess<? extends org.apache.hadoop.conf.Configuration> flowProcess, cascading.scheme.SinkCall<java.lang.Object[],org.apache.hadoop.mapred.OutputCollector> sinkCall)
protected void	sourceHandleInput(cascading.scheme.SourceCall<java.lang.Object[],org.apache.hadoop.mapred.RecordReader> sourceCall)

Methods inherited from class cascading.scheme.hadoop.TextLine

getCharsetName, getSinkCompression, makeEncodedString, presentSinkFields, presentSourceFields, setCharsetName, setSinkCompression, sinkConfInit, sinkPrepare, source, sourceCleanup, sourceConfInit, sourcePrepare, verify

Methods inherited from class cascading.scheme.Scheme

equals, getNumSinkParts, getSinkFields, getSourceFields, getTrace, hashCode, isSink, isSource, isSymmetrical, presentSinkFieldsInternal, presentSourceFieldsInternal, retrieveSinkFields, retrieveSourceFields, setNumSinkParts, setSinkFields, setSourceFields, sinkCleanup, sinkWrap, sourceRePrepare, sourceWrap, toString

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

DEFAULT_FIELDS

public static final cascading.tuple.Fields DEFAULT_FIELDS

Constructor Detail

JSONTextLine

public JSONTextLine()

Constructor JSONTextLine creates a new JSONTextLine instance for use with any of the Hadoop based FlowConnector instances returning results with the default field named "json".

JSONTextLine

public JSONTextLine(cascading.tuple.Fields fields)

Constructor JSONTextLine creates a new JSONTextLine instance for use with any of the Hadoop based FlowConnector instances.

Parameters:

fields - of Fields

JSONTextLine

public JSONTextLine(cascading.tuple.Fields fields,
                    java.lang.String charsetName)

Constructor JSONTextLine creates a new JSONTextLine instance for use with any of the Hadoop based FlowConnector instances.

Parameters:

fields - of Fields

charsetName - of String

JSONTextLine

public JSONTextLine(cascading.tuple.Fields fields,
                    cascading.scheme.hadoop.TextLine.Compress sinkCompression)

Constructor JSONTextLine creates a new JSONTextLine instance for use with any of the Hadoop based FlowConnector instances.

Parameters:

fields - of Fields

sinkCompression - of Compress

JSONTextLine

public JSONTextLine(cascading.tuple.Fields fields,
                    cascading.scheme.hadoop.TextLine.Compress sinkCompression,
                    java.lang.String charsetName)

Constructor JSONTextLine creates a new JSONTextLine instance for use with any of the Hadoop based FlowConnector instances.

Parameters:

fields - of Fields

sinkCompression - of Compress

charsetName - of String

JSONTextLine

public JSONTextLine(ObjectMapper mapper,
                    cascading.tuple.Fields fields)

Constructor JSONTextLine creates a new JSONTextLine instance for use with any of the Hadoop based FlowConnector instances.

Parameters:

mapper - of ObjectMapper

fields - of Fields

JSONTextLine

public JSONTextLine(ObjectMapper mapper,
                    cascading.tuple.Fields fields,
                    java.lang.String charsetName)

Constructor JSONTextLine creates a new JSONTextLine instance for use with any of the Hadoop based FlowConnector instances.

Parameters:

mapper - of ObjectMapper

fields - of Fields

charsetName - of String

JSONTextLine

public JSONTextLine(ObjectMapper mapper,
                    cascading.tuple.Fields fields,
                    cascading.scheme.hadoop.TextLine.Compress sinkCompression)

Constructor JSONTextLine creates a new JSONTextLine instance for use with any of the Hadoop based FlowConnector instances.

Parameters:

mapper - of ObjectMapper

fields - of Fields

sinkCompression - of Compress

JSONTextLine

public JSONTextLine(ObjectMapper mapper,
                    cascading.tuple.Fields fields,
                    cascading.scheme.hadoop.TextLine.Compress sinkCompression,
                    java.lang.String charsetName)

Constructor JSONTextLine creates a new JSONTextLine instance for use with any of the Hadoop based FlowConnector instances.

Parameters:

mapper - of ObjectMapper

fields - of Fields

sinkCompression - of Compress

charsetName - of String

Method Detail

sourceHandleInput

protected void sourceHandleInput(cascading.scheme.SourceCall<java.lang.Object[],org.apache.hadoop.mapred.RecordReader> sourceCall)
                          throws java.io.IOException

Overrides:

sourceHandleInput in class cascading.scheme.hadoop.TextLine

Throws:

java.io.IOException

sink

public void sink(cascading.flow.FlowProcess<? extends org.apache.hadoop.conf.Configuration> flowProcess,
                 cascading.scheme.SinkCall<java.lang.Object[],org.apache.hadoop.mapred.OutputCollector> sinkCall)
          throws java.io.IOException

Overrides:

sink in class cascading.scheme.hadoop.TextLine

Throws:

java.io.IOException

getExtension

public java.lang.String getExtension()

Specified by:

getExtension in interface cascading.scheme.FileFormat

Overrides:

getExtension in class cascading.scheme.hadoop.TextLine