cascading.tap.hadoop

Class Dfs

java.lang.Object

cascading.tap.Tap<Configuration,RecordReader,OutputCollector>

cascading.tap.hadoop.Hfs

cascading.tap.hadoop.Dfs

All Implemented Interfaces:

cascading.flow.FlowElement, cascading.flow.planner.ScopedElement, cascading.tap.type.FileType<Configuration>, cascading.tap.type.TapWith<Configuration,RecordReader,OutputCollector>, cascading.util.Traceable, java.io.Serializable

public class Dfs
extends Hfs

Class Dfs is a Tap class that provides access to the Hadoop Distributed File System.

Use the URI constructors to specify a different HDFS cluster than the default.

See Also:

Serialized Form

Field Summary

Fields inherited from class cascading.tap.hadoop.Hfs

stringPath

Fields inherited from interface cascading.tap.type.FileType

CASCADING_SOURCE_PATH

Constructor Summary

Constructors

Constructor and Description
Dfs(cascading.scheme.Scheme scheme, java.lang.String stringPath) Constructor Dfs creates a new Dfs instance.
Dfs(cascading.scheme.Scheme scheme, java.lang.String stringPath, cascading.tap.SinkMode sinkMode) Constructor Dfs creates a new Dfs instance.
Dfs(cascading.scheme.Scheme scheme, java.net.URI uri) Constructor Dfs creates a new Dfs instance.
Dfs(cascading.scheme.Scheme scheme, java.net.URI uri, cascading.tap.SinkMode sinkMode) Constructor Dfs creates a new Dfs instance.

Method Summary

Modifier and Type	Method and Description
protected FileSystem	getDefaultFileSystem(Configuration configuration)
protected void	setStringPath(java.lang.String stringPath)

Methods inherited from class cascading.tap.hadoop.Hfs

applySourceConfInitIdentifiers, create, createResource, deleteChildResource, deleteChildResource, deleteResource, getBlockSize, getBlockSize, getChildIdentifiers, getChildIdentifiers, getChildIdentifiers, getChildIdentifiers, getCombinedInputSafeMode, getDeclaredInputPaths, getDefaultFileSystemURIScheme, getFileStatus, getFileSystem, getFullIdentifier, getIdentifier, getLocalModeScheme, getModifiedTime, getPath, getReplication, getReplication, getSize, getSize, getTempPath, getURIScheme, getUseCombinedInput, isDirectory, isDirectory, makeTemporaryPathDirString, makeURIScheme, openForRead, openForWrite, resetFileStatuses, resourceExists, setUriScheme, sinkConfInit, sourceConfInit, sourceConfInitAddInputPath, sourceConfInitAddInputPaths, sourceConfInitComplete, verifyNoDuplicates, withChildIdentifier, withScheme, withSinkMode

Methods inherited from class cascading.tap.Tap

commitResource, entryStream, entryStream, entryStreamCopy, entryStreamCopy, equals, flowConfInit, getConfigDef, getNodeConfigDef, getScheme, getSinkFields, getSinkMode, getSourceFields, getStepConfigDef, getTrace, hasConfigDef, hashCode, hasNodeConfigDef, hasStepConfigDef, id, isKeep, isReplace, isSink, isSource, isTemporary, isUpdate, openForRead, openForReadUnchecked, openForWrite, outgoingScopeFor, prepareResourceForRead, prepareResourceForWrite, presentSinkFields, presentSourceFields, resolveIncomingOperationArgumentFields, resolveIncomingOperationPassThroughFields, retrieveSinkFields, retrieveSourceFields, rollbackResource, setScheme, spliterator, splititerator, taps, toString, tupleStream, tupleStream, tupleStreamCopy, tupleStreamCopy

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Methods inherited from interface cascading.tap.type.TapWith

asTap

Constructor Detail

Dfs

@ConstructorProperties(value={"scheme","uri"})
public Dfs(cascading.scheme.Scheme scheme,
                                                          java.net.URI uri)

Constructor Dfs creates a new Dfs instance.

Parameters:

scheme - of type Scheme

uri - of type URI

Dfs

@ConstructorProperties(value={"scheme","uri","sinkMode"})
public Dfs(cascading.scheme.Scheme scheme,
                                                                     java.net.URI uri,
                                                                     cascading.tap.SinkMode sinkMode)

Constructor Dfs creates a new Dfs instance.

Parameters:

scheme - of type Scheme

uri - of type URI

sinkMode - of type SinkMode

Dfs

@ConstructorProperties(value={"scheme","stringPath"})
public Dfs(cascading.scheme.Scheme scheme,
                                                                 java.lang.String stringPath)

Constructor Dfs creates a new Dfs instance.

Parameters:

scheme - of type Scheme

stringPath - of type String

Dfs

@ConstructorProperties(value={"scheme","stringPath","sinkMode"})
public Dfs(cascading.scheme.Scheme scheme,
                                                                            java.lang.String stringPath,
                                                                            cascading.tap.SinkMode sinkMode)

Constructor Dfs creates a new Dfs instance.

Parameters:

scheme - of type Scheme

stringPath - of type String

sinkMode - of type SinkMode

Method Detail

setStringPath

protected void setStringPath(java.lang.String stringPath)

Overrides:

setStringPath in class Hfs

getDefaultFileSystem

protected FileSystem getDefaultFileSystem(Configuration configuration)

Overrides:

getDefaultFileSystem in class Hfs