cascading.flow.hadoop3

Class Hadoop3MRFlowConnector

java.lang.Object

cascading.flow.FlowConnector

cascading.flow.hadoop3.Hadoop3MRFlowConnector

public class Hadoop3MRFlowConnector
extends cascading.flow.FlowConnector

Use the Hadoop3MRFlowConnector to link source and sink Tap instances with an assembly of Pipe instances into an executable HadoopFlow for execution on an Apache Hadoop cluster.

See Also:

AppProps, FlowConnectorProps, FlowDef, MapReduceFlow

Field Summary

Fields inherited from class cascading.flow.FlowConnector

properties

Constructor Summary

Constructors

Constructor and Description
Hadoop3MRFlowConnector() Constructor FlowConnector creates a new FlowConnector instance.
Hadoop3MRFlowConnector(java.util.Map<java.lang.Object,java.lang.Object> properties) Constructor FlowConnector creates a new FlowConnector instance using the given Properties instance as default value for the underlying jobs.
Hadoop3MRFlowConnector(java.util.Map<java.lang.Object,java.lang.Object> properties, cascading.flow.planner.rule.RuleRegistrySet ruleRegistrySet) Constructor HadoopFlowConnector creates a new HadoopFlowConnector instance using the given Properties instance as default value for the underlying jobs.
Hadoop3MRFlowConnector(cascading.flow.planner.rule.RuleRegistrySet ruleRegistrySet) Constructor HadoopFlowConnector creates a new HadoopFlowConnector instance.

Method Summary

Modifier and Type	Method and Description
protected cascading.flow.planner.rule.RuleRegistrySet	createDefaultRuleRegistrySet()
protected cascading.flow.planner.FlowPlanner	createFlowPlanner()
protected java.lang.Class<? extends cascading.scheme.Scheme>	getDefaultIntermediateSchemeClass()

Methods inherited from class cascading.flow.FlowConnector

connect, connect, connect, connect, connect, connect, connect, connect, connect, connect, connect, connect, connect, connect, connect, getIntermediateSchemeClass, getPlatformInfo, getProperties, getRuleRegistrySet

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Hadoop3MRFlowConnector

public Hadoop3MRFlowConnector()

Constructor FlowConnector creates a new FlowConnector instance.

All properties passed to Hadoop are retrieved from a default instantiation of the Hadoop JobConf which pulls all properties from the local CLASSPATH.

Hadoop3MRFlowConnector

@ConstructorProperties(value="properties")
public Hadoop3MRFlowConnector(java.util.Map<java.lang.Object,java.lang.Object> properties)

Constructor FlowConnector creates a new FlowConnector instance using the given Properties instance as default value for the underlying jobs. All properties are copied to a new native configuration instance.

Parameters:

properties - of type Properties

Hadoop3MRFlowConnector

@ConstructorProperties(value="ruleRegistrySet")
public Hadoop3MRFlowConnector(cascading.flow.planner.rule.RuleRegistrySet ruleRegistrySet)

Constructor HadoopFlowConnector creates a new HadoopFlowConnector instance.

All properties passed to Hadoop are retrieved from a default instantiation of the Hadoop JobConf which pulls all properties from the local CLASSPATH.

Parameters:

ruleRegistrySet - of type RuleRegistry

Hadoop3MRFlowConnector

@ConstructorProperties(value={"properties","ruleRegistrySet"})
public Hadoop3MRFlowConnector(java.util.Map<java.lang.Object,java.lang.Object> properties,
                                                                                             cascading.flow.planner.rule.RuleRegistrySet ruleRegistrySet)

Constructor HadoopFlowConnector creates a new HadoopFlowConnector instance using the given Properties instance as default value for the underlying jobs. All properties are copied to a new native configuration instance.

Parameters:

properties - of type Map

ruleRegistrySet - of type RuleRegistry

Method Detail

getDefaultIntermediateSchemeClass

protected java.lang.Class<? extends cascading.scheme.Scheme> getDefaultIntermediateSchemeClass()

Specified by:

getDefaultIntermediateSchemeClass in class cascading.flow.FlowConnector

createFlowPlanner

protected cascading.flow.planner.FlowPlanner createFlowPlanner()

Specified by:

createFlowPlanner in class cascading.flow.FlowConnector

createDefaultRuleRegistrySet

protected cascading.flow.planner.rule.RuleRegistrySet createDefaultRuleRegistrySet()

Specified by:

createDefaultRuleRegistrySet in class cascading.flow.FlowConnector