cascading.flow.hadoop.planner

## Class HadoopPlanner

- java.lang.Object
- - cascading.flow.planner.FlowPlanner<HadoopFlow,JobConf>
  - - cascading.flow.hadoop.planner.HadoopPlanner

    public class HadoopPlanner
    extends FlowPlanner<HadoopFlow,JobConf>

Class HadoopPlanner is the core Hadoop MapReduce planner used by default through a `FlowConnector` sub-class.

Notes:

Custom JobConf properties
A custom JobConf instance can be passed to this planner by calling `copyJobConf(java.util.Map, org.apache.hadoop.mapred.JobConf)` on a map properties object before constructing a new `FlowConnector` sub-class.

A better practice would be to set Hadoop properties directly on the map properties object handed to the FlowConnector. All values in the map will be passed to a new default JobConf instance to be used as defaults for all resulting Flow instances.

For example, `properties.set("mapred.child.java.opts","-Xmx512m");` would convince Hadoop to spawn all child jvms with a heap of 512MB.

### Nested Class Summary

### Nested classes/interfaces inherited from class cascading.flow.planner.FlowPlanner

`FlowPlanner.IdentityElementFactgory, FlowPlanner.TempTapElementFactory`

### Field Summary

Fields
`static java.lang.String` `PLATFORM_NAME`

### Fields inherited from class cascading.flow.planner.FlowPlanner

`checkpointTapRootPath, defaultAssertionLevel, defaultDebugLevel, defaultProperties, TRACE_PLAN_PATH, TRACE_PLAN_TRANSFORM_PATH, TRACE_STATS_PATH`

### Constructor Summary

Constructors
Constructor and Description
`HadoopPlanner()`

### Method Summary

Methods
`protected void` `checkPlatform(Configuration conf)`
`void` `configRuleRegistryDefaults(RuleRegistry ruleRegistry)`
`static void` `copyJobConf(java.util.Map<java.lang.Object,java.lang.Object> properties, JobConf jobConf)` Method copyJobConf adds the given JobConf values to the given properties object.
`static void` `copyProperties(JobConf jobConf, java.util.Map<java.lang.Object,java.lang.Object> properties)` Method copyProperties adds the given Map values to the given JobConf object.
`protected HadoopFlow` `createFlow(FlowDef flowDef)`
`static JobConf` `createJobConf(java.util.Map<java.lang.Object,java.lang.Object> properties)` Method createJobConf returns a new JobConf instance using the values in the given properties argument.
`JobConf` `getDefaultConfig()`
`java.net.URI` `getDefaultURIScheme(Tap tap)`
`FlowStepFactory<JobConf>` `getFlowStepFactory()`
`PlannerInfo` `getPlannerInfo(java.lang.String registryName)`
`PlatformInfo` `getPlatformInfo()`
`java.net.URI` `getURIScheme(Tap tap)`
`void` `initialize(FlowConnector flowConnector, java.util.Map<java.lang.Object,java.lang.Object> properties)`
`protected Tap` `makeTempTap(java.lang.String prefix, java.lang.String name)`

### Methods inherited from class cascading.flow.planner.FlowPlanner

`buildFlow, createFlowElementGraph, getAssertionLevel, getDebugLevel, getDefaultProperties, getFlowNodeFactory, handleExceptionDuringPlanning, makeTempTap, resolveAssemblyPlanners, resolveTails, verifyAllTaps, verifyAssembly, verifyCheckpoints, verifyPipeAssemblyEndPoints, verifyResult, verifyResultInternal, verifySourceNotSinks, verifyTaps, verifyTraps`

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Field Detail

#### PLATFORM_NAME

    public static final java.lang.String PLATFORM_NAME

See Also:
  Constant Field Values

### Constructor Detail

#### HadoopPlanner

    public HadoopPlanner()

### Method Detail

#### copyJobConf

    public static void copyJobConf(java.util.Map<java.lang.Object,java.lang.Object> properties,
                   JobConf jobConf)

Method copyJobConf adds the given JobConf values to the given properties object. Use this method to pass custom default Hadoop JobConf properties to Hadoop.

Parameters:
  `properties` - of type Map
  `jobConf` - of type JobConf

#### createJobConf

    public static JobConf createJobConf(java.util.Map<java.lang.Object,java.lang.Object> properties)

Method createJobConf returns a new JobConf instance using the values in the given properties argument.

Parameters:
  `properties` - of type Map
Returns:
  a JobConf instance

#### copyProperties

    public static void copyProperties(JobConf jobConf,
                      java.util.Map<java.lang.Object,java.lang.Object> properties)

Method copyProperties adds the given Map values to the given JobConf object.

Parameters:
  `jobConf` - of type JobConf
  `properties` - of type Map

#### getPlannerInfo

    public PlannerInfo getPlannerInfo(java.lang.String registryName)

Specified by:
  `getPlannerInfo` in class `FlowPlanner<HadoopFlow,JobConf>`

#### getDefaultConfig

    public JobConf getDefaultConfig()

Specified by:
  `getDefaultConfig` in class `FlowPlanner<HadoopFlow,JobConf>`

#### getPlatformInfo

    public PlatformInfo getPlatformInfo()

Specified by:
  `getPlatformInfo` in class `FlowPlanner<HadoopFlow,JobConf>`

#### initialize

    public void initialize(FlowConnector flowConnector,
                  java.util.Map<java.lang.Object,java.lang.Object> properties)

Overrides:
  `initialize` in class `FlowPlanner<HadoopFlow,JobConf>`

#### configRuleRegistryDefaults

    public void configRuleRegistryDefaults(RuleRegistry ruleRegistry)

Overrides:
  `configRuleRegistryDefaults` in class `FlowPlanner<HadoopFlow,JobConf>`

#### checkPlatform

    protected void checkPlatform(Configuration conf)

#### createFlow

    protected HadoopFlow createFlow(FlowDef flowDef)

Specified by:
  `createFlow` in class `FlowPlanner<HadoopFlow,JobConf>`

#### getFlowStepFactory

    public FlowStepFactory<JobConf> getFlowStepFactory()

Specified by:
  `getFlowStepFactory` in class `FlowPlanner<HadoopFlow,JobConf>`

#### getDefaultURIScheme

    public java.net.URI getDefaultURIScheme(Tap tap)

#### getURIScheme

    public java.net.URI getURIScheme(Tap tap)

#### makeTempTap

    protected Tap makeTempTap(java.lang.String prefix,
                  java.lang.String name)

Specified by:
  `makeTempTap` in class `FlowPlanner<HadoopFlow,JobConf>`