cascading.flow.hadoop

## Class HadoopFlowProcess

- java.lang.Object
- - cascading.flow.FlowProcess<JobConf>
  - - cascading.flow.hadoop.HadoopFlowProcess

All Implemented Interfaces:
  MapRed

    public class HadoopFlowProcess
    extends FlowProcess<JobConf>
    implements MapRed

Class HadoopFlowProcess is an implementation of `FlowProcess` for Hadoop. Use this interface to get direct access to the Hadoop JobConf and Reporter interfaces.

Be warned that coupling to this implementation will cause custom `Operation`s to fail if they are executed on a system other than Hadoop.

See Also:
  `FlowSession`, `JobConf`, `Reporter`

### Nested Class Summary

### Nested classes/interfaces inherited from class cascading.flow.FlowProcess

`FlowProcess.NullFlowProcess`

### Field Summary

### Fields inherited from class cascading.flow.FlowProcess

`NULL`

### Constructor Summary

Constructors
Constructor and Description
`HadoopFlowProcess()`
`HadoopFlowProcess(Configuration jobConf)`
`HadoopFlowProcess(FlowSession flowSession, JobConf jobConf)`
`HadoopFlowProcess(FlowSession flowSession, JobConf jobConf, boolean isMapper)` Constructor HadoopFlowProcess creates a new HadoopFlowProcess instance.
`HadoopFlowProcess(HadoopFlowProcess flowProcess, JobConf jobConf)`
`HadoopFlowProcess(JobConf jobConf)`

### Method Summary

Methods
`<C> C` `copyConfig(C config)`
`FlowProcess` `copyWith(JobConf jobConf)`
`<C> java.util.Map<java.lang.String,java.lang.String>` `diffConfigIntoMap(C defaultConfig, C updatedConfig)`
`JobConf` `getConfig()`
`JobConf` `getConfigCopy()`
`long` `getCounterValue(java.lang.Enum counter)`
`long` `getCounterValue(java.lang.String group, java.lang.String counter)`
`int` `getCurrentNumMappers()`
`int` `getCurrentNumReducers()`
`int` `getCurrentSliceNum()` Method getCurrentTaskNum returns the task number of this task.
`JobConf` `getJobConf()` Method getJobConf returns the jobConf of this HadoopFlowProcess object.
`int` `getNumProcessSlices()`
`OutputCollector` `getOutputCollector()`
`java.lang.Object` `getProperty(java.lang.String key)`
`java.util.Collection<java.lang.String>` `getPropertyKeys()`
`Reporter` `getReporter()`
`void` `increment(java.lang.Enum counter, long amount)`
`void` `increment(java.lang.String group, java.lang.String counter, long amount)`
`boolean` `isCounterStatusInitialized()`
`boolean` `isMapper()` Method isMapper returns true if this part of the FlowProcess is a MapReduce mapper.
`void` `keepAlive()`
`JobConf` `mergeMapIntoConfig(JobConf defaultConfig, java.util.Map<java.lang.String,java.lang.String> map)`
`java.lang.Object` `newInstance(java.lang.String className)`
`TupleEntryCollector` `openSystemIntermediateForWrite()`
`TupleEntryIterator` `openTapForRead(Tap tap)`
`TupleEntryCollector` `openTapForWrite(Tap tap)`
`TupleEntryCollector` `openTrapForWrite(Tap trap)`
`void` `setOutputCollector(OutputCollector outputCollector)`
`void` `setReporter(Reporter reporter)` Method setReporter sets the reporter of this HadoopFlowProcess object.
`void` `setStatus(java.lang.String status)`

### Methods inherited from class cascading.flow.FlowProcess

`closeTrapCollectors, getBooleanProperty, getBooleanProperty, getCurrentSession, getID, getIntegerProperty, getIntegerProperty, getStringProperty, getStringProperty, getTrapCollectorFor, getTrapCollectors, setCurrentSession`

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Constructor Detail

#### HadoopFlowProcess

    public HadoopFlowProcess()

#### HadoopFlowProcess

    public HadoopFlowProcess(Configuration jobConf)

#### HadoopFlowProcess

    public HadoopFlowProcess(JobConf jobConf)

#### HadoopFlowProcess

    public HadoopFlowProcess(FlowSession flowSession,
                     JobConf jobConf)

#### HadoopFlowProcess

    public HadoopFlowProcess(FlowSession flowSession,
                     JobConf jobConf,
                     boolean isMapper)

Constructor HadoopFlowProcess creates a new HadoopFlowProcess instance.

Parameters:
  `flowSession` - of type FlowSession
  `jobConf` - of type JobConf

#### HadoopFlowProcess

    public HadoopFlowProcess(HadoopFlowProcess flowProcess,
                     JobConf jobConf)

### Method Detail

#### copyWith

    public FlowProcess copyWith(JobConf jobConf)

Specified by:
  `copyWith` in class `FlowProcess<JobConf>`

#### getJobConf

    public JobConf getJobConf()

Method getJobConf returns the jobConf of this HadoopFlowProcess object.

Returns:
  the jobConf (type JobConf) of this HadoopFlowProcess object.

#### getConfig

    public JobConf getConfig()

Specified by:
  `getConfig` in class `FlowProcess<JobConf>`

#### getConfigCopy

    public JobConf getConfigCopy()

Specified by:
  `getConfigCopy` in class `FlowProcess<JobConf>`

#### isMapper

    public boolean isMapper()

Method isMapper returns true if this part of the FlowProcess is a MapReduce mapper. If false, it is a reducer.

Returns:
  boolean

#### getCurrentNumMappers

    public int getCurrentNumMappers()

#### getCurrentNumReducers

    public int getCurrentNumReducers()

#### getCurrentSliceNum

    public int getCurrentSliceNum()

Method getCurrentTaskNum returns the task number of this task. Task 0 is the first task.

Specified by:
  `getCurrentSliceNum` in class `FlowProcess<JobConf>`
Returns:
  int

#### getNumProcessSlices

    public int getNumProcessSlices()

Specified by:
  `getNumProcessSlices` in class `FlowProcess<JobConf>`

#### setReporter

    public void setReporter(Reporter reporter)

Method setReporter sets the reporter of this HadoopFlowProcess object.

Parameters:
  `reporter` - the reporter of this HadoopFlowProcess object.

#### getReporter

    public Reporter getReporter()

Specified by:
  `getReporter` in interface `MapRed`

#### setOutputCollector

    public void setOutputCollector(OutputCollector outputCollector)

#### getOutputCollector

    public OutputCollector getOutputCollector()

#### getProperty

    public java.lang.Object getProperty(java.lang.String key)

Specified by:
  `getProperty` in class `FlowProcess<JobConf>`

#### getPropertyKeys

    public java.util.Collection<java.lang.String> getPropertyKeys()

Specified by:
  `getPropertyKeys` in class `FlowProcess<JobConf>`

#### newInstance

    public java.lang.Object newInstance(java.lang.String className)

Specified by:
  `newInstance` in class `FlowProcess<JobConf>`

#### keepAlive

    public void keepAlive()

Specified by:
  `keepAlive` in class `FlowProcess<JobConf>`

#### increment

    public void increment(java.lang.Enum counter,
                 long amount)

Specified by:
  `increment` in class `FlowProcess<JobConf>`

#### increment

    public void increment(java.lang.String group,
                 java.lang.String counter,
                 long amount)

Specified by:
  `increment` in class `FlowProcess<JobConf>`

#### getCounterValue

    public long getCounterValue(java.lang.Enum counter)

Specified by:
  `getCounterValue` in class `FlowProcess<JobConf>`

#### getCounterValue

    public long getCounterValue(java.lang.String group,
                       java.lang.String counter)

Specified by:
  `getCounterValue` in class `FlowProcess<JobConf>`

#### setStatus

    public void setStatus(java.lang.String status)

Specified by:
  `setStatus` in class `FlowProcess<JobConf>`

#### isCounterStatusInitialized

    public boolean isCounterStatusInitialized()

Specified by:
  `isCounterStatusInitialized` in class `FlowProcess<JobConf>`

#### openTapForRead

    public TupleEntryIterator openTapForRead(Tap tap)
                                      throws java.io.IOException

Specified by:
  `openTapForRead` in class `FlowProcess<JobConf>`
Throws:
  `java.io.IOException`

#### openTapForWrite

    public TupleEntryCollector openTapForWrite(Tap tap)
                                        throws java.io.IOException

Specified by:
  `openTapForWrite` in class `FlowProcess<JobConf>`
Throws:
  `java.io.IOException`

#### openTrapForWrite

    public TupleEntryCollector openTrapForWrite(Tap trap)
                                         throws java.io.IOException

Specified by:
  `openTrapForWrite` in class `FlowProcess<JobConf>`
Throws:
  `java.io.IOException`

#### openSystemIntermediateForWrite

    public TupleEntryCollector openSystemIntermediateForWrite()
                                                       throws java.io.IOException

Specified by:
  `openSystemIntermediateForWrite` in class `FlowProcess<JobConf>`
Throws:
  `java.io.IOException`

#### copyConfig

    public <C> C copyConfig(C config)

Specified by:
  `copyConfig` in class `FlowProcess<JobConf>`

#### diffConfigIntoMap

    public <C> java.util.Map<java.lang.String,java.lang.String> diffConfigIntoMap(C defaultConfig,
                                                                         C updatedConfig)

Specified by:
  `diffConfigIntoMap` in class `FlowProcess<JobConf>`

#### mergeMapIntoConfig

    public JobConf mergeMapIntoConfig(JobConf defaultConfig,
                             java.util.Map<java.lang.String,java.lang.String> map)

Specified by:
  `mergeMapIntoConfig` in class `FlowProcess<JobConf>`