cascading.flow.hadoop

## Class HadoopFlowStep

- java.lang.Object
- - cascading.flow.planner.BaseFlowStep<org.apache.hadoop.mapred.JobConf>
  - - cascading.flow.hadoop.HadoopFlowStep

- All Implemented Interfaces:
    FlowStep<org.apache.hadoop.mapred.JobConf>, ProcessModel, ProcessLogger, java.io.Serializable
  Direct Known Subclasses:
    MapReduceFlowStep

      public class HadoopFlowStep
      extends BaseFlowStep<org.apache.hadoop.mapred.JobConf>

  See Also:
    Serialized Form

- - ### Field Summary

    - ### Fields inherited from class cascading.flow.planner.BaseFlowStep

      `elementGraph, flowNodeGraph, flowStepStats, sinks, sources, tempSink`

    - ### Fields inherited from interface cascading.flow.FlowStep

      `CASCADING_FLOW_STEP_ID`

    - ### Fields inherited from interface cascading.util.ProcessLogger

      `NULL`

  - ### Constructor Summary

    Constructors
    Modifier | Constructor and Description
    --- | ---
    `protected` | `HadoopFlowStep()`
    `` | `HadoopFlowStep(ElementGraph elementGraph, FlowNodeGraph flowNodeGraph)`
    `protected` | `HadoopFlowStep(java.lang.String name, int ordinal)`

  - ### Method Summary

    Methods
    Modifier and Type | Method and Description
    --- | ---
    `void` | `clean(org.apache.hadoop.mapred.JobConf config)` Method clean removes any temporary files used by this FlowStep instance.
    `protected void` | `cleanIntermediateData(org.apache.hadoop.mapred.JobConf config, Tap sink)`
    `protected FlowStepJob<org.apache.hadoop.mapred.JobConf>` | `createFlowStepJob(ClientState clientState, FlowProcess<org.apache.hadoop.mapred.JobConf> flowProcess, org.apache.hadoop.mapred.JobConf initializedStepConfig)`
    `org.apache.hadoop.mapred.JobConf` | `createInitializedConfig(FlowProcess<org.apache.hadoop.mapred.JobConf> flowProcess, org.apache.hadoop.mapred.JobConf parentConfig)` Public for testing.
    `java.util.Map<java.lang.Object,java.lang.Object>` | `getConfigAsProperties()` Method getConfigAsProperties converts the internal configuration object into a `Map` of key value pairs.
    `protected void` | `initFromSink(FlowProcess<org.apache.hadoop.mapred.JobConf> flowProcess, org.apache.hadoop.mapred.JobConf conf)`
    `protected void` | `initFromSources(FlowProcess<org.apache.hadoop.mapred.JobConf> flowProcess, org.apache.hadoop.mapred.JobConf conf)`
    `protected void` | `initFromTraps(FlowProcess<org.apache.hadoop.mapred.JobConf> flowProcess, org.apache.hadoop.mapred.JobConf conf)`
    `boolean` | `isHadoopLocalMode(org.apache.hadoop.mapred.JobConf conf)`

    - ### Methods inherited from class cascading.flow.planner.BaseFlowStep

      `addAllGroups, addGroup, addGroups, addListener, addProcessAnnotation, addProcessAnnotation, addSink, addSinks, addSource, addSources, clean, commitSinks, configure, containsPipeNamed, createClientState, equals, fireOnCompleted, fireOnRunning, fireOnStarting, fireOnStopping, fireOnThrowable, getAllAccumulatedSources, getAllOperations, getAllStreamedSources, getAnnotations, getConfig, getCreateFlowStepJob, getElementGraph, getFlow, getFlowID, getFlowName, getFlowNodeGraph, getFlowStepDescriptor, getFlowStepJob, getFlowStepStats, getGroup, getGroups, getID, getName, getNextFlowElement, getNextScope, getNodeDisplayName, getNumFlowNodes, getOrdinal, getPreviousScopes, getProcessAnnotations, getSink, getSinkElements, getSinkName, getSinkTaps, getSinkWith, getSourceElements, getSourceName, getSourceTaps, getSourceWith, getStepDisplayName, getStepDisplayName, getSubmitPriority, getTrap, getTrapMap, getTraps, hashCode, hasListeners, initConfFromNodeConfigDef, initConfFromStepConfigDef, isDebugEnabled, isInfoEnabled, logDebug, logError, logError, logInfo, logWarn, logWarn, logWarn, prepareResources, removeListener, rollbackSinks, setConfig, setFlow, setFlowName, setFlowStepDescriptor, setFlowStepStats, setName, setOrdinal, setSubmitPriority, toString`

    - ### Methods inherited from class java.lang.Object

      `clone, finalize, getClass, notify, notifyAll, wait, wait, wait`

- - ### Constructor Detail

    - #### HadoopFlowStep

          protected HadoopFlowStep()

    - #### HadoopFlowStep

          protected HadoopFlowStep(java.lang.String name,
                        int ordinal)

    - #### HadoopFlowStep

          public HadoopFlowStep(ElementGraph elementGraph,
                        FlowNodeGraph flowNodeGraph)

  - ### Method Detail

    - #### getConfigAsProperties

          public java.util.Map<java.lang.Object,java.lang.Object> getConfigAsProperties()

      Description copied from interface: `FlowStep`
      Method getConfigAsProperties converts the internal configuration object into a `Map` of key value pairs.
      Specified by:
        `getConfigAsProperties` in interface `FlowStep<org.apache.hadoop.mapred.JobConf>`
      Overrides:
        `getConfigAsProperties` in class `BaseFlowStep<org.apache.hadoop.mapred.JobConf>`
      Returns:
        a Map of key/value pairs, may return an empty collection if unsupported

    - #### createInitializedConfig

          public org.apache.hadoop.mapred.JobConf createInitializedConfig(FlowProcess<org.apache.hadoop.mapred.JobConf> flowProcess,
                                                                 org.apache.hadoop.mapred.JobConf parentConfig)

      Description copied from class: `BaseFlowStep`
      Public for testing.
      Specified by:
        `createInitializedConfig` in class `BaseFlowStep<org.apache.hadoop.mapred.JobConf>`
      Returns:

    - #### isHadoopLocalMode

          public boolean isHadoopLocalMode(org.apache.hadoop.mapred.JobConf conf)

    - #### createFlowStepJob

          protected FlowStepJob<org.apache.hadoop.mapred.JobConf> createFlowStepJob(ClientState clientState,
                                                                        FlowProcess<org.apache.hadoop.mapred.JobConf> flowProcess,
                                                                        org.apache.hadoop.mapred.JobConf initializedStepConfig)

      Specified by:
        `createFlowStepJob` in class `BaseFlowStep<org.apache.hadoop.mapred.JobConf>`

    - #### clean

          public void clean(org.apache.hadoop.mapred.JobConf config)

      Method clean removes any temporary files used by this FlowStep instance. It will log any IOExceptions thrown.
      Specified by:
        `clean` in class `BaseFlowStep<org.apache.hadoop.mapred.JobConf>`
      Parameters:
        `config` - of type JobConf

    - #### cleanIntermediateData

          protected void cleanIntermediateData(org.apache.hadoop.mapred.JobConf config,
                                   Tap sink)

    - #### initFromSources

          protected void initFromSources(FlowProcess<org.apache.hadoop.mapred.JobConf> flowProcess,
                             org.apache.hadoop.mapred.JobConf conf)

    - #### initFromSink

          protected void initFromSink(FlowProcess<org.apache.hadoop.mapred.JobConf> flowProcess,
                          org.apache.hadoop.mapred.JobConf conf)

    - #### initFromTraps

          protected void initFromTraps(FlowProcess<org.apache.hadoop.mapred.JobConf> flowProcess,
                           org.apache.hadoop.mapred.JobConf conf)

Copyright © 2007-2015 Xplenty, Inc. All Rights Reserved.