cascading.flow.hadoop

## Class BaseMapReduceFlow

- java.lang.Object
- - cascading.flow.BaseFlow<org.apache.hadoop.mapred.JobConf>
  - - cascading.flow.hadoop.HadoopFlow
    - - cascading.flow.hadoop.BaseMapReduceFlow

- All Implemented Interfaces:
    Flow<org.apache.hadoop.mapred.JobConf>, UnitOfWork<FlowStats>, ProcessLogger
  Direct Known Subclasses:
    MapReduceFlow, MultiMapReduceFlow

      public class BaseMapReduceFlow
      extends HadoopFlow

- - ### Nested Class Summary

    - ### Nested classes/interfaces inherited from class cascading.flow.BaseFlow

      `BaseFlow.FlowHolder`

  - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `protected boolean` | `deleteSinkOnInit` Field deleteSinkOnInit

    - ### Fields inherited from class cascading.flow.BaseFlow

      `completed, flowCanonicalHash, flowDescriptor, flowElementGraph, flowStats, flowStepGraph, platformInfo, sinks, sources, steps, stop, stopJobsOnExit, thread, throwable`

    - ### Fields inherited from interface cascading.flow.Flow

      `CASCADING_FLOW_ID`

    - ### Fields inherited from interface cascading.util.ProcessLogger

      `NULL`

  - ### Constructor Summary

    Constructors
    Modifier | Constructor and Description
    --- | ---
    `protected` | `BaseMapReduceFlow(PlatformInfo platformInfo, java.util.Map<java.lang.Object,java.lang.Object> properties, org.apache.hadoop.mapred.JobConf jobConf, java.lang.String name, java.util.Map<java.lang.String,java.lang.String> flowDescriptor, boolean deleteSinkOnInit)`
    `protected` | `BaseMapReduceFlow(PlatformInfo platformInfo, java.util.Map<java.lang.Object,java.lang.Object> properties, java.lang.String name, java.util.Map<java.lang.String,java.lang.String> flowDescriptor, boolean deleteSinkOnInit)`

  - ### Method Summary

    Methods
    Modifier and Type | Method and Description
    --- | ---
    `protected FlowStep<org.apache.hadoop.mapred.JobConf>` | `createFlowStep(org.apache.hadoop.mapred.JobConf jobConf, Tap sink)`
    `protected java.util.Map<java.lang.String,Tap>` | `createSinks(org.apache.hadoop.mapred.JobConf jobConf)`
    `protected java.util.Map<java.lang.String,Tap>` | `createSources(org.apache.hadoop.mapred.JobConf jobConf)`
    `protected Tap` | `createTap(org.apache.hadoop.mapred.JobConf jobConf, org.apache.hadoop.fs.Path path, SinkMode sinkMode)`
    `protected java.util.Map<java.lang.String,Tap>` | `createTraps(org.apache.hadoop.mapred.JobConf jobConf)`
    `protected java.util.Map<java.lang.String,Tap>` | `fileInputToTaps(org.apache.hadoop.mapred.JobConf jobConf)`
    `protected java.util.Map<java.lang.String,Tap>` | `fileOutputToTaps(org.apache.hadoop.mapred.JobConf jobConf)`
    `protected java.lang.String` | `makeNameFromPath(java.util.Map<java.lang.String,Tap> taps, org.apache.hadoop.fs.Path path)`
    `protected FlowStepGraph` | `makeStepGraph(org.apache.hadoop.mapred.JobConf jobConf)`
    `protected Tap` | `toSinkTap(org.apache.hadoop.mapred.JobConf jobConf, java.util.Map<java.lang.String,Tap> taps, org.apache.hadoop.fs.Path path)`
    `protected Tap` | `toSourceTap(org.apache.hadoop.mapred.JobConf jobConf, java.util.Map<java.lang.String,Tap> taps, org.apache.hadoop.fs.Path path)`

    - ### Methods inherited from class cascading.flow.hadoop.HadoopFlow

      `copyToDistributedCache, getConfig, getConfigAsProperties, getConfigCopy, getFlowProcess, getMaxNumParallelSteps, getProperty, getTotalSliceCPUMilliSeconds, initConfig, initFromProperties, internalClean, internalShutdown, internalStart, isPreserveTemporaryFiles, newConfig, registerHadoopShutdownHook, setConfigProperty, stepsAreLocal`

    - ### Methods inherited from class cascading.flow.BaseFlow

      `addListener, addPlannerProperties, addSessionProperties, addStepListener, areSinksStale, areSourcesNewer, cleanup, complete, createConfig, createFlowCanonicalHash, createFlowStats, createFlowThread, createPrepareFlowStats, deleteCheckpointsIfNotUpdate, deleteCheckpointsIfReplace, deleteSinks, deleteSinksIfNotUpdate, deleteSinksIfReplace, deleteTrapsIfNotUpdate, deleteTrapsIfReplace, fireOnCompleted, fireOnStarting, fireOnStopping, fireOnThrowable, fireOnThrowable, getCascadeID, getCascadingServices, getCheckpointNames, getCheckpoints, getCheckpointsCollection, getClassPath, getClientState, getEligibleJobsSize, getFieldsFor, getFlowCanonicalHash, getFlowDescriptor, getFlowElementGraph, getFlowSession, getFlowSkipStrategy, getFlowStats, getFlowStepGraph, getFlowSteps, getFlowStepStrategy, getHolder, getID, getJobMapCallables, getName, getPlannerInfo, getPlatformInfo, getRunID, getSink, getSink, getSinkModified, getSinkNames, getSinks, getSinksCollection, getSource, getSourceNames, getSources, getSourcesCollection, getSpawnStrategy, getStats, getSubmitPriority, getTags, getTrapNames, getTraps, getTrapsCollection, handleExecutorShutdown, hasListeners, hasStepListeners, initialize, initializeChildStats, initializeNewJobsMap, initSteps, internalStopAllJobs, isDebugEnabled, isInfoEnabled, isJobsMapInitialized, isSkipFlow, isStopJobsOnExit, logDebug, logError, logError, logInfo, logWarn, logWarn, logWarn, openSink, openSink, openSource, openSource, openTapForRead, openTapForWrite, openTrap, openTrap, prepare, presentSinkFields, presentSourceFields, registerShutdownHook, removeListener, removeListeners, removeStepListener, resourceExists, retrieveSinkFields, retrieveSourceFields, setCascade, setCheckpoints, setFlowElementGraph, setFlowSkipStrategy, setFlowStepGraph, setFlowStepStrategy, setName, setPlannerInfo, setSinks, setSources, setSpawnStrategy, setSubmitPriority, setTraps, spawnSteps, start, stop, toString, updateJobsMap, updateSchemes, writeDOT, writeStepsDOT`

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait`

- - ### Field Detail

    - #### deleteSinkOnInit

          protected boolean deleteSinkOnInit

      Field deleteSinkOnInit

  - ### Constructor Detail

    - #### BaseMapReduceFlow

          protected BaseMapReduceFlow(PlatformInfo platformInfo,
                           java.util.Map<java.lang.Object,java.lang.Object> properties,
                           org.apache.hadoop.mapred.JobConf jobConf,
                           java.lang.String name,
                           java.util.Map<java.lang.String,java.lang.String> flowDescriptor,
                           boolean deleteSinkOnInit)

    - #### BaseMapReduceFlow

          protected BaseMapReduceFlow(PlatformInfo platformInfo,
                           java.util.Map<java.lang.Object,java.lang.Object> properties,
                           java.lang.String name,
                           java.util.Map<java.lang.String,java.lang.String> flowDescriptor,
                           boolean deleteSinkOnInit)

  - ### Method Detail

    - #### makeStepGraph

          protected FlowStepGraph makeStepGraph(org.apache.hadoop.mapred.JobConf jobConf)

    - #### createFlowStep

          protected FlowStep<org.apache.hadoop.mapred.JobConf> createFlowStep(org.apache.hadoop.mapred.JobConf jobConf,
                                                                  Tap sink)

    - #### createSources

          protected java.util.Map<java.lang.String,Tap> createSources(org.apache.hadoop.mapred.JobConf jobConf)

    - #### fileInputToTaps

          protected java.util.Map<java.lang.String,Tap> fileInputToTaps(org.apache.hadoop.mapred.JobConf jobConf)

    - #### toSourceTap

          protected Tap toSourceTap(org.apache.hadoop.mapred.JobConf jobConf,
                        java.util.Map<java.lang.String,Tap> taps,
                        org.apache.hadoop.fs.Path path)

    - #### createSinks

          protected java.util.Map<java.lang.String,Tap> createSinks(org.apache.hadoop.mapred.JobConf jobConf)

    - #### fileOutputToTaps

          protected java.util.Map<java.lang.String,Tap> fileOutputToTaps(org.apache.hadoop.mapred.JobConf jobConf)

    - #### toSinkTap

          protected Tap toSinkTap(org.apache.hadoop.mapred.JobConf jobConf,
                      java.util.Map<java.lang.String,Tap> taps,
                      org.apache.hadoop.fs.Path path)

    - #### createTap

          protected Tap createTap(org.apache.hadoop.mapred.JobConf jobConf,
                      org.apache.hadoop.fs.Path path,
                      SinkMode sinkMode)

    - #### makeNameFromPath

          protected java.lang.String makeNameFromPath(java.util.Map<java.lang.String,Tap> taps,
                                          org.apache.hadoop.fs.Path path)

    - #### createTraps

          protected java.util.Map<java.lang.String,Tap> createTraps(org.apache.hadoop.mapred.JobConf jobConf)

Copyright © 2007-2015 Xplenty, Inc. All Rights Reserved.