cascading.flow.process

## Class ProcessFlow<Process,Config>

- java.lang.Object
- - cascading.flow.BaseFlow<Config>
  - - cascading.flow.process.ProcessFlow<Process,Config>

- All Implemented Interfaces:
    Flow<Config>, UnitOfWork<FlowStats>, ProcessLogger

      public class ProcessFlow<Process,Config>
      extends BaseFlow<Config>

  Class ProcessFlow is a `Flow` subclass that supports custom Riffle jobs.

  Use this class to allow custom Riffle jobs to participate in the `Cascade` scheduler. If other Flow instances in the Cascade share resources with this Flow instance, all participants will be scheduled according to their dependencies (topologically).

  Currently `FlowListener`s are supported but the `FlowListener.onThrowable(cascading.flow.Flow, Throwable)` event is not.

- - ### Nested Class Summary

    - ### Nested classes/interfaces inherited from class cascading.flow.BaseFlow

      `BaseFlow.FlowHolder`

  - ### Field Summary

    - ### Fields inherited from class cascading.flow.BaseFlow

      `completed, flowCanonicalHash, flowDescriptor, flowElementGraph, flowStats, flowStepGraph, platformInfo, shutdownHook, sinks, sources, steps, stop, stopJobsOnExit, thread, throwable`

    - ### Fields inherited from interface cascading.flow.Flow

      `CASCADING_FLOW_ID`

    - ### Fields inherited from interface cascading.util.ProcessLogger

      `NULL`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `ProcessFlow(java.util.Map<java.lang.Object,java.lang.Object> properties, java.lang.String name, Process process)` Constructor ProcessFlow creates a new ProcessFlow instance.
    `ProcessFlow(java.util.Map<java.lang.Object,java.lang.Object> properties, java.lang.String name, Process process, java.util.Map<java.lang.String,java.lang.String> flowDescriptor)` Constructor ProcessFlow creates a new ProcessFlow instance.
    `ProcessFlow(java.lang.String name, Process process)` Constructor ProcessFlow creates a new ProcessFlow instance.

  - ### Method Summary

    Methods
    Modifier and Type | Method and Description
    --- | ---
    `void` | `cleanup()`
    `void` | `complete()` Method complete starts the current Flow instance if it has not be previously started, then block until completion.
    `Config` | `getConfig()` Method getConfig returns the internal configuration object.
    `java.util.Map<java.lang.Object,java.lang.Object>` | `getConfigAsProperties()` Method getConfigAsProperties converts the internal configuration object into a `Map` of key value pairs.
    `Config` | `getConfigCopy()` Method getConfigCopy returns a copy of the internal configuration object.
    `FlowProcess<Config>` | `getFlowProcess()`
    `protected int` | `getMaxNumParallelSteps()`
    `Process` | `getProcess()` Method getProcess returns the process of this ProcessFlow object.
    `java.lang.String` | `getProperty(java.lang.String key)` Returns the String property associated with the given key from the current Configuration instance.
    `protected void` | `initConfig(java.util.Map<java.lang.Object,java.lang.Object> properties, Config parentConfig)` This method creates a new internal Config with the parentConfig as defaults using the properties to override the defaults.
    `protected void` | `internalClean(boolean stop)`
    `protected void` | `internalShutdown()`
    `protected void` | `internalStart()`
    `protected Config` | `newConfig(Config defaultConfig)`
    `void` | `prepare()` Method prepare is used by a `Cascade` to notify the given Flow it should initialize or clear any resources necessary for `Flow.start()` to be called successfully.
    `protected void` | `setConfigProperty(Config properties, java.lang.Object key, java.lang.Object value)`
    `void` | `setTapFromProcess()` Method setTapFromProcess build `Tap` instance for the give process incoming and outgoing dependencies.
    `void` | `start()` Method start begins the execution of this Flow instance.
    `boolean` | `stepsAreLocal()` Method jobsAreLocal returns true if all jobs are executed in-process as a single map and reduce task.
    `void` | `stop()` Method stop stops all running jobs, killing any currently executing.
    `java.lang.String` | `toString()`

    - ### Methods inherited from class cascading.flow.BaseFlow

      `addListener, addPlannerProperties, addSessionProperties, addStepListener, areSinksStale, areSourcesNewer, createConfig, createFlowCanonicalHash, createFlowStats, createFlowThread, createPrepareFlowStats, deleteCheckpointsIfNotUpdate, deleteCheckpointsIfReplace, deleteSinks, deleteSinksIfNotUpdate, deleteSinksIfReplace, deleteTrapsIfNotUpdate, deleteTrapsIfReplace, fireOnCompleted, fireOnStarting, fireOnStopping, fireOnThrowable, fireOnThrowable, getCascadeID, getCascadingServices, getCheckpointNames, getCheckpoints, getCheckpointsCollection, getClassPath, getClientState, getEligibleJobsSize, getFieldsFor, getFlowCanonicalHash, getFlowDescriptor, getFlowElementGraph, getFlowSession, getFlowSkipStrategy, getFlowStats, getFlowStepGraph, getFlowSteps, getFlowStepStrategy, getHolder, getID, getJobMapCallables, getName, getPlannerInfo, getPlatformInfo, getRunID, getSink, getSink, getSinkModified, getSinkNames, getSinks, getSinksCollection, getSource, getSourceNames, getSources, getSourcesCollection, getSpawnStrategy, getStats, getSubmitPriority, getTags, getTotalSliceCPUMilliSeconds, getTrapNames, getTraps, getTrapsCollection, handleExecutorShutdown, hasListeners, hasStepListeners, initFromProperties, initialize, initializeChildStats, initializeNewJobsMap, initSteps, internalStopAllJobs, isDebugEnabled, isInfoEnabled, isJobsMapInitialized, isSkipFlow, isStopJobsOnExit, logDebug, logError, logError, logInfo, logWarn, logWarn, logWarn, openSink, openSink, openSource, openSource, openTapForRead, openTapForWrite, openTrap, openTrap, presentSinkFields, presentSourceFields, registerShutdownHook, removeListener, removeListeners, removeStepListener, resourceExists, retrieveSinkFields, retrieveSourceFields, setCascade, setCheckpoints, setFlowElementGraph, setFlowSkipStrategy, setFlowStepGraph, setFlowStepStrategy, setName, setPlannerInfo, setSinks, setSources, setSpawnStrategy, setSubmitPriority, setTraps, spawnSteps, updateJobsMap, updateSchemes, writeDOT, writeStepsDOT`

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait`

- - ### Constructor Detail

    - #### ProcessFlow

          @ConstructorProperties(value={"name","process"})
          public ProcessFlow(java.lang.String name,
                                           Process process)

      Constructor ProcessFlow creates a new ProcessFlow instance.
      Parameters:
        `name` - of type String
        `process` - of type JobConf

    - #### ProcessFlow

          @ConstructorProperties(value={"properties","name","process"})
          public ProcessFlow(java.util.Map<java.lang.Object,java.lang.Object> properties,
                                           java.lang.String name,
                                           Process process)

      Constructor ProcessFlow creates a new ProcessFlow instance.
      Parameters:
        `properties` - of type Map
        `name` - of type String
        `process` - of type P

    - #### ProcessFlow

          @ConstructorProperties(value={"properties","name","process","flowDescriptor"})
          public ProcessFlow(java.util.Map<java.lang.Object,java.lang.Object> properties,
                                           java.lang.String name,
                                           Process process,
                                           java.util.Map<java.lang.String,java.lang.String> flowDescriptor)

      Constructor ProcessFlow creates a new ProcessFlow instance.
      Parameters:
        `properties` - of type Map
        `name` - of type String
        `process` - of type P
        `flowDescriptor` - pf type LinkedHashMap

  - ### Method Detail

    - #### setTapFromProcess

          public void setTapFromProcess()

      Method setTapFromProcess build `Tap` instance for the give process incoming and outgoing dependencies.

      This method may be called repeatedly to re-configure the source and sink taps.

    - #### getProcess

          public Process getProcess()

      Method getProcess returns the process of this ProcessFlow object.
      Returns:
        the process (type P) of this ProcessFlow object.

    - #### initConfig

          protected void initConfig(java.util.Map<java.lang.Object,java.lang.Object> properties,
                        Config parentConfig)

      Description copied from class: `BaseFlow`
      This method creates a new internal Config with the parentConfig as defaults using the properties to override the defaults.
      Specified by:
        `initConfig` in class `BaseFlow<Config>`
      Parameters:
        `properties` - of type Map
        `parentConfig` - of type Config

    - #### setConfigProperty

          protected void setConfigProperty(Config properties,
                               java.lang.Object key,
                               java.lang.Object value)

      Specified by:
        `setConfigProperty` in class `BaseFlow<Config>`

    - #### newConfig

          protected Config newConfig(Config defaultConfig)

      Specified by:
        `newConfig` in class `BaseFlow<Config>`

    - #### getConfig

          public Config getConfig()

      Description copied from interface: `Flow`
      Method getConfig returns the internal configuration object.

      Any changes to this object will not be reflected in child steps. See `FlowConnector` for setting default properties visible to children. Or see `FlowStepStrategy` for setting properties on individual steps before they are executed.
      Returns:
        the default configuration of this Flow

    - #### getConfigCopy

          public Config getConfigCopy()

      Description copied from interface: `Flow`
      Method getConfigCopy returns a copy of the internal configuration object. This object can be safely modified.
      Returns:
        a copy of the default configuration of this Flow

    - #### getConfigAsProperties

          public java.util.Map<java.lang.Object,java.lang.Object> getConfigAsProperties()

      Description copied from interface: `Flow`
      Method getConfigAsProperties converts the internal configuration object into a `Map` of key value pairs.
      Returns:
        a Map of key/value pairs

    - #### getProperty

          public java.lang.String getProperty(java.lang.String key)

      Description copied from interface: `Flow`
      Returns the String property associated with the given key from the current Configuration instance.
      Parameters:
        `key` - of type String
      Returns:
        the String value

    - #### getFlowProcess

          public FlowProcess<Config> getFlowProcess()

    - #### stepsAreLocal

          public boolean stepsAreLocal()

      Description copied from interface: `Flow`
      Method jobsAreLocal returns true if all jobs are executed in-process as a single map and reduce task.
      Returns:
        boolean

    - #### prepare

          public void prepare()

      Description copied from interface: `Flow`
      Method prepare is used by a `Cascade` to notify the given Flow it should initialize or clear any resources necessary for `Flow.start()` to be called successfully.

      Specifically, this implementation calls `BaseFlow.deleteSinksIfNotUpdate()` && `BaseFlow.deleteTrapsIfNotUpdate()`.
      Specified by:
        `prepare` in interface `Flow<Config>`
      Specified by:
        `prepare` in interface `UnitOfWork<FlowStats>`
      Overrides:
        `prepare` in class `BaseFlow<Config>`

    - #### start

          public void start()

      Description copied from interface: `Flow`
      Method start begins the execution of this Flow instance. It will return immediately. Use the method `Flow.complete()` to block until this Flow completes.
      Specified by:
        `start` in interface `Flow<Config>`
      Specified by:
        `start` in interface `UnitOfWork<FlowStats>`
      Overrides:
        `start` in class `BaseFlow<Config>`

    - #### internalStart

          protected void internalStart()

      Specified by:
        `internalStart` in class `BaseFlow<Config>`

    - #### stop

          public void stop()

      Description copied from interface: `Flow`
      Method stop stops all running jobs, killing any currently executing.
      Specified by:
        `stop` in interface `Flow<Config>`
      Specified by:
        `stop` in interface `UnitOfWork<FlowStats>`
      Overrides:
        `stop` in class `BaseFlow<Config>`

    - #### internalClean

          protected void internalClean(boolean stop)

      Specified by:
        `internalClean` in class `BaseFlow<Config>`

    - #### complete

          public void complete()

      Description copied from interface: `Flow`
      Method complete starts the current Flow instance if it has not be previously started, then block until completion.
      Specified by:
        `complete` in interface `Flow<Config>`
      Specified by:
        `complete` in interface `UnitOfWork<FlowStats>`
      Overrides:
        `complete` in class `BaseFlow<Config>`

    - #### cleanup

          public void cleanup()

      Specified by:
        `cleanup` in interface `Flow<Config>`
      Specified by:
        `cleanup` in interface `UnitOfWork<FlowStats>`
      Overrides:
        `cleanup` in class `BaseFlow<Config>`

    - #### getMaxNumParallelSteps

          protected int getMaxNumParallelSteps()

      Specified by:
        `getMaxNumParallelSteps` in class `BaseFlow<Config>`

    - #### internalShutdown

          protected void internalShutdown()

      Specified by:
        `internalShutdown` in class `BaseFlow<Config>`

    - #### toString

          public java.lang.String toString()

      Overrides:
        `toString` in class `BaseFlow<Config>`

Copyright © 2007-2015 Xplenty, Inc. All Rights Reserved.