cascading.flow

## Class BaseFlow<Config>

- java.lang.Object
- - cascading.flow.BaseFlow<Config>

All Implemented Interfaces:
  Flow<Config>, UnitOfWork<FlowStats>, ProcessLogger

Direct Known Subclasses:
  ProcessFlow

    @Process
    public abstract class BaseFlow<Config>
    extends java.lang.Object
    implements Flow<Config>, ProcessLogger

### Nested Class Summary

Nested Classes
`static class` `BaseFlow.FlowHolder` Class FlowHolder is a helper class for wrapping Flow instances.

### Field Summary

Fields
`protected boolean` `completed` Field completed
`protected java.lang.String` `flowCanonicalHash` Field flowCanonicalHash
`protected java.util.HashMap<java.lang.String,java.lang.String>` `flowDescriptor`
`protected FlowElementGraph` `flowElementGraph` Field flowElementGraph
`protected FlowStats` `flowStats` Field flowStats
`protected FlowStepGraph` `flowStepGraph` Field stepGraph
`protected PlatformInfo` `platformInfo`
`protected ShutdownUtil.Hook` `shutdownHook`
`protected java.util.Map<java.lang.String,Tap>` `sinks` Field sinks
`protected java.util.Map<java.lang.String,Tap>` `sources` Field sources
`protected java.util.List<FlowStep<Config>>` `steps` Field steps
`protected boolean` `stop` Field stop
`protected boolean` `stopJobsOnExit` Field stopJobsOnExit
`protected java.lang.Thread` `thread` Field thread
`protected java.lang.Throwable` `throwable` Field throwable

### Fields inherited from interface cascading.flow.Flow

`CASCADING_FLOW_ID`

### Fields inherited from interface cascading.util.ProcessLogger

`NULL`

### Constructor Summary

Constructors
`protected` `BaseFlow()` Used for testing.
`protected` `BaseFlow(PlatformInfo platformInfo, java.util.Map<java.lang.Object,java.lang.Object> properties, Config defaultConfig, FlowDef flowDef)`
`protected` `BaseFlow(PlatformInfo platformInfo, java.util.Map<java.lang.Object,java.lang.Object> properties, Config defaultConfig, java.lang.String name, java.util.Map<java.lang.String,java.lang.String> flowDescriptor)`
`protected` `BaseFlow(PlatformInfo platformInfo, java.lang.String name)` Does not initialize stats

### Method Summary

Methods
`void` `addListener(FlowListener flowListener)` Method addListener registers the given flowListener with this instance.
`protected void` `addPlannerProperties()`
`protected void` `addSessionProperties(java.util.Map<java.lang.Object,java.lang.Object> properties)`
`void` `addStepListener(FlowStepListener flowStepListener)` Method addStepListener registers the given flowStepListener with this instance.
`boolean` `areSinksStale()` Method areSinksStale returns true if any of the sinks referenced are out of date in relation to the sources.
`boolean` `areSourcesNewer(long sinkModified)` Method areSourcesNewer returns true if any source is newer than the given sinkModified date value.
`void` `cleanup()`
`void` `complete()` Method complete starts the current Flow instance if it has not be previously started, then block until completion.
`Config` `createConfig(java.util.Map<java.lang.Object,java.lang.Object> properties, Config defaultConfig)`
`protected java.lang.String` `createFlowCanonicalHash(FlowElementGraph flowElementGraph)`
`protected FlowStats` `createFlowStats()`
`protected java.lang.Thread` `createFlowThread(java.lang.String threadName)`
`protected FlowStats` `createPrepareFlowStats()`
`void` `deleteCheckpointsIfNotUpdate()`
`void` `deleteCheckpointsIfReplace()`
`void` `deleteSinks()` Method deleteSinks deletes all sinks, whether or not they are configured for `SinkMode.UPDATE`.
`void` `deleteSinksIfNotUpdate()` Method deleteSinksIfNotUpdate deletes all sinks if they are not configured with the `SinkMode.UPDATE` flag.
`void` `deleteSinksIfReplace()` Method deleteSinksIfReplace deletes all sinks that are configured with the `SinkMode.REPLACE` flag.
`void` `deleteTrapsIfNotUpdate()`
`void` `deleteTrapsIfReplace()`
`protected void` `fireOnCompleted()`
`protected void` `fireOnStarting()`
`protected void` `fireOnStopping()`
`protected void` `fireOnThrowable()`
`protected void` `fireOnThrowable(java.lang.Throwable throwable)`
`java.lang.String` `getCascadeID()` Returns the parent Cascade ID that owns this Flow instance.
`CascadingServices` `getCascadingServices()`
`java.util.List<java.lang.String>` `getCheckpointNames()`
`java.util.Map<java.lang.String,Tap>` `getCheckpoints()` Method getCheckpoints returns the checkpoint taps of this Flow object.
`java.util.Collection<Tap>` `getCheckpointsCollection()` Method getCheckpointsCollection returns a `Collection` of checkpoint `Tap`s for this Flow object.
`java.util.List<java.lang.String>` `getClassPath()`
`protected ClientState` `getClientState()`
`protected int` `getEligibleJobsSize()`
`protected Fields` `getFieldsFor(FlowElementGraph pipeGraph, Tap tap)`
`java.lang.String` `getFlowCanonicalHash()` The hash value can be used to determine if two unique Flow instances performed the same work.
`java.util.Map<java.lang.String,java.lang.String>` `getFlowDescriptor()` Returns an immutable map of properties giving more details about the Flow object.
`protected FlowElementGraph` `getFlowElementGraph()`
`FlowSession` `getFlowSession()`
`FlowSkipStrategy` `getFlowSkipStrategy()` Method getFlowSkipStrategy returns the current `FlowSkipStrategy` used by this Flow.
`FlowStats` `getFlowStats()` Method getFlowStats returns the flowStats of this Flow object.
`protected FlowStepGraph` `getFlowStepGraph()`
`java.util.List<FlowStep<Config>>` `getFlowSteps()` Method getFlowSteps returns the flowSteps of this Flow object.
`FlowStepStrategy` `getFlowStepStrategy()` Returns the current `FlowStepStrategy` instance.
`BaseFlow.FlowHolder` `getHolder()` Used to return a simple wrapper for use as an edge in a graph where there can only be one instance of every edge.
`java.lang.String` `getID()` Method getID returns the ID of this Flow object.
`protected java.util.List<java.util.concurrent.Callable<java.lang.Throwable>>` `getJobMapCallables()`
`protected abstract int` `getMaxNumParallelSteps()`
`java.lang.String` `getName()` Method getName returns the name of this Flow object.
`PlannerInfo` `getPlannerInfo()` Returns any meta-data about the planner that created this Flow instance.
`PlatformInfo` `getPlatformInfo()` Returns any meta-data about the underlying platform this Flow instance will run against.
`java.lang.String` `getRunID()` Returns the run ID given when this Flow instance was defined in the FlowDef.
`Tap` `getSink()` Method getSink returns the first sink of this Flow object.
`Tap` `getSink(java.lang.String name)`
`long` `getSinkModified()` Method getSinkModified returns the youngest modified date of any sink `Tap` managed by this Flow instance.
`java.util.List<java.lang.String>` `getSinkNames()`
`java.util.Map<java.lang.String,Tap>` `getSinks()` Method getSinks returns the sinks of this Flow object.
`java.util.Collection<Tap>` `getSinksCollection()` Method getSinksCollection returns a `Collection` of sink `Tap`s for this Flow object.
`Tap` `getSource(java.lang.String name)`
`java.util.List<java.lang.String>` `getSourceNames()`
`java.util.Map<java.lang.String,Tap>` `getSources()` Method getSources returns the sources of this Flow object.
`java.util.Collection<Tap>` `getSourcesCollection()` Method getSourcesCollection returns a `Collection` of source `Tap`s for this Flow object.
`UnitOfWorkSpawnStrategy` `getSpawnStrategy()`
`FlowStats` `getStats()`
`int` `getSubmitPriority()` Method getSubmitPriority returns the submitPriority of this Flow object.
`java.lang.String` `getTags()`
`protected long` `getTotalSliceCPUMilliSeconds()`
`java.util.List<java.lang.String>` `getTrapNames()`
`java.util.Map<java.lang.String,Tap>` `getTraps()` Method getTraps returns the traps of this Flow object.
`java.util.Collection<Tap>` `getTrapsCollection()` Method getTrapsCollection returns a `Collection` of trap `Tap`s for this Flow object.
`protected void` `handleExecutorShutdown()`
`boolean` `hasListeners()` Method hasListeners returns true if `FlowListener` instances have been registered.
`boolean` `hasStepListeners()` Method hasStepListeners returns true if `FlowStepListener` instances have been registered with any of the `FlowStep`s belonging to this instance
`protected abstract void` `initConfig(java.util.Map<java.lang.Object,java.lang.Object> properties, Config parentConfig)` This method creates a new internal Config with the parentConfig as defaults using the properties to override the defaults.
`protected void` `initFromProperties(java.util.Map<java.lang.Object,java.lang.Object> properties)`
`void` `initialize(FlowElementGraph flowElementGraph, FlowStepGraph flowStepGraph)`
`protected void` `initializeChildStats()`
`protected void` `initializeNewJobsMap()`
`protected void` `initSteps()`
`protected abstract void` `internalClean(boolean stop)`
`protected abstract void` `internalShutdown()`
`protected abstract void` `internalStart()`
`protected void` `internalStopAllJobs()`
`boolean` `isDebugEnabled()`
`boolean` `isInfoEnabled()`
`protected boolean` `isJobsMapInitialized()`
`boolean` `isSkipFlow()` Method isSkipFlow returns true if the parent `Cascade` should skip this Flow instance.
`boolean` `isStopJobsOnExit()` Method isStopJobsOnExit returns the stopJobsOnExit of this Flow object.
`void` `logDebug(java.lang.String message, java.lang.Object... arguments)`
`void` `logError(java.lang.String message, java.lang.Object... arguments)`
`void` `logError(java.lang.String message, java.lang.Throwable throwable)`
`void` `logInfo(java.lang.String message, java.lang.Object... arguments)`
`void` `logWarn(java.lang.String message)`
`void` `logWarn(java.lang.String message, java.lang.Object... arguments)`
`void` `logWarn(java.lang.String message, java.lang.Throwable throwable)`
`protected abstract Config` `newConfig(Config defaultConfig)`
`TupleEntryIterator` `openSink()` Method openSink opens the first sink Tap.
`TupleEntryIterator` `openSink(java.lang.String name)` Method openSink opens the named sink Tap.
`TupleEntryIterator` `openSource()` Method openSource opens the first source Tap.
`TupleEntryIterator` `openSource(java.lang.String name)` Method openSource opens the named source Tap.
`TupleEntryIterator` `openTapForRead(Tap tap)` Method openTapForRead return a `TupleEntryIterator` for the given Tap instance.
`TupleEntryCollector` `openTapForWrite(Tap tap)` Method openTapForWrite returns a (@link TupleCollector} for the given Tap instance.
`TupleEntryIterator` `openTrap()` Method openTrap opens the first trap Tap.
`TupleEntryIterator` `openTrap(java.lang.String name)` Method openTrap opens the named trap Tap.
`void` `prepare()` Method prepare is used by a `Cascade` to notify the given Flow it should initialize or clear any resources necessary for `Flow.start()` to be called successfully.
`protected void` `presentSinkFields(FlowElementGraph pipeGraph)` Present the current resolved fields for the Tap
`protected void` `presentSourceFields(FlowElementGraph pipeGraph)` Present the current resolved fields for the Tap
`protected void` `registerShutdownHook()`
`boolean` `removeListener(FlowListener flowListener)` Method removeListener removes the given flowListener from this instance.
`protected void` `removeListeners(java.util.Collection listeners)`
`boolean` `removeStepListener(FlowStepListener flowStepListener)` Method removeStepListener removes the given flowStepListener from this instance.
`boolean` `resourceExists(Tap tap)` Method resourceExists returns true if the resource represented by the given Tap instance exists.
`protected void` `retrieveSinkFields()` Force a Scheme to fetch any fields from a meta-data store
`protected void` `retrieveSourceFields()` Force a Scheme to fetch any fields from a meta-data store
`void` `setCascade(Cascade cascade)`
`protected void` `setCheckpoints(java.util.Map<java.lang.String,Tap> checkpoints)`
`protected abstract void` `setConfigProperty(Config config, java.lang.Object key, java.lang.Object value)`
`protected void` `setFlowElementGraph(FlowElementGraph flowElementGraph)`
`FlowSkipStrategy` `setFlowSkipStrategy(FlowSkipStrategy flowSkipStrategy)` Method setFlowSkipStrategy sets a new `FlowSkipStrategy`, the current strategy is returned.
`protected void` `setFlowStepGraph(FlowStepGraph flowStepGraph)`
`void` `setFlowStepStrategy(FlowStepStrategy flowStepStrategy)` Sets a default `FlowStepStrategy` instance.
`protected void` `setName(java.lang.String name)`
`void` `setPlannerInfo(PlannerInfo plannerInfo)`
`protected void` `setSinks(java.util.Map<java.lang.String,Tap> sinks)`
`protected void` `setSources(java.util.Map<java.lang.String,Tap> sources)`
`void` `setSpawnStrategy(UnitOfWorkSpawnStrategy spawnStrategy)`
`void` `setSubmitPriority(int submitPriority)` Method setSubmitPriority sets the submitPriority of this Flow object.
`protected void` `setTraps(java.util.Map<java.lang.String,Tap> traps)`
`protected boolean` `spawnSteps()` Reentrant method to launch all step jobs, returns false if any of the current steps failed.
`void` `start()` Method start begins the execution of this Flow instance.
`void` `stop()` Method stop stops all running jobs, killing any currently executing.
`java.lang.String` `toString()`
`protected void` `updateJobsMap()`
`FlowElementGraph` `updateSchemes(FlowElementGraph pipeGraph)`
`void` `writeDOT(java.lang.String filename)` Method writeDOT writes this Flow instance to the given filename as a DOT file for import into a graphics package.
`void` `writeStepsDOT(java.lang.String filename)` Method writeStepsDOT writes this Flow step graph to the given filename as a DOT file for import into a graphics package.

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait`

### Methods inherited from interface cascading.flow.Flow

`getConfig, getConfigAsProperties, getConfigCopy, getFlowProcess, getProperty, stepsAreLocal`

### Field Detail

#### platformInfo

    protected PlatformInfo platformInfo

#### flowStats

    protected FlowStats flowStats

Field flowStats

#### sources

    protected java.util.Map<java.lang.String,Tap> sources

Field sources

#### sinks

    protected java.util.Map<java.lang.String,Tap> sinks

Field sinks

#### stopJobsOnExit

    protected boolean stopJobsOnExit

Field stopJobsOnExit

#### flowStepGraph

    protected FlowStepGraph flowStepGraph

Field stepGraph

#### thread

    protected transient java.lang.Thread thread

Field thread

#### throwable

    protected java.lang.Throwable throwable

Field throwable

#### stop

    protected volatile boolean stop

Field stop

#### completed

    protected volatile boolean completed

Field completed

#### flowCanonicalHash

    protected java.lang.String flowCanonicalHash

Field flowCanonicalHash

#### flowElementGraph

    protected FlowElementGraph flowElementGraph

Field flowElementGraph

#### steps

    protected transient java.util.List<FlowStep<Config>> steps

Field steps

#### shutdownHook

    protected ShutdownUtil.Hook shutdownHook

#### flowDescriptor

    protected java.util.HashMap<java.lang.String,java.lang.String> flowDescriptor

### Constructor Detail

#### BaseFlow

    protected BaseFlow()

Used for testing.

#### BaseFlow

    protected BaseFlow(PlatformInfo platformInfo,
            java.lang.String name)

Does not initialize stats

Parameters:
  `name` -

#### BaseFlow

    protected BaseFlow(PlatformInfo platformInfo,
            java.util.Map<java.lang.Object,java.lang.Object> properties,
            Config defaultConfig,
            java.lang.String name,
            java.util.Map<java.lang.String,java.lang.String> flowDescriptor)

#### BaseFlow

    protected BaseFlow(PlatformInfo platformInfo,
            java.util.Map<java.lang.Object,java.lang.Object> properties,
            Config defaultConfig,
            FlowDef flowDef)

### Method Detail

#### setPlannerInfo

    public void setPlannerInfo(PlannerInfo plannerInfo)

#### getPlannerInfo

    public PlannerInfo getPlannerInfo()

Description copied from interface: `Flow`
Returns any meta-data about the planner that created this Flow instance.

Specified by:
  `getPlannerInfo` in interface `Flow<Config>`
Returns:
  an instance of PlannerInfo

#### getPlatformInfo

    public PlatformInfo getPlatformInfo()

Description copied from interface: `Flow`
Returns any meta-data about the underlying platform this Flow instance will run against.

Specified by:
  `getPlatformInfo` in interface `Flow<Config>`
Returns:
  an instance of PlatformInfo

#### initialize

    public void initialize(FlowElementGraph flowElementGraph,
                  FlowStepGraph flowStepGraph)

#### updateSchemes

    public FlowElementGraph updateSchemes(FlowElementGraph pipeGraph)

#### retrieveSourceFields

    protected void retrieveSourceFields()

Force a Scheme to fetch any fields from a meta-data store

#### presentSourceFields

    protected void presentSourceFields(FlowElementGraph pipeGraph)

Present the current resolved fields for the Tap

Parameters:
  `pipeGraph` -

#### retrieveSinkFields

    protected void retrieveSinkFields()

Force a Scheme to fetch any fields from a meta-data store

#### presentSinkFields

    protected void presentSinkFields(FlowElementGraph pipeGraph)

Present the current resolved fields for the Tap

Parameters:
  `pipeGraph` -

#### getFieldsFor

    protected Fields getFieldsFor(FlowElementGraph pipeGraph,
                      Tap tap)

#### addSessionProperties

    protected void addSessionProperties(java.util.Map<java.lang.Object,java.lang.Object> properties)

#### addPlannerProperties

    protected void addPlannerProperties()

#### createPrepareFlowStats

    protected FlowStats createPrepareFlowStats()

#### createFlowStats

    protected FlowStats createFlowStats()

#### getCascadingServices

    public CascadingServices getCascadingServices()

#### getClientState

    protected ClientState getClientState()

#### initSteps

    protected void initSteps()

#### getName

    public java.lang.String getName()

Description copied from interface: `Flow`
Method getName returns the name of this Flow object.

Specified by:
  `getName` in interface `Flow<Config>`
Specified by:
  `getName` in interface `UnitOfWork<FlowStats>`
Returns:
  the name (type String) of this Flow object.

#### setName

    protected void setName(java.lang.String name)

#### getID

    public java.lang.String getID()

Description copied from interface: `Flow`
Method getID returns the ID of this Flow object.

The ID value is a long HEX String used to identify this instance globally. Subsequent Flow instances created with identical parameters will not return the same ID.

Specified by:
  `getID` in interface `Flow<Config>`
Specified by:
  `getID` in interface `UnitOfWork<FlowStats>`
Returns:
  the ID (type String) of this Flow object.

#### getTags

    public java.lang.String getTags()

Specified by:
  `getTags` in interface `Flow<Config>`
Specified by:
  `getTags` in interface `UnitOfWork<FlowStats>`

#### getSubmitPriority

    public int getSubmitPriority()

Description copied from interface: `Flow`
Method getSubmitPriority returns the submitPriority of this Flow object.

10 is lowest, 1 is the highest, 5 is the default.

Specified by:
  `getSubmitPriority` in interface `Flow<Config>`
Returns:
  the submitPriority (type int) of this FlowStep object.

#### setSubmitPriority

    public void setSubmitPriority(int submitPriority)

Description copied from interface: `Flow`
Method setSubmitPriority sets the submitPriority of this Flow object.

10 is lowest, 1 is the highest, 5 is the default.

Specified by:
  `setSubmitPriority` in interface `Flow<Config>`
Parameters:
  `submitPriority` - the submitPriority of this FlowStep object.

#### getFlowCanonicalHash

    public java.lang.String getFlowCanonicalHash()

The hash value can be used to determine if two unique Flow instances performed the same work.

The source and sink taps are not relevant to the hash.

Returns:
  a String value

#### createFlowCanonicalHash

    protected java.lang.String createFlowCanonicalHash(FlowElementGraph flowElementGraph)

#### getFlowElementGraph

    protected FlowElementGraph getFlowElementGraph()

#### setFlowElementGraph

    protected void setFlowElementGraph(FlowElementGraph flowElementGraph)

#### getFlowStepGraph

    protected FlowStepGraph getFlowStepGraph()

#### setFlowStepGraph

    protected void setFlowStepGraph(FlowStepGraph flowStepGraph)

#### setSources

    protected void setSources(java.util.Map<java.lang.String,Tap> sources)

#### setSinks

    protected void setSinks(java.util.Map<java.lang.String,Tap> sinks)

#### setTraps

    protected void setTraps(java.util.Map<java.lang.String,Tap> traps)

#### setCheckpoints

    protected void setCheckpoints(java.util.Map<java.lang.String,Tap> checkpoints)

#### initConfig

    protected abstract void initConfig(java.util.Map<java.lang.Object,java.lang.Object> properties,
                  Config parentConfig)

This method creates a new internal Config with the parentConfig as defaults using the properties to override the defaults.

Parameters:
  `properties` - of type Map
  `parentConfig` - of type Config

#### createConfig

    public Config createConfig(java.util.Map<java.lang.Object,java.lang.Object> properties,
                      Config defaultConfig)

#### setConfigProperty

    protected abstract void setConfigProperty(Config config,
                         java.lang.Object key,
                         java.lang.Object value)

#### newConfig

    protected abstract Config newConfig(Config defaultConfig)

#### initFromProperties

    protected void initFromProperties(java.util.Map<java.lang.Object,java.lang.Object> properties)

#### getFlowSession

    public FlowSession getFlowSession()

#### getFlowStats

    public FlowStats getFlowStats()

Description copied from interface: `Flow`
Method getFlowStats returns the flowStats of this Flow object.

Specified by:
  `getFlowStats` in interface `Flow<Config>`
Returns:
  the flowStats (type FlowStats) of this Flow object.

#### getFlowDescriptor

    public java.util.Map<java.lang.String,java.lang.String> getFlowDescriptor()

Description copied from interface: `Flow`
Returns an immutable map of properties giving more details about the Flow object.

See `FlowDef.addDescription(String, String)` to set values on a given Flow.

Flow descriptions provide meta-data to monitoring systems describing the workload a given Flow represents. For known description types, see `FlowDescriptors`.

Specified by:
  `getFlowDescriptor` in interface `Flow<Config>`
Returns:
  Map<String,String>

#### getStats

    public FlowStats getStats()

Specified by:
  `getStats` in interface `UnitOfWork<FlowStats>`

#### removeListeners

    protected void removeListeners(java.util.Collection listeners)

#### hasListeners

    public boolean hasListeners()

Description copied from interface: `Flow`
Method hasListeners returns true if `FlowListener` instances have been registered.

Specified by:
  `hasListeners` in interface `Flow<Config>`
Returns:
  boolean

#### addListener

    public void addListener(FlowListener flowListener)

Description copied from interface: `Flow`
Method addListener registers the given flowListener with this instance.

Specified by:
  `addListener` in interface `Flow<Config>`
Parameters:
  `flowListener` - of type FlowListener

#### removeListener

    public boolean removeListener(FlowListener flowListener)

Description copied from interface: `Flow`
Method removeListener removes the given flowListener from this instance.

Specified by:
  `removeListener` in interface `Flow<Config>`
Parameters:
  `flowListener` - of type FlowListener
Returns:
  true if the listener was removed

#### hasStepListeners

    public boolean hasStepListeners()

Description copied from interface: `Flow`
Method hasStepListeners returns true if `FlowStepListener` instances have been registered with any of the `FlowStep`s belonging to this instance

Specified by:
  `hasStepListeners` in interface `Flow<Config>`
Returns:
  boolean

#### addStepListener

    public void addStepListener(FlowStepListener flowStepListener)

Description copied from interface: `Flow`
Method addStepListener registers the given flowStepListener with this instance.

Specified by:
  `addStepListener` in interface `Flow<Config>`
Parameters:
  `flowStepListener` - of type addStepListener

#### removeStepListener

    public boolean removeStepListener(FlowStepListener flowStepListener)

Description copied from interface: `Flow`
Method removeStepListener removes the given flowStepListener from this instance.

Specified by:
  `removeStepListener` in interface `Flow<Config>`
Parameters:
  `flowStepListener` - of type FlowStepListener
Returns:
  true if the listener was removed from all the `FlowStep` belonging to this instance

#### getSources

    public java.util.Map<java.lang.String,Tap> getSources()

Description copied from interface: `Flow`
Method getSources returns the sources of this Flow object.

Specified by:
  `getSources` in interface `Flow<Config>`
Returns:
  the sources (type Map) of this Flow object.

#### getSourceNames

    public java.util.List<java.lang.String> getSourceNames()

Specified by:
  `getSourceNames` in interface `Flow<Config>`

#### getSource

    public Tap getSource(java.lang.String name)

Specified by:
  `getSource` in interface `Flow<Config>`

#### getSourcesCollection

    @DependencyIncoming
    public java.util.Collection<Tap> getSourcesCollection()

Description copied from interface: `Flow`
Method getSourcesCollection returns a `Collection` of source `Tap`s for this Flow object.

Specified by:
  `getSourcesCollection` in interface `Flow<Config>`
Returns:
  the sourcesCollection (type Collection<Tap>) of this Flow object.

#### getSinks

    public java.util.Map<java.lang.String,Tap> getSinks()

Description copied from interface: `Flow`
Method getSinks returns the sinks of this Flow object.

Specified by:
  `getSinks` in interface `Flow<Config>`
Returns:
  the sinks (type Map) of this Flow object.

#### getSinkNames

    public java.util.List<java.lang.String> getSinkNames()

Specified by:
  `getSinkNames` in interface `Flow<Config>`

#### getSink

    public Tap getSink(java.lang.String name)

Specified by:
  `getSink` in interface `Flow<Config>`

#### getSinksCollection

    @DependencyOutgoing
    public java.util.Collection<Tap> getSinksCollection()

Description copied from interface: `Flow`
Method getSinksCollection returns a `Collection` of sink `Tap`s for this Flow object.

Specified by:
  `getSinksCollection` in interface `Flow<Config>`
Returns:
  the sinkCollection (type Collection<Tap>) of this Flow object.

#### getSink

    public Tap getSink()

Description copied from interface: `Flow`
Method getSink returns the first sink of this Flow object.

Specified by:
  `getSink` in interface `Flow<Config>`
Returns:
  the sink (type Tap) of this Flow object.

#### getTraps

    public java.util.Map<java.lang.String,Tap> getTraps()

Description copied from interface: `Flow`
Method getTraps returns the traps of this Flow object.

Specified by:
  `getTraps` in interface `Flow<Config>`
Returns:
  the traps (type Map<String, Tap>) of this Flow object.

#### getTrapNames

    public java.util.List<java.lang.String> getTrapNames()

Specified by:
  `getTrapNames` in interface `Flow<Config>`

#### getTrapsCollection

    public java.util.Collection<Tap> getTrapsCollection()

Description copied from interface: `Flow`
Method getTrapsCollection returns a `Collection` of trap `Tap`s for this Flow object.

Specified by:
  `getTrapsCollection` in interface `Flow<Config>`
Returns:
  the trapsCollection (type Collection<Tap>) of this Flow object.

#### getCheckpoints

    public java.util.Map<java.lang.String,Tap> getCheckpoints()

Description copied from interface: `Flow`
Method getCheckpoints returns the checkpoint taps of this Flow object.

Specified by:
  `getCheckpoints` in interface `Flow<Config>`
Returns:
  the traps (type Map<String, Tap>) of this Flow object.

#### getCheckpointNames

    public java.util.List<java.lang.String> getCheckpointNames()

Specified by:
  `getCheckpointNames` in interface `Flow<Config>`

#### getCheckpointsCollection

    public java.util.Collection<Tap> getCheckpointsCollection()

Description copied from interface: `Flow`
Method getCheckpointsCollection returns a `Collection` of checkpoint `Tap`s for this Flow object.

Specified by:
  `getCheckpointsCollection` in interface `Flow<Config>`
Returns:
  the trapsCollection (type Collection<Tap>) of this Flow object.

#### isStopJobsOnExit

    public boolean isStopJobsOnExit()

Description copied from interface: `Flow`
Method isStopJobsOnExit returns the stopJobsOnExit of this Flow object. Defaults to `true`.

Specified by:
  `isStopJobsOnExit` in interface `Flow<Config>`
Returns:
  the stopJobsOnExit (type boolean) of this Flow object.

#### getFlowSkipStrategy

    public FlowSkipStrategy getFlowSkipStrategy()

Description copied from interface: `Flow`
Method getFlowSkipStrategy returns the current `FlowSkipStrategy` used by this Flow.

Specified by:
  `getFlowSkipStrategy` in interface `Flow<Config>`
Returns:
  FlowSkipStrategy

#### setFlowSkipStrategy

    public FlowSkipStrategy setFlowSkipStrategy(FlowSkipStrategy flowSkipStrategy)

Description copied from interface: `Flow`
Method setFlowSkipStrategy sets a new `FlowSkipStrategy`, the current strategy is returned.

FlowSkipStrategy instances define when a Flow instance should be skipped. The default strategy is `FlowSkipIfSinkNotStale`. An alternative strategy would be `FlowSkipIfSinkExists`.

A FlowSkipStrategy will not be consulted when executing a Flow directly through `Flow.start()` or `Flow.complete()`. Only when the Flow is executed through a `Cascade` instance.

Specified by:
  `setFlowSkipStrategy` in interface `Flow<Config>`
Parameters:
  `flowSkipStrategy` - of type FlowSkipStrategy
Returns:
  FlowSkipStrategy

#### isSkipFlow

    public boolean isSkipFlow()
                       throws java.io.IOException

Description copied from interface: `Flow`
Method isSkipFlow returns true if the parent `Cascade` should skip this Flow instance. True is returned if the current `FlowSkipStrategy` returns true.

Specified by:
  `isSkipFlow` in interface `Flow<Config>`
Returns:
  the skipFlow (type boolean) of this Flow object.
Throws:
  `java.io.IOException` - when

#### areSinksStale

    public boolean areSinksStale()
                          throws java.io.IOException

Description copied from interface: `Flow`
Method areSinksStale returns true if any of the sinks referenced are out of date in relation to the sources. Or if any sink method `Tap.isReplace()` returns true.

Specified by:
  `areSinksStale` in interface `Flow<Config>`
Returns:
  boolean
Throws:
  `java.io.IOException` - when

#### areSourcesNewer

    public boolean areSourcesNewer(long sinkModified)
                            throws java.io.IOException

Description copied from interface: `Flow`
Method areSourcesNewer returns true if any source is newer than the given sinkModified date value.

Specified by:
  `areSourcesNewer` in interface `Flow<Config>`
Parameters:
  `sinkModified` - of type long
Returns:
  boolean
Throws:
  `java.io.IOException` - when

#### getSinkModified

    public long getSinkModified()
                         throws java.io.IOException

Description copied from interface: `Flow`
Method getSinkModified returns the youngest modified date of any sink `Tap` managed by this Flow instance.

If zero (0) is returned, at least one of the sink resources does not exist. If minus one (-1) is returned, atleast one of the sinks are marked for delete (`returns true`).

Specified by:
  `getSinkModified` in interface `Flow<Config>`
Returns:
  the sinkModified (type long) of this Flow object.
Throws:
  `java.io.IOException` - when

#### getFlowStepStrategy

    public FlowStepStrategy getFlowStepStrategy()

Description copied from interface: `Flow`
Returns the current `FlowStepStrategy` instance.

Specified by:
  `getFlowStepStrategy` in interface `Flow<Config>`
Returns:
  FlowStepStrategy

#### setFlowStepStrategy

    public void setFlowStepStrategy(FlowStepStrategy flowStepStrategy)

Description copied from interface: `Flow`
Sets a default `FlowStepStrategy` instance.

Use a FlowStepStrategy to change `FlowStep` configuration properties before the properties are submitted to the underlying platform for the step unit of work.

Specified by:
  `setFlowStepStrategy` in interface `Flow<Config>`
Parameters:
  `flowStepStrategy` - The FlowStepStrategy to use.

#### getFlowSteps

    public java.util.List<FlowStep<Config>> getFlowSteps()

Description copied from interface: `Flow`
Method getFlowSteps returns the flowSteps of this Flow object. They will be in topological order.

Specified by:
  `getFlowSteps` in interface `Flow<Config>`
Returns:
  the steps (type List<FlowStep>) of this Flow object.

#### prepare

    @ProcessPrepare
    public void prepare()

Description copied from interface: `Flow`
Method prepare is used by a `Cascade` to notify the given Flow it should initialize or clear any resources necessary for `Flow.start()` to be called successfully.

Specifically, this implementation calls `deleteSinksIfNotUpdate()` && `deleteTrapsIfNotUpdate()`.

Specified by:
  `prepare` in interface `Flow<Config>`
Specified by:
  `prepare` in interface `UnitOfWork<FlowStats>`

#### start

    @ProcessStart
    public void start()

Description copied from interface: `Flow`
Method start begins the execution of this Flow instance. It will return immediately. Use the method `Flow.complete()` to block until this Flow completes.

Specified by:
  `start` in interface `Flow<Config>`
Specified by:
  `start` in interface `UnitOfWork<FlowStats>`

#### createFlowThread

    protected java.lang.Thread createFlowThread(java.lang.String threadName)

#### internalStart

    protected abstract void internalStart()

#### stop

    @ProcessStop
    public void stop()

Description copied from interface: `Flow`
Method stop stops all running jobs, killing any currently executing.

Specified by:
  `stop` in interface `Flow<Config>`
Specified by:
  `stop` in interface `UnitOfWork<FlowStats>`

#### internalClean

    protected abstract void internalClean(boolean stop)

#### complete

    @ProcessComplete
    public void complete()

Description copied from interface: `Flow`
Method complete starts the current Flow instance if it has not be previously started, then block until completion.

Specified by:
  `complete` in interface `Flow<Config>`
Specified by:
  `complete` in interface `UnitOfWork<FlowStats>`

#### cleanup

    @ProcessCleanup
    public void cleanup()

Specified by:
  `cleanup` in interface `Flow<Config>`
Specified by:
  `cleanup` in interface `UnitOfWork<FlowStats>`

#### openSource

    public TupleEntryIterator openSource()
                                  throws java.io.IOException

Description copied from interface: `Flow`
Method openSource opens the first source Tap.

Specified by:
  `openSource` in interface `Flow<Config>`
Returns:
  TupleIterator
Throws:
  `java.io.IOException` - when

#### openSource

    public TupleEntryIterator openSource(java.lang.String name)
                                  throws java.io.IOException

Description copied from interface: `Flow`
Method openSource opens the named source Tap.

Specified by:
  `openSource` in interface `Flow<Config>`
Parameters:
  `name` - of type String
Returns:
  TupleIterator
Throws:
  `java.io.IOException` - when

#### openSink

    public TupleEntryIterator openSink()
                                throws java.io.IOException

Description copied from interface: `Flow`
Method openSink opens the first sink Tap.

Specified by:
  `openSink` in interface `Flow<Config>`
Returns:
  TupleIterator
Throws:
  `java.io.IOException` - when

#### openSink

    public TupleEntryIterator openSink(java.lang.String name)
                                throws java.io.IOException

Description copied from interface: `Flow`
Method openSink opens the named sink Tap.

Specified by:
  `openSink` in interface `Flow<Config>`
Parameters:
  `name` - of type String
Returns:
  TupleIterator
Throws:
  `java.io.IOException` - when

#### openTrap

    public TupleEntryIterator openTrap()
                                throws java.io.IOException

Description copied from interface: `Flow`
Method openTrap opens the first trap Tap.

Specified by:
  `openTrap` in interface `Flow<Config>`
Returns:
  TupleIterator
Throws:
  `java.io.IOException` - when

#### openTrap

    public TupleEntryIterator openTrap(java.lang.String name)
                                throws java.io.IOException

Description copied from interface: `Flow`
Method openTrap opens the named trap Tap.

Specified by:
  `openTrap` in interface `Flow<Config>`
Parameters:
  `name` - of type String
Returns:
  TupleIterator
Throws:
  `java.io.IOException` - when

#### deleteSinks

    public void deleteSinks()
                     throws java.io.IOException

Method deleteSinks deletes all sinks, whether or not they are configured for `SinkMode.UPDATE`.

Use with caution.

Throws:
  `java.io.IOException` - when
See Also:
  `deleteSinksIfNotUpdate()`

#### deleteSinksIfNotUpdate

    public void deleteSinksIfNotUpdate()
                                throws java.io.IOException

Method deleteSinksIfNotUpdate deletes all sinks if they are not configured with the `SinkMode.UPDATE` flag.

Typically used by a `Cascade` before executing the flow if the sinks are stale.

Use with caution.

Throws:
  `java.io.IOException` - when

#### deleteSinksIfReplace

    public void deleteSinksIfReplace()
                              throws java.io.IOException

Method deleteSinksIfReplace deletes all sinks that are configured with the `SinkMode.REPLACE` flag.

Throws:
  `java.io.IOException`

#### deleteTrapsIfNotUpdate

    public void deleteTrapsIfNotUpdate()
                                throws java.io.IOException

Throws:
  `java.io.IOException`

#### deleteCheckpointsIfNotUpdate

    public void deleteCheckpointsIfNotUpdate()
                                      throws java.io.IOException

Throws:
  `java.io.IOException`

#### deleteTrapsIfReplace

    public void deleteTrapsIfReplace()
                              throws java.io.IOException

Throws:
  `java.io.IOException`

#### deleteCheckpointsIfReplace

    public void deleteCheckpointsIfReplace()
                                    throws java.io.IOException

Throws:
  `java.io.IOException`

#### resourceExists

    public boolean resourceExists(Tap tap)
                           throws java.io.IOException

Description copied from interface: `Flow`
Method resourceExists returns true if the resource represented by the given Tap instance exists.

Specified by:
  `resourceExists` in interface `Flow<Config>`
Parameters:
  `tap` - of type Tap
Returns:
  boolean
Throws:
  `java.io.IOException` - when

#### openTapForRead

    public TupleEntryIterator openTapForRead(Tap tap)
                                      throws java.io.IOException

Description copied from interface: `Flow`
Method openTapForRead return a `TupleEntryIterator` for the given Tap instance.

Note the returned iterator will return the same instance of `TupleEntry` on every call, thus a copy must be made of either the TupleEntry or the underlying `Tuple` instance if they are to be stored in a Collection.

Specified by:
  `openTapForRead` in interface `Flow<Config>`
Parameters:
  `tap` - of type Tap
Returns:
  TupleIterator
Throws:
  `java.io.IOException` - when there is an error opening the resource

#### openTapForWrite

    public TupleEntryCollector openTapForWrite(Tap tap)
                                        throws java.io.IOException

Description copied from interface: `Flow`
Method openTapForWrite returns a (@link TupleCollector} for the given Tap instance.

Specified by:
  `openTapForWrite` in interface `Flow<Config>`
Parameters:
  `tap` - of type Tap
Returns:
  TupleCollector
Throws:
  `java.io.IOException` - when there is an error opening the resource

#### spawnSteps

    protected boolean spawnSteps()
                          throws java.lang.InterruptedException,
                                 java.util.concurrent.ExecutionException

Reentrant method to launch all step jobs, returns false if any of the current steps failed.

Returns:
Throws:
  `java.lang.InterruptedException`
  `java.util.concurrent.ExecutionException`

#### getTotalSliceCPUMilliSeconds

    protected long getTotalSliceCPUMilliSeconds()

#### getMaxNumParallelSteps

    protected abstract int getMaxNumParallelSteps()

#### internalShutdown

    protected abstract void internalShutdown()

#### isJobsMapInitialized

    protected boolean isJobsMapInitialized()

#### getEligibleJobsSize

    protected int getEligibleJobsSize()

#### getJobMapCallables

    protected java.util.List<java.util.concurrent.Callable<java.lang.Throwable>> getJobMapCallables()

#### initializeNewJobsMap

    protected void initializeNewJobsMap()

#### updateJobsMap

    protected void updateJobsMap()

#### initializeChildStats

    protected void initializeChildStats()

#### internalStopAllJobs

    protected void internalStopAllJobs()

#### handleExecutorShutdown

    protected void handleExecutorShutdown()

#### fireOnCompleted

    protected void fireOnCompleted()

#### fireOnThrowable

    protected void fireOnThrowable(java.lang.Throwable throwable)

#### fireOnThrowable

    protected void fireOnThrowable()

#### fireOnStopping

    protected void fireOnStopping()

#### fireOnStarting

    protected void fireOnStarting()

#### toString

    public java.lang.String toString()

Overrides:
  `toString` in class `java.lang.Object`

#### isInfoEnabled

    public final boolean isInfoEnabled()

Specified by:
  `isInfoEnabled` in interface `ProcessLogger`

#### isDebugEnabled

    public final boolean isDebugEnabled()

Specified by:
  `isDebugEnabled` in interface `ProcessLogger`

#### logInfo

    public void logInfo(java.lang.String message,
               java.lang.Object... arguments)

Specified by:
  `logInfo` in interface `ProcessLogger`

#### logDebug

    public void logDebug(java.lang.String message,
                java.lang.Object... arguments)

Specified by:
  `logDebug` in interface `ProcessLogger`

#### logWarn

    public void logWarn(java.lang.String message)

Specified by:
  `logWarn` in interface `ProcessLogger`

#### logWarn

    public void logWarn(java.lang.String message,
               java.lang.Throwable throwable)

Specified by:
  `logWarn` in interface `ProcessLogger`

#### logWarn

    public void logWarn(java.lang.String message,
               java.lang.Object... arguments)

Specified by:
  `logWarn` in interface `ProcessLogger`

#### logError

    public void logError(java.lang.String message,
                java.lang.Object... arguments)

Specified by:
  `logError` in interface `ProcessLogger`

#### logError

    public void logError(java.lang.String message,
                java.lang.Throwable throwable)

Specified by:
  `logError` in interface `ProcessLogger`

#### writeDOT

    public void writeDOT(java.lang.String filename)

Description copied from interface: `Flow`
Method writeDOT writes this Flow instance to the given filename as a DOT file for import into a graphics package.

Specified by:
  `writeDOT` in interface `Flow<Config>`
Parameters:
  `filename` - of type String

#### writeStepsDOT

    public void writeStepsDOT(java.lang.String filename)

Description copied from interface: `Flow`
Method writeStepsDOT writes this Flow step graph to the given filename as a DOT file for import into a graphics package.

Specified by:
  `writeStepsDOT` in interface `Flow<Config>`
Parameters:
  `filename` - of type String

#### getHolder

    public BaseFlow.FlowHolder getHolder()

Used to return a simple wrapper for use as an edge in a graph where there can only be one instance of every edge.

Returns:
  FlowHolder

#### setCascade

    public void setCascade(Cascade cascade)

#### getCascadeID

    public java.lang.String getCascadeID()

Description copied from interface: `Flow`
Returns the parent Cascade ID that owns this Flow instance.

Specified by:
  `getCascadeID` in interface `Flow<Config>`
Returns:
  of type String

#### getRunID

    public java.lang.String getRunID()

Description copied from interface: `Flow`
Returns the run ID given when this Flow instance was defined in the FlowDef.

Specified by:
  `getRunID` in interface `Flow<Config>`
Returns:
  of type String

#### getClassPath

    public java.util.List<java.lang.String> getClassPath()

#### setSpawnStrategy

    public void setSpawnStrategy(UnitOfWorkSpawnStrategy spawnStrategy)

Specified by:
  `setSpawnStrategy` in interface `UnitOfWork<FlowStats>`

#### getSpawnStrategy

    public UnitOfWorkSpawnStrategy getSpawnStrategy()

Specified by:
  `getSpawnStrategy` in interface `UnitOfWork<FlowStats>`

#### registerShutdownHook

    protected void registerShutdownHook()