cascading.flow

## Class FlowRuntimeProps

- java.lang.Object
- - cascading.property.Props
  - - cascading.flow.FlowRuntimeProps

-     public class FlowRuntimeProps
      extends Props

  Class FlowRuntimeProps is a fluent helper class for setting `Flow` specific runtime properties through a `FlowConnector`.

  These properties apply to the cluster or remote side of the Flow execution. For client (or local) side properties see `FlowProps`.

  Available properties are:

  - gather partitions - number of slices (partitions) to gather keys within each `FlowNode`. In MapReduce this is the number of reducers. In Tez DAG this is the scatter gather parallelization.
  - log counters - counter names to log to INFO when a cluster side slice completes.

  Note, if the num of gather partitions is not set, the Flow may fail during planning or setup, depending on the platform.

- - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `static java.lang.String` | `COMBINE_SPLITS`
    `static java.lang.String` | `GATHER_PARTITIONS`
    `static java.lang.String` | `LOG_COUNTERS`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `FlowRuntimeProps()`

  - ### Method Summary

    Methods
    Modifier and Type | Method and Description
    --- | ---
    `FlowRuntimeProps` | `addLogCounter(java.lang.Enum counter)` Method addLogCounter adds a new counter to be logged when a cluster side slice completes.
    `FlowRuntimeProps` | `addLogCounter(java.lang.String group, java.lang.String counter)` Method addLogCounter adds a new counter to be logged when a cluster side slice completes.
    `protected void` | `addPropertiesTo(java.util.Properties properties)`
    `static FlowRuntimeProps` | `flowRuntimeProps()`
    `java.lang.Boolean` | `getCombineSplits()`
    `int` | `getGatherPartitions()` Method getGatherPartitions returns the number of gather partitions
    `FlowRuntimeProps` | `setCombineSplits(java.lang.Boolean combineSplits)` Method setCombineSplits will enable or disable combining of 'splits' on sources.
    `FlowRuntimeProps` | `setGatherPartitions(int gatherPartitions)` Method setGatherPartitions sets the default number of gather partitions each `FlowNode` should use.

    - ### Methods inherited from class cascading.property.Props

      `buildProperties, buildProperties, buildProperties, buildProperties, setProperties, setProperties`

    - ### Methods inherited from class java.lang.Object

      `clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

- - ### Field Detail

    - #### GATHER_PARTITIONS

          public static final java.lang.String GATHER_PARTITIONS

      See Also:
        Constant Field Values

    - #### LOG_COUNTERS

          public static final java.lang.String LOG_COUNTERS

      See Also:
        Constant Field Values

    - #### COMBINE_SPLITS

          public static final java.lang.String COMBINE_SPLITS

      See Also:
        Constant Field Values

  - ### Constructor Detail

    - #### FlowRuntimeProps

          public FlowRuntimeProps()

  - ### Method Detail

    - #### flowRuntimeProps

          public static FlowRuntimeProps flowRuntimeProps()

    - #### getGatherPartitions

          public int getGatherPartitions()

      Method getGatherPartitions returns the number of gather partitions
      Returns:
        number of gather partitions

    - #### setGatherPartitions

          public FlowRuntimeProps setGatherPartitions(int gatherPartitions)

      Method setGatherPartitions sets the default number of gather partitions each `FlowNode` should use.
      Parameters:
        `gatherPartitions` - number of gather partitions to use per node
      Returns:
        this

    - #### addLogCounter

          public FlowRuntimeProps addLogCounter(java.lang.Enum counter)

      Method addLogCounter adds a new counter to be logged when a cluster side slice completes.

      The given counters will be logged using the default cluster side logging mechanism.
      Parameters:
        `counter` - the Enum counter to log
      Returns:
        this

    - #### addLogCounter

          public FlowRuntimeProps addLogCounter(java.lang.String group,
                                       java.lang.String counter)

      Method addLogCounter adds a new counter to be logged when a cluster side slice completes.

      The given counters will be logged using the default cluster side logging mechanism.
      Parameters:
        `group` - the String counter group to log
        `counter` - the String counter name to log
      Returns:
        this

    - #### getCombineSplits

          public java.lang.Boolean getCombineSplits()

    - #### setCombineSplits

          public FlowRuntimeProps setCombineSplits(java.lang.Boolean combineSplits)

      Method setCombineSplits will enable or disable combining of 'splits' on sources.

      A split is a sub-set of data from a `Tap` source resource. Combining small splits into larger ones both reduce parallelism, but also reduce overhead of starting work on a very small data set.

      This is commonly done when sourcing large numbers of very small files.

      Setting this value will change the default, which is a platform dependent value.
      Parameters:
        `combineSplits` -
      Returns:

    - #### addPropertiesTo

          protected void addPropertiesTo(java.util.Properties properties)

      Specified by:
        `addPropertiesTo` in class `Props`

Copyright © 2007-2015 Xplenty, Inc. All Rights Reserved.