cascading.tap

## Class MultiSourceTap<Child extends Tap,Config,Input>

- java.lang.Object
- - cascading.tap.Tap<Config,Input,java.lang.Void>
  - - cascading.tap.SourceTap<Config,Input>
    - - cascading.tap.MultiSourceTap<Child,Config,Input>

- All Implemented Interfaces:
    FlowElement, ScopedElement, CompositeTap<Child>, Traceable, java.io.Serializable
  Direct Known Subclasses:
    GlobHfs

      public class MultiSourceTap<Child extends Tap,Config,Input>
      extends SourceTap<Config,Input>
      implements CompositeTap<Child>

  Class MultiSourceTap is used to tie multiple `Tap` instances into a single resource. Effectively this will allow multiple files to be concatenated into the requesting pipe assembly, if they all share the same `Scheme` instance.

  Note that order is not maintained by virtue of the underlying model. If order is necessary, use a unique sequence key to span the resources, like a line number.

  Note that if multiple input files have the same Scheme (like `TextLine`), they may not contain the same semi-structure internally. For example, one file might be an Apache log file, and another might be a Log4J log file. If each one should be parsed differently, then they must be handled by different pipe assembly branches.

  As of 3.3, MultiSourceTap can aggregate `cascading.scheme.hadoop.PartitionTap` and `cascading.scheme.local.PartitionTap` instances.

  But it may not be safe to aggregate `cascading.scheme.hadoop.GlobHfs` and `cascading.scheme.hadoop.PartitionTap` instances as GlobHfs identifiers cannot be fully resolved, preventing the cluster side runtime from distinguishing which Tap instance to open for reading.

  See Also:
    Serialized Form

- - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `protected java.lang.String` | `commonPrefix`
    `protected Trie<Child>` | `prefixMap`
    `protected Child[]` | `taps`

  - ### Constructor Summary

    Constructors
    Modifier | Constructor and Description
    --- | ---
    `` | `MultiSourceTap(Child... taps)` Constructor MultiSourceTap creates a new MultiSourceTap instance.
    `protected` | `MultiSourceTap(Scheme<Config,Input,?,?,?> scheme)`

  - ### Method Summary

    Methods
    Modifier and Type | Method and Description
    --- | ---
    `boolean` | `equals(java.lang.Object object)`
    `protected Child` | `findMatchingTap(FlowProcess<? extends Config> flowProcess)`
    `java.util.Iterator<Child>` | `getChildTaps()`
    `java.lang.String` | `getIdentifier()` Method getIdentifier returns a String representing the resource this Tap instance represents.
    `long` | `getModifiedTime(Config conf)` Returns the most current modified time.
    `long` | `getNumChildTaps()`
    `Scheme` | `getScheme()` Method getScheme returns the scheme of this Tap object.
    `protected Trie<Child>` | `getTapPrefixMap(FlowProcess<? extends Config> flowProcess)`
    `protected Child[]` | `getTaps()` Method getTaps returns the taps of this MultiTap object.
    `protected java.lang.String` | `getTapsCommonPrefix(FlowProcess<? extends Config> flowProcess)`
    `int` | `hashCode()`
    `boolean` | `isReplace()` Method isReplace indicates whether the resource represented by this instance should be deleted if it already exists when the Flow is started.
    `TupleEntryIterator` | `openForRead(FlowProcess<? extends Config> flowProcess, Input input)` Method openForRead opens the resource represented by this Tap instance for reading.
    `boolean` | `resourceExists(Config conf)` Method resourceExists returns true if the path represented by this instance exists.
    `void` | `sourceConfInit(FlowProcess<? extends Config> process, Config conf)` Method sourceConfInit initializes this instance as a source.
    `java.lang.String` | `toString()`

    - ### Methods inherited from class cascading.tap.SourceTap

      `commitResource, createResource, deleteResource, getSinkFields, isSink, openForWrite, prepareResourceForWrite, rollbackResource, sinkConfInit`

    - ### Methods inherited from class cascading.tap.Tap

      `createResource, deleteResource, flowConfInit, getConfigDef, getFullIdentifier, getFullIdentifier, getModifiedTime, getNodeConfigDef, getSinkMode, getSourceFields, getStepConfigDef, getTrace, hasConfigDef, hasNodeConfigDef, hasStepConfigDef, id, isKeep, isSource, isTemporary, isUpdate, openForRead, openForWrite, outgoingScopeFor, prepareResourceForRead, presentSinkFields, presentSourceFields, resolveIncomingOperationArgumentFields, resolveIncomingOperationPassThroughFields, resourceExists, retrieveSinkFields, retrieveSourceFields, setScheme, taps`

    - ### Methods inherited from class java.lang.Object

      `clone, finalize, getClass, notify, notifyAll, wait, wait, wait`

- - ### Field Detail

    - #### taps

          protected Child extends Tap[] taps

    - #### prefixMap

          protected transient Trie<Child extends Tap> prefixMap

    - #### commonPrefix

          protected transient java.lang.String commonPrefix

  - ### Constructor Detail

    - #### MultiSourceTap

          protected MultiSourceTap(Scheme<Config,Input,?,?,?> scheme)

    - #### MultiSourceTap

          @ConstructorProperties(value="taps")
          public MultiSourceTap(Child... taps)

      Constructor MultiSourceTap creates a new MultiSourceTap instance.
      Parameters:
        `taps` - of type Tap...

  - ### Method Detail

    - #### getTaps

          protected Child[] getTaps()

      Method getTaps returns the taps of this MultiTap object.
      Returns:
        the taps (type Tap[]) of this MultiTap object.

    - #### getChildTaps

          public java.util.Iterator<Child> getChildTaps()

      Specified by:
        `getChildTaps` in interface `CompositeTap<Child extends Tap>`

    - #### getNumChildTaps

          public long getNumChildTaps()

      Specified by:
        `getNumChildTaps` in interface `CompositeTap<Child extends Tap>`

    - #### getIdentifier

          public java.lang.String getIdentifier()

      Description copied from class: `Tap`
      Method getIdentifier returns a String representing the resource this Tap instance represents.

      Often, if the tap accesses a filesystem, the identifier is nothing more than the path to the file or directory. In other cases it may be a an URL or URI representing a connection string or remote resource.

      Any two Tap instances having the same value for the identifier are considered equal.
      Specified by:
        `getIdentifier` in class `Tap<Config,Input,java.lang.Void>`
      Returns:
        String

    - #### getScheme

          public Scheme getScheme()

      Description copied from class: `Tap`
      Method getScheme returns the scheme of this Tap object.
      Overrides:
        `getScheme` in class `Tap<Config,Input,java.lang.Void>`
      Returns:
        the scheme (type Scheme) of this Tap object.

    - #### isReplace

          public boolean isReplace()

      Description copied from class: `Tap`
      Method isReplace indicates whether the resource represented by this instance should be deleted if it already exists when the Flow is started.
      Overrides:
        `isReplace` in class `Tap<Config,Input,java.lang.Void>`
      Returns:
        boolean

    - #### sourceConfInit

          public void sourceConfInit(FlowProcess<? extends Config> process,
                            Config conf)

      Description copied from class: `Tap`
      Method sourceConfInit initializes this instance as a source.

      This method maybe called more than once if this Tap instance is used outside the scope of a `Flow` instance or if it participates in multiple times in a given Flow or across different Flows in a `Cascade`.

      In the context of a Flow, it will be called after `FlowListener.onStarting(cascading.flow.Flow)`

      Note that no resources or services should be modified by this method.
      Overrides:
        `sourceConfInit` in class `Tap<Config,Input,java.lang.Void>`
      Parameters:
        `process` - of type FlowProcess
        `conf` - of type Config

    - #### resourceExists

          public boolean resourceExists(Config conf)
                                 throws java.io.IOException

      Description copied from class: `Tap`
      Method resourceExists returns true if the path represented by this instance exists.
      Specified by:
        `resourceExists` in class `Tap<Config,Input,java.lang.Void>`
      Parameters:
        `conf` - of type Config
      Returns:
        true if the underlying resource already exists
      Throws:
        `java.io.IOException` - when the status cannot be determined

    - #### getModifiedTime

          public long getModifiedTime(Config conf)
                               throws java.io.IOException

      Returns the most current modified time.
      Specified by:
        `getModifiedTime` in class `Tap<Config,Input,java.lang.Void>`
      Parameters:
        `conf` - of type Config
      Returns:
        The date this resource was last modified.
      Throws:
        `java.io.IOException`

    - #### openForRead

          public TupleEntryIterator openForRead(FlowProcess<? extends Config> flowProcess,
                                       Input input)
                                         throws java.io.IOException

      Description copied from class: `Tap`
      Method openForRead opens the resource represented by this Tap instance for reading.

      `input` value may be null, if so, sub-classes must inquire with the underlying `Scheme` via `Scheme.sourceConfInit(cascading.flow.FlowProcess, Tap, Object)` to get the proper input type and instantiate it before calling `super.openForRead()`.

      Note the returned iterator will return the same instance of `TupleEntry` on every call, thus a copy must be made of either the TupleEntry or the underlying `Tuple` instance if they are to be stored in a Collection.
      Specified by:
        `openForRead` in class `Tap<Config,Input,java.lang.Void>`
      Parameters:
        `flowProcess` - of type FlowProcess
        `input` - of type Input
      Returns:
        TupleEntryIterator
      Throws:
        `java.io.IOException` - when the resource cannot be opened

    - #### findMatchingTap

          protected Child findMatchingTap(FlowProcess<? extends Config> flowProcess)

    - #### getTapsCommonPrefix

          protected java.lang.String getTapsCommonPrefix(FlowProcess<? extends Config> flowProcess)

    - #### getTapPrefixMap

          protected Trie<Child> getTapPrefixMap(FlowProcess<? extends Config> flowProcess)

    - #### equals

          public boolean equals(java.lang.Object object)

      Overrides:
        `equals` in class `Tap<Config,Input,java.lang.Void>`

    - #### hashCode

          public int hashCode()

      Overrides:
        `hashCode` in class `Tap<Config,Input,java.lang.Void>`

    - #### toString

          public java.lang.String toString()

      Overrides:
        `toString` in class `Tap<Config,Input,java.lang.Void>`

Copyright © 2007-2015 Xplenty, Inc. All Rights Reserved.