cascading.tap.hadoop

## Class GlobHfs

- java.lang.Object
- - cascading.tap.Tap<Config,Input,java.lang.Void>
  - - cascading.tap.SourceTap<Config,Input>
    - - cascading.tap.MultiSourceTap<Hfs,org.apache.hadoop.conf.Configuration,org.apache.hadoop.mapred.RecordReader>
      - - cascading.tap.hadoop.GlobHfs

All Implemented Interfaces:
  FlowElement, ScopedElement, CompositeTap<Hfs>, Traceable, java.io.Serializable

    public class GlobHfs
    extends MultiSourceTap<Hfs,org.apache.hadoop.conf.Configuration,org.apache.hadoop.mapred.RecordReader>

Class GlobHfs is a type of `MultiSourceTap` that accepts Hadoop style 'file globing' expressions so multiple files that match the given pattern may be used as the input sources for a given `Flow`.

See `FileSystem.globStatus(org.apache.hadoop.fs.Path)` for details on the globing syntax. But in short it is similar to standard regular expressions except alternation is done via {foo,bar} instead of (foo|bar).

Note that a `Flow` sourcing from GlobHfs is not currently compatible with the `Cascade` scheduler. GlobHfs expects the files and paths to exist so the wildcards can be resolved into concrete values so that the scheduler can order the Flows properly.

Note that globing can match files or directories. It may consume less resources to match directories and let Hadoop include all sub-files immediately contained in the directory instead of enumerating every individual file. Ending the glob path with a `/` should match only directories.

See Also:
  `Hfs`, `MultiSourceTap`, `FileSystem`, Serialized Form

### Field Summary

### Fields inherited from class cascading.tap.MultiSourceTap

`commonPrefix, prefixMap, taps`

### Constructor Summary

Constructors
Constructor and Description
`GlobHfs(Scheme<org.apache.hadoop.conf.Configuration,org.apache.hadoop.mapred.RecordReader,?,?,?> scheme, java.lang.String pathPattern)` Constructor GlobHfs creates a new GlobHfs instance.
`GlobHfs(Scheme<org.apache.hadoop.conf.Configuration,org.apache.hadoop.mapred.RecordReader,?,?,?> scheme, java.lang.String pathPattern, org.apache.hadoop.fs.PathFilter pathFilter)` Constructor GlobHfs creates a new GlobHfs instance.

### Method Summary

Methods
`boolean` `equals(java.lang.Object object)`
`java.lang.String` `getIdentifier()` Method getIdentifier returns a String representing the resource this Tap instance represents.
`protected Hfs[]` `getTaps()` Method getTaps returns the taps of this MultiTap object.
`int` `hashCode()`
`void` `sourceConfInit(FlowProcess<? extends org.apache.hadoop.conf.Configuration> process, org.apache.hadoop.conf.Configuration conf)` Method sourceConfInit initializes this instance as a source.
`java.lang.String` `toString()`

### Methods inherited from class cascading.tap.MultiSourceTap

`findMatchingTap, getChildTaps, getModifiedTime, getNumChildTaps, getScheme, getTapPrefixMap, getTapsCommonPrefix, isReplace, openForRead, resourceExists`

### Methods inherited from class cascading.tap.SourceTap

`commitResource, createResource, deleteResource, getSinkFields, isSink, openForWrite, prepareResourceForWrite, rollbackResource, sinkConfInit`

### Methods inherited from class cascading.tap.Tap

`createResource, deleteResource, flowConfInit, getConfigDef, getFullIdentifier, getFullIdentifier, getModifiedTime, getNodeConfigDef, getSinkMode, getSourceFields, getStepConfigDef, getTrace, hasConfigDef, hasNodeConfigDef, hasStepConfigDef, id, isKeep, isSource, isTemporary, isUpdate, openForRead, openForWrite, outgoingScopeFor, prepareResourceForRead, presentSinkFields, presentSourceFields, resolveIncomingOperationArgumentFields, resolveIncomingOperationPassThroughFields, resourceExists, retrieveSinkFields, retrieveSourceFields, setScheme, taps`

### Methods inherited from class java.lang.Object

`clone, finalize, getClass, notify, notifyAll, wait, wait, wait`

### Constructor Detail

#### GlobHfs

    @ConstructorProperties(value={"scheme","pathPattern"})
    public GlobHfs(Scheme<org.apache.hadoop.conf.Configuration,org.apache.hadoop.mapred.RecordReader,?,?,?> scheme,
                                 java.lang.String pathPattern)

Constructor GlobHfs creates a new GlobHfs instance.

Parameters:
  `scheme` - of type Scheme
  `pathPattern` - of type String

#### GlobHfs

    @ConstructorProperties(value={"scheme","pathPattern","pathFilter"})
    public GlobHfs(Scheme<org.apache.hadoop.conf.Configuration,org.apache.hadoop.mapred.RecordReader,?,?,?> scheme,
                                 java.lang.String pathPattern,
                                 org.apache.hadoop.fs.PathFilter pathFilter)

Constructor GlobHfs creates a new GlobHfs instance.

Parameters:
  `scheme` - of type Scheme
  `pathPattern` - of type String
  `pathFilter` - of type PathFilter

### Method Detail

#### getIdentifier

    public java.lang.String getIdentifier()

Description copied from class: `Tap`
Method getIdentifier returns a String representing the resource this Tap instance represents.

Often, if the tap accesses a filesystem, the identifier is nothing more than the path to the file or directory. In other cases it may be a an URL or URI representing a connection string or remote resource.

Any two Tap instances having the same value for the identifier are considered equal.

Overrides:
  `getIdentifier` in class `MultiSourceTap<Hfs,org.apache.hadoop.conf.Configuration,org.apache.hadoop.mapred.RecordReader>`
Returns:
  String

#### getTaps

    protected Hfs[] getTaps()

Description copied from class: `MultiSourceTap`
Method getTaps returns the taps of this MultiTap object.

Overrides:
  `getTaps` in class `MultiSourceTap<Hfs,org.apache.hadoop.conf.Configuration,org.apache.hadoop.mapred.RecordReader>`
Returns:
  the taps (type Tap[]) of this MultiTap object.

#### sourceConfInit

    public void sourceConfInit(FlowProcess<? extends org.apache.hadoop.conf.Configuration> process,
                      org.apache.hadoop.conf.Configuration conf)

Description copied from class: `Tap`
Method sourceConfInit initializes this instance as a source.

This method maybe called more than once if this Tap instance is used outside the scope of a `Flow` instance or if it participates in multiple times in a given Flow or across different Flows in a `Cascade`.

In the context of a Flow, it will be called after `FlowListener.onStarting(cascading.flow.Flow)`

Note that no resources or services should be modified by this method.

Overrides:
  `sourceConfInit` in class `MultiSourceTap<Hfs,org.apache.hadoop.conf.Configuration,org.apache.hadoop.mapred.RecordReader>`
Parameters:
  `process` - of type FlowProcess
  `conf` - of type Config

#### equals

    public boolean equals(java.lang.Object object)

Overrides:
  `equals` in class `MultiSourceTap<Hfs,org.apache.hadoop.conf.Configuration,org.apache.hadoop.mapred.RecordReader>`

#### hashCode

    public int hashCode()

Overrides:
  `hashCode` in class `MultiSourceTap<Hfs,org.apache.hadoop.conf.Configuration,org.apache.hadoop.mapred.RecordReader>`

#### toString

    public java.lang.String toString()

Overrides:
  `toString` in class `MultiSourceTap<Hfs,org.apache.hadoop.conf.Configuration,org.apache.hadoop.mapred.RecordReader>`