cascading.tap.local

## Class DirTap

- java.lang.Object
- - cascading.tap.Tap<Properties,InputStream,OutputStream>
  - - cascading.tap.local.FileTap
    - - cascading.tap.local.DirTap

- All Implemented Interfaces:
    cascading.flow.FlowElement, cascading.flow.planner.ScopedElement, cascading.tap.type.FileType<Properties>, cascading.util.Traceable, Serializable

      public class DirTap
      extends FileTap

  Class DirTap processes all files in the given directory that match the given glob or regex.

  A DirTap can be used as a sink or a source.

  When used as a source, the given pattern and depth are used to identify input files from the filesystem.

  When used as a sink, a single file is created in the directory for all the output data. The file is name is returned by `getOutputIdentifier()` and can be overridden (see `getOutputFilename()` and `getOutputFileBasename()`).

  When deleting the resource identified by this Tap, the value of `getOutputIdentifier()` will be deleted, if it exists.

  DirTap must be used with the `LocalFlowConnector` to create `Flow` instances that run in "local" mode.

  The given pattern must adhere to the syntax supported by `FileSystem.getPathMatcher(String)`.

  Or a sub-class may override `getPathMatcher()` and return a custom matcher.

  The maxDepth parameter is the maximum number of levels of directories to visit. A value of 0 means that only the starting file is visited, unless denied by the security manager.

  A value of MAX_VALUE (the default) may be used to indicate that all levels should be visited.

  See Also:
    Serialized Form

- - ### Field Summary

    - ### Fields inherited from interface cascading.tap.type.FileType

      `CASCADING_SOURCE_PATH`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `DirTap(cascading.scheme.Scheme<Properties,InputStream,OutputStream,?,?> scheme, Path directory)` Constructor DirTap creates a new DirTap instance using the given `Scheme` and file `directory`.
    `DirTap(cascading.scheme.Scheme<Properties,InputStream,OutputStream,?,?> scheme, Path directory, cascading.tap.SinkMode sinkMode)` Constructor DirTap creates a new DirTap instance using the given `Scheme` and file `directory`.
    `DirTap(cascading.scheme.Scheme<Properties,InputStream,OutputStream,?,?> scheme, Path directory, String pattern)` Constructor DirTap creates a new DirTap instance using the given `Scheme`, file `directory`, and `pattern`.
    `DirTap(cascading.scheme.Scheme<Properties,InputStream,OutputStream,?,?> scheme, Path directory, String pattern, int maxDepth)` Constructor DirTap creates a new DirTap instance using the given `Scheme`, file `directory`, `pattern`, and `maxDepth`
    `DirTap(cascading.scheme.Scheme<Properties,InputStream,OutputStream,?,?> scheme, Path directory, String pattern, int maxDepth, cascading.tap.SinkMode sinkMode)` Constructor DirTap creates a new DirTap instance using the given `Scheme`, file `directory`, `pattern`, and `maxDepth`
    `DirTap(cascading.scheme.Scheme<Properties,InputStream,OutputStream,?,?> scheme, Path directory, String pattern, cascading.tap.SinkMode sinkMode)` Constructor DirTap creates a new DirTap instance using the given `Scheme`, file `directory`, and `pattern`.
    `DirTap(cascading.scheme.Scheme<Properties,InputStream,OutputStream,?,?> scheme, String directory)` Constructor DirTap creates a new DirTap instance using the given `Scheme` and file `directory`.
    `DirTap(cascading.scheme.Scheme<Properties,InputStream,OutputStream,?,?> scheme, String directory, cascading.tap.SinkMode sinkMode)` Constructor DirTap creates a new DirTap instance using the given `Scheme` and file `directory`.
    `DirTap(cascading.scheme.Scheme<Properties,InputStream,OutputStream,?,?> scheme, String directory, String pattern)` Constructor DirTap creates a new DirTap instance using the given `Scheme`, file `directory`, and `pattern`.
    `DirTap(cascading.scheme.Scheme<Properties,InputStream,OutputStream,?,?> scheme, String directory, String pattern, int maxDepth)` Constructor DirTap creates a new DirTap instance using the given `Scheme`, file `directory`, `pattern`, and `maxDepth`
    `DirTap(cascading.scheme.Scheme<Properties,InputStream,OutputStream,?,?> scheme, String directory, String pattern, int maxDepth, cascading.tap.SinkMode sinkMode)` Constructor DirTap creates a new DirTap instance using the given `Scheme`, file `directory`, `pattern`, and `maxDepth`
    `DirTap(cascading.scheme.Scheme<Properties,InputStream,OutputStream,?,?> scheme, String directory, String pattern, cascading.tap.SinkMode sinkMode)` Constructor DirTap creates a new DirTap instance using the given `Scheme`, file `directory`, and `pattern`.

  - ### Method Summary

    All Methods Static Methods Instance Methods Concrete Methods
    Modifier and Type | Method and Description
    --- | ---
    `static boolean` | `deleteDirTap(DirTap dirTap, Properties conf)` Method deleteDirTap will recursively delete all files referenced by the given DirTap.
    `boolean` | `deleteResource(Properties conf)`
    `String[]` | `getChildIdentifiers(Properties conf, int depth, boolean fullyQualified)`
    `int` | `getMaxDepth()`
    `protected String` | `getOutputFileBasename()`
    `String` | `getOutputFilename()`
    `String` | `getOutputIdentifier()` Only used with `FileTap.openForWrite(FlowProcess, OutputStream)` calls.
    `protected PathMatcher` | `getPathMatcher()`
    `String` | `getPattern()`
    `cascading.tuple.TupleEntryIterator` | `openForRead(cascading.flow.FlowProcess<? extends Properties> flowProcess, InputStream input)`
    `protected void` | `verify()`

    - ### Methods inherited from class cascading.tap.local.FileTap

      `commitResource, createResource, getChildIdentifiers, getChildIdentifiers, getChildIdentifiers, getFullIdentifier, getIdentifier, getModifiedTime, getPath, getSize, getSize, isDirectory, isDirectory, openForWrite, resourceExists`

    - ### Methods inherited from class cascading.tap.Tap

      `createResource, deleteResource, entryStream, entryStream, entryStreamCopy, entryStreamCopy, equals, flowConfInit, getConfigDef, getFullIdentifier, getModifiedTime, getNodeConfigDef, getScheme, getSinkFields, getSinkMode, getSourceFields, getStepConfigDef, getTrace, hasConfigDef, hashCode, hasNodeConfigDef, hasStepConfigDef, id, isKeep, isReplace, isSink, isSource, isTemporary, isUpdate, openForRead, openForReadUnchecked, openForWrite, outgoingScopeFor, prepareResourceForRead, prepareResourceForWrite, presentSinkFields, presentSourceFields, resolveIncomingOperationArgumentFields, resolveIncomingOperationPassThroughFields, resourceExists, retrieveSinkFields, retrieveSourceFields, rollbackResource, setScheme, sinkConfInit, sourceConfInit, spliterator, splititerator, taps, toString, tupleStream, tupleStream, tupleStreamCopy, tupleStreamCopy`

    - ### Methods inherited from class java.lang.Object

      `clone, finalize, getClass, notify, notifyAll, wait, wait, wait`

- - ### Constructor Detail

    - #### DirTap

          public DirTap(cascading.scheme.Scheme<Properties,InputStream,OutputStream,?,?> scheme,
                        String directory)

      Constructor DirTap creates a new DirTap instance using the given `Scheme` and file `directory`.
      Parameters:
        `scheme` - of type Scheme
        `directory` - of type String

    - #### DirTap

          public DirTap(cascading.scheme.Scheme<Properties,InputStream,OutputStream,?,?> scheme,
                        String directory,
                        String pattern)

      Constructor DirTap creates a new DirTap instance using the given `Scheme`, file `directory`, and `pattern`.
      Parameters:
        `scheme` - of type Scheme
        `directory` - of type String
        `pattern` - of type String

    - #### DirTap

          public DirTap(cascading.scheme.Scheme<Properties,InputStream,OutputStream,?,?> scheme,
                        String directory,
                        String pattern,
                        int maxDepth)

      Constructor DirTap creates a new DirTap instance using the given `Scheme`, file `directory`, `pattern`, and `maxDepth`
      Parameters:
        `scheme` - of type Scheme
        `directory` - of type String
        `pattern` - of type String
        `maxDepth` - of type int

    - #### DirTap

          public DirTap(cascading.scheme.Scheme<Properties,InputStream,OutputStream,?,?> scheme,
                        String directory,
                        cascading.tap.SinkMode sinkMode)

      Constructor DirTap creates a new DirTap instance using the given `Scheme` and file `directory`.
      Parameters:
        `scheme` - of type Scheme
        `directory` - of type String
        `sinkMode` - of type SinkMode

    - #### DirTap

          public DirTap(cascading.scheme.Scheme<Properties,InputStream,OutputStream,?,?> scheme,
                        String directory,
                        String pattern,
                        cascading.tap.SinkMode sinkMode)

      Constructor DirTap creates a new DirTap instance using the given `Scheme`, file `directory`, and `pattern`.
      Parameters:
        `scheme` - of type Scheme
        `directory` - of type String
        `pattern` - of type String
        `sinkMode` - of type SinkMode

    - #### DirTap

          public DirTap(cascading.scheme.Scheme<Properties,InputStream,OutputStream,?,?> scheme,
                        String directory,
                        String pattern,
                        int maxDepth,
                        cascading.tap.SinkMode sinkMode)

      Constructor DirTap creates a new DirTap instance using the given `Scheme`, file `directory`, `pattern`, and `maxDepth`
      Parameters:
        `scheme` - of type Scheme
        `directory` - of type String
        `pattern` - of type String
        `maxDepth` - of type int
        `sinkMode` - of type SinkMode

    - #### DirTap

          public DirTap(cascading.scheme.Scheme<Properties,InputStream,OutputStream,?,?> scheme,
                        Path directory)

      Constructor DirTap creates a new DirTap instance using the given `Scheme` and file `directory`.
      Parameters:
        `scheme` - of type Scheme
        `directory` - of type Path

    - #### DirTap

          public DirTap(cascading.scheme.Scheme<Properties,InputStream,OutputStream,?,?> scheme,
                        Path directory,
                        String pattern)

      Constructor DirTap creates a new DirTap instance using the given `Scheme`, file `directory`, and `pattern`.
      Parameters:
        `scheme` - of type Scheme
        `directory` - of type Path
        `pattern` - of type String

    - #### DirTap

          public DirTap(cascading.scheme.Scheme<Properties,InputStream,OutputStream,?,?> scheme,
                        Path directory,
                        String pattern,
                        int maxDepth)

      Constructor DirTap creates a new DirTap instance using the given `Scheme`, file `directory`, `pattern`, and `maxDepth`
      Parameters:
        `scheme` - of type Scheme
        `directory` - of type Path
        `pattern` - of type String
        `maxDepth` - of type int

    - #### DirTap

          public DirTap(cascading.scheme.Scheme<Properties,InputStream,OutputStream,?,?> scheme,
                        Path directory,
                        cascading.tap.SinkMode sinkMode)

      Constructor DirTap creates a new DirTap instance using the given `Scheme` and file `directory`.
      Parameters:
        `scheme` - of type Scheme
        `directory` - of type Path
        `sinkMode` - of type SinkMode

    - #### DirTap

          public DirTap(cascading.scheme.Scheme<Properties,InputStream,OutputStream,?,?> scheme,
                        Path directory,
                        String pattern,
                        cascading.tap.SinkMode sinkMode)

      Constructor DirTap creates a new DirTap instance using the given `Scheme`, file `directory`, and `pattern`.
      Parameters:
        `scheme` - of type Scheme
        `directory` - of type Path
        `pattern` - of type String
        `sinkMode` - of type SinkMode

    - #### DirTap

          public DirTap(cascading.scheme.Scheme<Properties,InputStream,OutputStream,?,?> scheme,
                        Path directory,
                        String pattern,
                        int maxDepth,
                        cascading.tap.SinkMode sinkMode)

      Constructor DirTap creates a new DirTap instance using the given `Scheme`, file `directory`, `pattern`, and `maxDepth`
      Parameters:
        `scheme` - of type Scheme
        `directory` - of type Path
        `pattern` - of type String
        `maxDepth` - of type int
        `sinkMode` - of type SinkMode

  - ### Method Detail

    - #### verify

          protected void verify()

      Overrides:
        `verify` in class `FileTap`

    - #### getOutputIdentifier

          public String getOutputIdentifier()

      Description copied from class: `FileTap`
      Only used with `FileTap.openForWrite(FlowProcess, OutputStream)` calls.
      Overrides:
        `getOutputIdentifier` in class `FileTap`

    - #### getOutputFilename

          public String getOutputFilename()

    - #### getOutputFileBasename

          protected String getOutputFileBasename()

    - #### getPattern

          public String getPattern()

    - #### getMaxDepth

          public int getMaxDepth()

    - #### deleteResource

          public boolean deleteResource(Properties conf)
                                 throws IOException

      Overrides:
        `deleteResource` in class `FileTap`
      Throws:
        `IOException`

    - #### openForRead

          public cascading.tuple.TupleEntryIterator openForRead(cascading.flow.FlowProcess<? extends Properties> flowProcess,
                                                                InputStream input)
                                                         throws IOException

      Overrides:
        `openForRead` in class `FileTap`
      Throws:
        `IOException`

    - #### getChildIdentifiers

          public String[] getChildIdentifiers(Properties conf,
                                              int depth,
                                              boolean fullyQualified)
                                       throws IOException

      Specified by:
        `getChildIdentifiers` in interface `cascading.tap.type.FileType<Properties>`
      Overrides:
        `getChildIdentifiers` in class `FileTap`
      Throws:
        `IOException`

    - #### getPathMatcher

          protected PathMatcher getPathMatcher()

    - #### deleteDirTap

          public static boolean deleteDirTap(DirTap dirTap,
                                             Properties conf)
                                      throws IOException

      Method deleteDirTap will recursively delete all files referenced by the given DirTap.
      Parameters:
        `dirTap` - the directory to delete
      Throws:
        `IOException`

Copyright © 2007-2015 Xplenty, Inc. All Rights Reserved.