cascading.platform.hadoop

## Class BaseHadoopPlatform<Config extends Configuration>

- java.lang.Object
- - cascading.platform.TestPlatform
  - - cascading.platform.hadoop.BaseHadoopPlatform<Config>

Direct Known Subclasses:
  HadoopPlatform

    public abstract class BaseHadoopPlatform<Config extends Configuration>
    extends cascading.platform.TestPlatform

### Field Summary

Fields
`static Configuration` `configuration`
`static FileSystem` `fileSys`
`protected java.lang.String` `logger`
`static java.util.Map<java.lang.Object,java.lang.Object>` `properties`

### Fields inherited from class cascading.platform.TestPlatform

`CLUSTER_TESTING_PROPERTY, numGatherPartitions, numMappers, numReducers, PLATFORM_PREFIX`

### Constructor Summary

Constructors
Constructor and Description
`BaseHadoopPlatform()`

### Method Summary

Methods
`void` `copyFromLocal(java.lang.String inputFile)`
`void` `copyToLocal(java.lang.String outputFile)`
`abstract Config` `getConfiguration()`
`Tap` `getDelimitedFile(Fields fields, boolean skipHeader, boolean writeHeader, java.lang.String delimiter, java.lang.String quote, java.lang.Class[] types, java.lang.String filename, SinkMode mode)`
`Tap` `getDelimitedFile(Fields fields, boolean hasHeader, java.lang.String delimiter, java.lang.String quote, java.lang.Class[] types, java.lang.String filename, SinkMode mode)`
`Tap` `getDelimitedFile(java.lang.String delimiter, java.lang.String quote, FieldTypeResolver fieldTypeResolver, java.lang.String filename, SinkMode mode)`
`java.lang.String` `getHiddenTemporaryPath()`
`java.util.Comparator` `getLongComparator(boolean reverseSort)`
`Tap` `getPartitionTap(Tap sink, Partition partition, int openThreshold)`
`java.util.Map<java.lang.Object,java.lang.Object>` `getProperties()`
`java.util.Comparator` `getStringComparator(boolean reverseSort)`
`Tap` `getTap(Scheme scheme, java.lang.String filename, SinkMode mode)`
`Scheme` `getTestConfigDefScheme()`
`Scheme` `getTestFailScheme()`
`Tap` `getTextFile(Fields sourceFields, Fields sinkFields, java.lang.String filename, SinkMode mode)`
`boolean` `isHDFSAvailable()`
`boolean` `isMapReduce()`
`boolean` `remoteExists(java.lang.String outputFile)`
`boolean` `remoteRemove(java.lang.String outputFile, boolean recursive)`
`protected java.lang.String` `safeFileName(java.lang.String filename)` Replaces characters, that are not allowed by HDFS with an "_".
`void` `setNumGatherPartitions(int numGatherPartitions)`
`void` `setNumMappers(int numMapTasks)`
`void` `setNumReducers(int numReduceTasks)`
`void` `tearDown()`

### Methods inherited from class cascading.platform.TestPlatform

`getApplicationJar, getDelimitedFile, getDelimitedFile, getDelimitedFile, getDelimitedFile, getDelimitedFile, getFlowConnector, getFlowConnector, getFlowProcess, getGlobalProperties, getName, getNumGatherPartitions, getNumGatherPartitionTasks, getNumMappers, getNumMapTasks, getNumReducers, getNumReduceTasks, getTabDelimitedFile, getTabDelimitedFile, getTextFile, getTextFile, getTextFile, getTextFile, isDAG, isUseCluster, setNumGatherPartitionTasks, setNumMapTasks, setNumReduceTasks, setUp, setUseCluster, supportsGroupByAfterMerge`

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Field Detail

#### fileSys

    public static transient FileSystem fileSys

#### configuration

    public static transient Configuration configuration

#### properties

    public static transient java.util.Map<java.lang.Object,java.lang.Object> properties

#### logger

    protected java.lang.String logger

### Constructor Detail

#### BaseHadoopPlatform

    public BaseHadoopPlatform()

### Method Detail

#### isMapReduce

    public boolean isMapReduce()

Overrides:
  `isMapReduce` in class `cascading.platform.TestPlatform`

#### setNumMappers

    public void setNumMappers(int numMapTasks)

Overrides:
  `setNumMappers` in class `cascading.platform.TestPlatform`

#### setNumReducers

    public void setNumReducers(int numReduceTasks)

Overrides:
  `setNumReducers` in class `cascading.platform.TestPlatform`

#### setNumGatherPartitions

    public void setNumGatherPartitions(int numGatherPartitions)

Overrides:
  `setNumGatherPartitions` in class `cascading.platform.TestPlatform`

#### getProperties

    public java.util.Map<java.lang.Object,java.lang.Object> getProperties()

Specified by:
  `getProperties` in class `cascading.platform.TestPlatform`

#### tearDown

    public void tearDown()

Specified by:
  `tearDown` in class `cascading.platform.TestPlatform`

#### getConfiguration

    public abstract Config getConfiguration()

#### isHDFSAvailable

    public boolean isHDFSAvailable()

#### copyFromLocal

    public void copyFromLocal(java.lang.String inputFile)
                       throws java.io.IOException

Specified by:
  `copyFromLocal` in class `cascading.platform.TestPlatform`
Throws:
  `java.io.IOException`

#### copyToLocal

    public void copyToLocal(java.lang.String outputFile)
                     throws java.io.IOException

Specified by:
  `copyToLocal` in class `cascading.platform.TestPlatform`
Throws:
  `java.io.IOException`

#### remoteExists

    public boolean remoteExists(java.lang.String outputFile)
                         throws java.io.IOException

Specified by:
  `remoteExists` in class `cascading.platform.TestPlatform`
Throws:
  `java.io.IOException`

#### remoteRemove

    public boolean remoteRemove(java.lang.String outputFile,
                       boolean recursive)
                         throws java.io.IOException

Specified by:
  `remoteRemove` in class `cascading.platform.TestPlatform`
Throws:
  `java.io.IOException`

#### getTap

    public Tap getTap(Scheme scheme,
             java.lang.String filename,
             SinkMode mode)

Specified by:
  `getTap` in class `cascading.platform.TestPlatform`

#### getTextFile

    public Tap getTextFile(Fields sourceFields,
                  Fields sinkFields,
                  java.lang.String filename,
                  SinkMode mode)

Specified by:
  `getTextFile` in class `cascading.platform.TestPlatform`

#### getDelimitedFile

    public Tap getDelimitedFile(Fields fields,
                       boolean hasHeader,
                       java.lang.String delimiter,
                       java.lang.String quote,
                       java.lang.Class[] types,
                       java.lang.String filename,
                       SinkMode mode)

Specified by:
  `getDelimitedFile` in class `cascading.platform.TestPlatform`

#### getDelimitedFile

    public Tap getDelimitedFile(Fields fields,
                       boolean skipHeader,
                       boolean writeHeader,
                       java.lang.String delimiter,
                       java.lang.String quote,
                       java.lang.Class[] types,
                       java.lang.String filename,
                       SinkMode mode)

Specified by:
  `getDelimitedFile` in class `cascading.platform.TestPlatform`

#### getDelimitedFile

    public Tap getDelimitedFile(java.lang.String delimiter,
                       java.lang.String quote,
                       FieldTypeResolver fieldTypeResolver,
                       java.lang.String filename,
                       SinkMode mode)

Specified by:
  `getDelimitedFile` in class `cascading.platform.TestPlatform`

#### getPartitionTap

    public Tap getPartitionTap(Tap sink,
                      Partition partition,
                      int openThreshold)

Specified by:
  `getPartitionTap` in class `cascading.platform.TestPlatform`

#### getTestConfigDefScheme

    public Scheme getTestConfigDefScheme()

Specified by:
  `getTestConfigDefScheme` in class `cascading.platform.TestPlatform`

#### getTestFailScheme

    public Scheme getTestFailScheme()

Specified by:
  `getTestFailScheme` in class `cascading.platform.TestPlatform`

#### getLongComparator

    public java.util.Comparator getLongComparator(boolean reverseSort)

Specified by:
  `getLongComparator` in class `cascading.platform.TestPlatform`

#### getStringComparator

    public java.util.Comparator getStringComparator(boolean reverseSort)

Specified by:
  `getStringComparator` in class `cascading.platform.TestPlatform`

#### getHiddenTemporaryPath

    public java.lang.String getHiddenTemporaryPath()

Specified by:
  `getHiddenTemporaryPath` in class `cascading.platform.TestPlatform`

#### safeFileName

    protected java.lang.String safeFileName(java.lang.String filename)

Replaces characters, that are not allowed by HDFS with an "_".

Parameters:
  `filename` - The filename to make safe
Returns:
  The filename with all non-supported characters removed.