cascading.platform.hadoop

## Class HadoopPlatform

- java.lang.Object
- - cascading.platform.TestPlatform
  - - cascading.platform.hadoop.BaseHadoopPlatform<JobConf>
    - - cascading.platform.hadoop.HadoopPlatform

    public class HadoopPlatform
    extends BaseHadoopPlatform<JobConf>

Class HadoopPlatform is automatically loaded and injected into a `PlatformTestCase` instance so that all *PlatformTest classes can be tested against Apache Hadoop.

This platform works in three modes.

Hadoop standalone mode is when Hadoop is NOT run as a cluster, and all child tasks are in process and in memory of the "client" side code.

Hadoop mini cluster mode where a cluster is created on demand using the Hadoop MiniDFSCluster and MiniMRCluster utilities. When a PlatformTestCase requests to use a cluster, this is the default cluster. All properties are pulled from the current CLASSPATH via the JobConf.

Lastly remote cluster mode is enabled when the System property "mapred.jar" is set. This is a Hadoop property specifying the Hadoop "job jar" to be used cluster side. This MUST be the Cascading test suite and dependencies packaged in a Hadoop compatible way. This is left to be implemented by the framework using this mode. Additionally these properties may optionally be set if not already in the CLASSPATH; fs.default.name and mapred.job.tracker.

### Field Summary

Fields
`static org.apache.hadoop.hdfs.MiniDFSCluster` `dfs`
`static MiniMRCluster` `mr`

### Fields inherited from class cascading.platform.hadoop.BaseHadoopPlatform

`configuration, fileSys, logger, properties`

### Fields inherited from class cascading.platform.TestPlatform

`CLUSTER_TESTING_PROPERTY, numGatherPartitions, numMappers, numReducers, PLATFORM_PREFIX`

### Constructor Summary

Constructors
Constructor and Description
`HadoopPlatform()`

### Method Summary

Methods
`JobConf` `getConfiguration()`
`FlowConnector` `getFlowConnector(java.util.Map<java.lang.Object,java.lang.Object> properties)`
`FlowProcess` `getFlowProcess()`
`java.lang.Integer` `getNumMapTasks(java.util.Map<java.lang.Object,java.lang.Object> properties)`
`java.lang.Integer` `getNumReduceTasks(java.util.Map<java.lang.Object,java.lang.Object> properties)`
`void` `setNumMapTasks(java.util.Map<java.lang.Object,java.lang.Object> properties, int numMapTasks)`
`void` `setNumReduceTasks(java.util.Map<java.lang.Object,java.lang.Object> properties, int numReduceTasks)`
`void` `setUp()`

### Methods inherited from class cascading.platform.hadoop.BaseHadoopPlatform

`copyFromLocal, copyToLocal, getDelimitedFile, getDelimitedFile, getDelimitedFile, getHiddenTemporaryPath, getLongComparator, getPartitionTap, getProperties, getStringComparator, getTap, getTestConfigDefScheme, getTestFailScheme, getTextFile, isHDFSAvailable, isMapReduce, remoteExists, remoteRemove, safeFileName, setNumGatherPartitions, setNumMappers, setNumReducers, tearDown`

### Methods inherited from class cascading.platform.TestPlatform

`getApplicationJar, getDelimitedFile, getDelimitedFile, getDelimitedFile, getDelimitedFile, getDelimitedFile, getFlowConnector, getGlobalProperties, getName, getNumGatherPartitions, getNumGatherPartitionTasks, getNumMappers, getNumReducers, getTabDelimitedFile, getTabDelimitedFile, getTextFile, getTextFile, getTextFile, getTextFile, isDAG, isUseCluster, setNumGatherPartitionTasks, setUseCluster, supportsGroupByAfterMerge`

### Methods inherited from class java.lang.Object

`clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait`

### Field Detail

#### dfs

    public static transient org.apache.hadoop.hdfs.MiniDFSCluster dfs

#### mr

    public static transient MiniMRCluster mr

### Constructor Detail

#### HadoopPlatform

    public HadoopPlatform()

### Method Detail

#### getFlowConnector

    public FlowConnector getFlowConnector(java.util.Map<java.lang.Object,java.lang.Object> properties)

Specified by:
  `getFlowConnector` in class `cascading.platform.TestPlatform`

#### setNumMapTasks

    public void setNumMapTasks(java.util.Map<java.lang.Object,java.lang.Object> properties,
                      int numMapTasks)

Overrides:
  `setNumMapTasks` in class `cascading.platform.TestPlatform`

#### setNumReduceTasks

    public void setNumReduceTasks(java.util.Map<java.lang.Object,java.lang.Object> properties,
                         int numReduceTasks)

Overrides:
  `setNumReduceTasks` in class `cascading.platform.TestPlatform`

#### getNumMapTasks

    public java.lang.Integer getNumMapTasks(java.util.Map<java.lang.Object,java.lang.Object> properties)

Overrides:
  `getNumMapTasks` in class `cascading.platform.TestPlatform`

#### getNumReduceTasks

    public java.lang.Integer getNumReduceTasks(java.util.Map<java.lang.Object,java.lang.Object> properties)

Overrides:
  `getNumReduceTasks` in class `cascading.platform.TestPlatform`

#### getConfiguration

    public JobConf getConfiguration()

Specified by:
  `getConfiguration` in class `BaseHadoopPlatform<JobConf>`

#### getFlowProcess

    public FlowProcess getFlowProcess()

Specified by:
  `getFlowProcess` in class `cascading.platform.TestPlatform`

#### setUp

    public void setUp()
               throws java.io.IOException

Specified by:
  `setUp` in class `cascading.platform.TestPlatform`
Throws:
  `java.io.IOException`