cascading.scheme.hadoop

## Class WritableSequenceFile

- java.lang.Object
- - cascading.scheme.Scheme<org.apache.hadoop.conf.Configuration,org.apache.hadoop.mapred.RecordReader,org.apache.hadoop.mapred.OutputCollector,java.lang.Object[],java.lang.Void>
  - - cascading.scheme.hadoop.SequenceFile
    - - cascading.scheme.hadoop.WritableSequenceFile

- All Implemented Interfaces:
    Traceable, java.io.Serializable

      public class WritableSequenceFile
      extends SequenceFile

  Class WritableSequenceFile is a sub-class of `SequenceFile` that reads and writes values of the given `writableType` `Class`, instead of `Tuple` instances used by default in SequenceFile.

  This Class is a convenience for those who need to read/write specific types from existing sequence files without them being wrapped in a Tuple instance.

  Note due to the nature of sequence files, only one type can be stored in the key and value positions, they they can be uniquely different types (LongWritable, Text).

  If keyType is null, valueType must not be null, and vice versa, assuming you only wish to store a single value.

  `NullWritable` is used as the empty type for either a null keyType or valueType.
  See Also:
    Serialized Form

- - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `protected java.lang.Class<? extends org.apache.hadoop.io.Writable>` | `keyType`
    `protected java.lang.Class<? extends org.apache.hadoop.io.Writable>` | `valueType`

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `WritableSequenceFile(Fields fields, java.lang.Class<? extends org.apache.hadoop.io.Writable> valueType)` Constructor WritableSequenceFile creates a new WritableSequenceFile instance.
    `WritableSequenceFile(Fields fields, java.lang.Class<? extends org.apache.hadoop.io.Writable> keyType, java.lang.Class<? extends org.apache.hadoop.io.Writable> valueType)` Constructor WritableSequenceFile creates a new WritableSequenceFile instance.

  - ### Method Summary

    Methods
    Modifier and Type | Method and Description
    --- | ---
    `boolean` | `equals(java.lang.Object object)`
    `int` | `hashCode()`
    `void` | `sink(FlowProcess<? extends org.apache.hadoop.conf.Configuration> flowProcess, SinkCall<java.lang.Void,org.apache.hadoop.mapred.OutputCollector> sinkCall)` Method sink writes out the given `Tuple` found on `SinkCall.getOutgoingEntry()` to the `SinkCall.getOutput()`.
    `void` | `sinkConfInit(FlowProcess<? extends org.apache.hadoop.conf.Configuration> flowProcess, Tap<org.apache.hadoop.conf.Configuration,org.apache.hadoop.mapred.RecordReader,org.apache.hadoop.mapred.OutputCollector> tap, org.apache.hadoop.conf.Configuration conf)` Method sinkInit initializes this instance as a sink.
    `boolean` | `source(FlowProcess<? extends org.apache.hadoop.conf.Configuration> flowProcess, SourceCall<java.lang.Object[],org.apache.hadoop.mapred.RecordReader> sourceCall)` Method source will read a new "record" or value from `SourceCall.getInput()` and populate the available `Tuple` via `SourceCall.getIncomingEntry()` and return `true` on success or `false` if no more values available.

    - ### Methods inherited from class cascading.scheme.hadoop.SequenceFile

      `sourceCleanup, sourceConfInit, sourcePrepare`

    - ### Methods inherited from class cascading.scheme.Scheme

      `getNumSinkParts, getSinkFields, getSourceFields, getTrace, isSink, isSource, isSymmetrical, presentSinkFields, presentSinkFieldsInternal, presentSourceFields, presentSourceFieldsInternal, retrieveSinkFields, retrieveSourceFields, setNumSinkParts, setSinkFields, setSourceFields, sinkCleanup, sinkPrepare, sourceRePrepare, toString`

    - ### Methods inherited from class java.lang.Object

      `clone, finalize, getClass, notify, notifyAll, wait, wait, wait`

- - ### Field Detail

    - #### keyType

          protected final java.lang.Class<? extends org.apache.hadoop.io.Writable> keyType

    - #### valueType

          protected final java.lang.Class<? extends org.apache.hadoop.io.Writable> valueType

  - ### Constructor Detail

    - #### WritableSequenceFile

          @ConstructorProperties(value={"fields","valueType"})
          public WritableSequenceFile(Fields fields,
                                                    java.lang.Class<? extends org.apache.hadoop.io.Writable> valueType)

      Constructor WritableSequenceFile creates a new WritableSequenceFile instance.
      Parameters:
        `fields` - of type Fields
        `valueType` - of type Class, may not be null

    - #### WritableSequenceFile

          @ConstructorProperties(value={"fields","keyType","valueType"})
          public WritableSequenceFile(Fields fields,
                                                    java.lang.Class<? extends org.apache.hadoop.io.Writable> keyType,
                                                    java.lang.Class<? extends org.apache.hadoop.io.Writable> valueType)

      Constructor WritableSequenceFile creates a new WritableSequenceFile instance.
      Parameters:
        `fields` - of type Fields
        `keyType` - of type Class
        `valueType` - of type Class

  - ### Method Detail

    - #### sinkConfInit

          public void sinkConfInit(FlowProcess<? extends org.apache.hadoop.conf.Configuration> flowProcess,
                          Tap<org.apache.hadoop.conf.Configuration,org.apache.hadoop.mapred.RecordReader,org.apache.hadoop.mapred.OutputCollector> tap,
                          org.apache.hadoop.conf.Configuration conf)

      Description copied from class: `Scheme`
      Method sinkInit initializes this instance as a sink.

      This method is executed client side as a means to provide necessary configuration parameters used by the underlying platform.

      It is not intended to initialize resources that would be necessary during the execution of this class, like a "formatter" or "parser".

      See `Scheme.sinkPrepare(cascading.flow.FlowProcess, SinkCall)` if resources much be initialized before use. And `Scheme.sinkCleanup(cascading.flow.FlowProcess, SinkCall)` if resources must be destroyed after use.
      Overrides:
        `sinkConfInit` in class `SequenceFile`
      Parameters:
        `flowProcess` - of type FlowProcess
        `tap` - of type Tap
        `conf` - of type Config

    - #### source

          public boolean source(FlowProcess<? extends org.apache.hadoop.conf.Configuration> flowProcess,
                       SourceCall<java.lang.Object[],org.apache.hadoop.mapred.RecordReader> sourceCall)
                         throws java.io.IOException

      Description copied from class: `Scheme`
      Method source will read a new "record" or value from `SourceCall.getInput()` and populate the available `Tuple` via `SourceCall.getIncomingEntry()` and return `true` on success or `false` if no more values available.

      It's ok to set a new Tuple instance on the `incomingEntry` `TupleEntry`, or to simply re-use the existing instance.

      Note this is only time it is safe to modify a Tuple instance handed over via a method call.

      This method may optionally throw a `TapException` if it cannot process a particular instance of data. If the payload Tuple is set on the TapException, that Tuple will be written to any applicable failure trap Tap.
      Overrides:
        `source` in class `SequenceFile`
      Parameters:
        `flowProcess` - of type FlowProcess
        `sourceCall` - of SourceCall
      Returns:
        returns `true` when a Tuple was successfully read
      Throws:
        `java.io.IOException`

    - #### sink

          public void sink(FlowProcess<? extends org.apache.hadoop.conf.Configuration> flowProcess,
                  SinkCall<java.lang.Void,org.apache.hadoop.mapred.OutputCollector> sinkCall)
                    throws java.io.IOException

      Description copied from class: `Scheme`
      Method sink writes out the given `Tuple` found on `SinkCall.getOutgoingEntry()` to the `SinkCall.getOutput()`.

      This method may optionally throw a `TapException` if it cannot process a particular instance of data. If the payload Tuple is set on the TapException, that Tuple will be written to any applicable failure trap Tap. If not set, the incoming Tuple will be written instead.
      Overrides:
        `sink` in class `SequenceFile`
      Parameters:
        `flowProcess` - of Process
        `sinkCall` - of SinkCall
      Throws:
        `java.io.IOException`

    - #### equals

          public boolean equals(java.lang.Object object)

      Overrides:
        `equals` in class `Scheme<org.apache.hadoop.conf.Configuration,org.apache.hadoop.mapred.RecordReader,org.apache.hadoop.mapred.OutputCollector,java.lang.Object[],java.lang.Void>`

    - #### hashCode

          public int hashCode()

      Overrides:
        `hashCode` in class `Scheme<org.apache.hadoop.conf.Configuration,org.apache.hadoop.mapred.RecordReader,org.apache.hadoop.mapred.OutputCollector,java.lang.Object[],java.lang.Void>`

Copyright © 2007-2015 Xplenty, Inc. All Rights Reserved.