cascading.scheme.hadoop

## Class TextLine

- java.lang.Object
- - cascading.scheme.Scheme<Configuration,RecordReader,OutputCollector,java.lang.Object[],java.lang.Object[]>
  - - cascading.scheme.hadoop.TextLine

- All Implemented Interfaces:
    Traceable, java.io.Serializable
  Direct Known Subclasses:
    TextDelimited

      public class TextLine
      extends Scheme<Configuration,RecordReader,OutputCollector,java.lang.Object[],java.lang.Object[]>

  A TextLine is a type of `Scheme` for plain text files. Files are broken into lines. Either line-feed or carriage-return are used to signal end of line.

  By default, this scheme returns a `Tuple` with two fields, "offset" and "line".

  Many of the constructors take both "sourceFields" and "sinkFields". sourceFields denote the field names to be used instead of the names "offset" and "line". sinkFields is a selector and is by default `Fields.ALL`. Any available field names can be given if only a subset of the incoming fields should be used.

  If a `Fields` instance is passed on the constructor as sourceFields having only one field, the return tuples will simply be the "line" value using the given field name.

  Note that TextLine will concatenate all the Tuple values for the selected fields with a TAB delimiter before writing out the line.

  Note sink compression is `TextLine.Compress.DISABLE` by default. If `null` is passed to the constructor for the compression value, it will remain disabled.

  If any of the input files end with ".zip", an error will be thrown. *

  By default, all text is encoded/decoded as UTF-8. This can be changed via the `charsetName` constructor argument.
  See Also:
    Serialized Form

- - ### Nested Class Summary

    Nested Classes
    Modifier and Type | Class and Description
    --- | ---
    `static class` | `TextLine.Compress`

  - ### Field Summary

    Fields
    Modifier and Type | Field and Description
    --- | ---
    `static java.lang.String` | `DEFAULT_CHARSET`
    `static Fields` | `DEFAULT_SOURCE_FIELDS` Field DEFAULT_SOURCE_FIELDS

  - ### Constructor Summary

    Constructors
    Constructor and Description
    `TextLine()` Creates a new TextLine instance that sources "offset" and "line" fields, and sinks all incoming fields, where "offset" is the byte offset in the input file.
    `TextLine(Fields sourceFields)` Creates a new TextLine instance.
    `TextLine(Fields sourceFields, Fields sinkFields)` Creates a new TextLine instance.
    `TextLine(Fields sourceFields, Fields sinkFields, int numSinkParts)` Creates a new TextLine instance.
    `TextLine(Fields sourceFields, Fields sinkFields, java.lang.String charsetName)` Creates a new TextLine instance.
    `TextLine(Fields sourceFields, Fields sinkFields, TextLine.Compress sinkCompression)` Constructor TextLine creates a new TextLine instance.
    `TextLine(Fields sourceFields, Fields sinkFields, TextLine.Compress sinkCompression, int numSinkParts)` Constructor TextLine creates a new TextLine instance.
    `TextLine(Fields sourceFields, Fields sinkFields, TextLine.Compress sinkCompression, int numSinkParts, java.lang.String charsetName)` Constructor TextLine creates a new TextLine instance.
    `TextLine(Fields sourceFields, Fields sinkFields, TextLine.Compress sinkCompression, java.lang.String charsetName)` Constructor TextLine creates a new TextLine instance.
    `TextLine(Fields sourceFields, int numSinkParts)` Creates a new TextLine instance.
    `TextLine(Fields sourceFields, java.lang.String charsetName)` Creates a new TextLine instance.
    `TextLine(int numSinkParts)` Creates a new TextLine instance that sources "offset" and "line" fields, and sinks all incoming fields, where "offset" is the byte offset in the input file.
    `TextLine(TextLine.Compress sinkCompression)` Creates a new TextLine instance that sources "offset" and "line" fields, and sinks all incoming fields, where "offset" is the byte offset in the input file.

  - ### Method Summary

    Methods
    Modifier and Type | Method and Description
    --- | ---
    `java.lang.String` | `getCharsetName()`
    `TextLine.Compress` | `getSinkCompression()` Method getSinkCompression returns the sinkCompression of this TextLine object.
    `protected java.lang.String` | `makeEncodedString(java.lang.Object[] context)`
    `void` | `presentSinkFields(FlowProcess<? extends Configuration> flowProcess, Tap tap, Fields fields)`
    `void` | `presentSourceFields(FlowProcess<? extends Configuration> flowProcess, Tap tap, Fields fields)`
    `protected void` | `setCharsetName(java.lang.String charsetName)`
    `void` | `setSinkCompression(TextLine.Compress sinkCompression)` Method setSinkCompression sets the sinkCompression of this TextLine object.
    `void` | `sink(FlowProcess<? extends Configuration> flowProcess, SinkCall<java.lang.Object[],OutputCollector> sinkCall)`
    `void` | `sinkConfInit(FlowProcess<? extends Configuration> flowProcess, Tap<Configuration,RecordReader,OutputCollector> tap, Configuration conf)`
    `void` | `sinkPrepare(FlowProcess<? extends Configuration> flowProcess, SinkCall<java.lang.Object[],OutputCollector> sinkCall)`
    `boolean` | `source(FlowProcess<? extends Configuration> flowProcess, SourceCall<java.lang.Object[],RecordReader> sourceCall)`
    `void` | `sourceCleanup(FlowProcess<? extends Configuration> flowProcess, SourceCall<java.lang.Object[],RecordReader> sourceCall)`
    `void` | `sourceConfInit(FlowProcess<? extends Configuration> flowProcess, Tap<Configuration,RecordReader,OutputCollector> tap, Configuration conf)`
    `protected void` | `sourceHandleInput(SourceCall<java.lang.Object[],RecordReader> sourceCall)`
    `void` | `sourcePrepare(FlowProcess<? extends Configuration> flowProcess, SourceCall<java.lang.Object[],RecordReader> sourceCall)`
    `protected void` | `verify(Fields sourceFields)`

    - ### Methods inherited from class cascading.scheme.Scheme

      `equals, getNumSinkParts, getSinkFields, getSourceFields, getTrace, hashCode, isSink, isSource, isSymmetrical, presentSinkFieldsInternal, presentSourceFieldsInternal, retrieveSinkFields, retrieveSourceFields, setNumSinkParts, setSinkFields, setSourceFields, sinkCleanup, sourceRePrepare, toString`

    - ### Methods inherited from class java.lang.Object

      `clone, finalize, getClass, notify, notifyAll, wait, wait, wait`

- - ### Field Detail

    - #### DEFAULT_CHARSET

          public static final java.lang.String DEFAULT_CHARSET

      See Also:
        Constant Field Values

    - #### DEFAULT_SOURCE_FIELDS

          public static final Fields DEFAULT_SOURCE_FIELDS

      Field DEFAULT_SOURCE_FIELDS

  - ### Constructor Detail

    - #### TextLine

          public TextLine()

      Creates a new TextLine instance that sources "offset" and "line" fields, and sinks all incoming fields, where "offset" is the byte offset in the input file.

    - #### TextLine

          @ConstructorProperties(value="numSinkParts")
          public TextLine(int numSinkParts)

      Creates a new TextLine instance that sources "offset" and "line" fields, and sinks all incoming fields, where "offset" is the byte offset in the input file.
      Parameters:
        `numSinkParts` - of type int

    - #### TextLine

          @ConstructorProperties(value="sinkCompression")
          public TextLine(TextLine.Compress sinkCompression)

      Creates a new TextLine instance that sources "offset" and "line" fields, and sinks all incoming fields, where "offset" is the byte offset in the input file.
      Parameters:
        `sinkCompression` - of type Compress

    - #### TextLine

          @ConstructorProperties(value={"sourceFields","sinkFields"})
          public TextLine(Fields sourceFields,
                                        Fields sinkFields)

      Creates a new TextLine instance. If sourceFields has one field, only the text line will be returned in the subsequent tuples.
      Parameters:
        `sourceFields` - the source fields for this scheme
        `sinkFields` - the sink fields for this scheme

    - #### TextLine

          @ConstructorProperties(value={"sourceFields","sinkFields","charsetName"})
          public TextLine(Fields sourceFields,
                                        Fields sinkFields,
                                        java.lang.String charsetName)

      Creates a new TextLine instance. If sourceFields has one field, only the text line will be returned in the subsequent tuples.
      Parameters:
        `sourceFields` - the source fields for this scheme
        `sinkFields` - the sink fields for this scheme
        `charsetName` - of type String

    - #### TextLine

          @ConstructorProperties(value={"sourceFields","sinkFields","numSinkParts"})
          public TextLine(Fields sourceFields,
                                        Fields sinkFields,
                                        int numSinkParts)

      Creates a new TextLine instance. If sourceFields has one field, only the text line will be returned in the subsequent tuples.
      Parameters:
        `sourceFields` - the source fields for this scheme
        `sinkFields` - the sink fields for this scheme
        `numSinkParts` - of type int

    - #### TextLine

          @ConstructorProperties(value={"sourceFields","sinkFields","sinkCompression"})
          public TextLine(Fields sourceFields,
                                        Fields sinkFields,
                                        TextLine.Compress sinkCompression)

      Constructor TextLine creates a new TextLine instance. If sourceFields has one field, only the text line will be returned in the subsequent tuples.
      Parameters:
        `sourceFields` - of type Fields
        `sinkFields` - of type Fields
        `sinkCompression` - of type Compress

    - #### TextLine

          @ConstructorProperties(value={"sourceFields","sinkFields","sinkCompression","charsetName"})
          public TextLine(Fields sourceFields,
                                        Fields sinkFields,
                                        TextLine.Compress sinkCompression,
                                        java.lang.String charsetName)

      Constructor TextLine creates a new TextLine instance. If sourceFields has one field, only the text line will be returned in the subsequent tuples.
      Parameters:
        `sourceFields` - of type Fields
        `sinkFields` - of type Fields
        `sinkCompression` - of type Compress
        `charsetName` - of type String

    - #### TextLine

          @ConstructorProperties(value={"sourceFields","sinkFields","sinkCompression","numSinkParts"})
          public TextLine(Fields sourceFields,
                                        Fields sinkFields,
                                        TextLine.Compress sinkCompression,
                                        int numSinkParts)

      Constructor TextLine creates a new TextLine instance. If sourceFields has one field, only the text line will be returned in the subsequent tuples.
      Parameters:
        `sourceFields` - of type Fields
        `sinkFields` - of type Fields
        `sinkCompression` - of type Compress
        `numSinkParts` - of type int

    - #### TextLine

          @ConstructorProperties(value={"sourceFields","sinkFields","sinkCompression","numSinkParts","charsetName"})
          public TextLine(Fields sourceFields,
                                        Fields sinkFields,
                                        TextLine.Compress sinkCompression,
                                        int numSinkParts,
                                        java.lang.String charsetName)

      Constructor TextLine creates a new TextLine instance. If sourceFields has one field, only the text line will be returned in the subsequent tuples.
      Parameters:
        `sourceFields` - of type Fields
        `sinkFields` - of type Fields
        `sinkCompression` - of type Compress
        `numSinkParts` - of type int
        `charsetName` - of type String

    - #### TextLine

          @ConstructorProperties(value="sourceFields")
          public TextLine(Fields sourceFields)

      Creates a new TextLine instance. If sourceFields has one field, only the text line will be returned in the subsequent tuples.
      Parameters:
        `sourceFields` - the source fields for this scheme

    - #### TextLine

          @ConstructorProperties(value={"sourceFields","charsetName"})
          public TextLine(Fields sourceFields,
                                        java.lang.String charsetName)

      Creates a new TextLine instance. If sourceFields has one field, only the text line will be returned in the subsequent tuples.
      Parameters:
        `sourceFields` - the source fields for this scheme
        `charsetName` - of type String

    - #### TextLine

          @ConstructorProperties(value={"sourceFields","numSinkParts"})
          public TextLine(Fields sourceFields,
                                        int numSinkParts)

      Creates a new TextLine instance. If sourceFields has one field, only the text line will be returned in the subsequent tuples. The resulting data set will have numSinkParts.
      Parameters:
        `sourceFields` - the source fields for this scheme
        `numSinkParts` - of type int

  - ### Method Detail

    - #### setCharsetName

          protected void setCharsetName(java.lang.String charsetName)

    - #### getCharsetName

          public java.lang.String getCharsetName()

    - #### verify

          protected void verify(Fields sourceFields)

    - #### getSinkCompression

          public TextLine.Compress getSinkCompression()

      Method getSinkCompression returns the sinkCompression of this TextLine object.
      Returns:
        the sinkCompression (type Compress) of this TextLine object.

    - #### setSinkCompression

          public void setSinkCompression(TextLine.Compress sinkCompression)

      Method setSinkCompression sets the sinkCompression of this TextLine object. If null, compression will remain disabled.
      Parameters:
        `sinkCompression` - the sinkCompression of this TextLine object.

    - #### sourceConfInit

          public void sourceConfInit(FlowProcess<? extends Configuration> flowProcess,
                            Tap<Configuration,RecordReader,OutputCollector> tap,
                            Configuration conf)

      Specified by:
        `sourceConfInit` in class `Scheme<Configuration,RecordReader,OutputCollector,java.lang.Object[],java.lang.Object[]>`

    - #### presentSourceFields

          public void presentSourceFields(FlowProcess<? extends Configuration> flowProcess,
                                 Tap tap,
                                 Fields fields)

      Overrides:
        `presentSourceFields` in class `Scheme<Configuration,RecordReader,OutputCollector,java.lang.Object[],java.lang.Object[]>`

    - #### presentSinkFields

          public void presentSinkFields(FlowProcess<? extends Configuration> flowProcess,
                               Tap tap,
                               Fields fields)

      Overrides:
        `presentSinkFields` in class `Scheme<Configuration,RecordReader,OutputCollector,java.lang.Object[],java.lang.Object[]>`

    - #### sinkConfInit

          public void sinkConfInit(FlowProcess<? extends Configuration> flowProcess,
                          Tap<Configuration,RecordReader,OutputCollector> tap,
                          Configuration conf)

      Specified by:
        `sinkConfInit` in class `Scheme<Configuration,RecordReader,OutputCollector,java.lang.Object[],java.lang.Object[]>`

    - #### sourcePrepare

          public void sourcePrepare(FlowProcess<? extends Configuration> flowProcess,
                           SourceCall<java.lang.Object[],RecordReader> sourceCall)

      Overrides:
        `sourcePrepare` in class `Scheme<Configuration,RecordReader,OutputCollector,java.lang.Object[],java.lang.Object[]>`

    - #### source

          public boolean source(FlowProcess<? extends Configuration> flowProcess,
                       SourceCall<java.lang.Object[],RecordReader> sourceCall)
                         throws java.io.IOException

      Specified by:
        `source` in class `Scheme<Configuration,RecordReader,OutputCollector,java.lang.Object[],java.lang.Object[]>`
      Throws:
        `java.io.IOException`

    - #### sourceHandleInput

          protected void sourceHandleInput(SourceCall<java.lang.Object[],RecordReader> sourceCall)

    - #### makeEncodedString

          protected java.lang.String makeEncodedString(java.lang.Object[] context)

    - #### sourceCleanup

          public void sourceCleanup(FlowProcess<? extends Configuration> flowProcess,
                           SourceCall<java.lang.Object[],RecordReader> sourceCall)

      Overrides:
        `sourceCleanup` in class `Scheme<Configuration,RecordReader,OutputCollector,java.lang.Object[],java.lang.Object[]>`

    - #### sinkPrepare

          public void sinkPrepare(FlowProcess<? extends Configuration> flowProcess,
                         SinkCall<java.lang.Object[],OutputCollector> sinkCall)
                           throws java.io.IOException

      Overrides:
        `sinkPrepare` in class `Scheme<Configuration,RecordReader,OutputCollector,java.lang.Object[],java.lang.Object[]>`
      Throws:
        `java.io.IOException`

    - #### sink

          public void sink(FlowProcess<? extends Configuration> flowProcess,
                  SinkCall<java.lang.Object[],OutputCollector> sinkCall)
                    throws java.io.IOException

      Specified by:
        `sink` in class `Scheme<Configuration,RecordReader,OutputCollector,java.lang.Object[],java.lang.Object[]>`
      Throws:
        `java.io.IOException`

Copyright © 2007-2015 Xplenty, Inc. All Rights Reserved.