cascading.scheme.hadoop

Class SequenceFile

java.lang.Object

cascading.scheme.Scheme<Configuration,RecordReader,OutputCollector,java.lang.Object[],java.lang.Void>

cascading.scheme.hadoop.SequenceFile

All Implemented Interfaces:

Traceable, java.io.Serializable

Direct Known Subclasses:

WritableSequenceFile

public class SequenceFile
extends Scheme<Configuration,RecordReader,OutputCollector,java.lang.Object[],java.lang.Void>

A SequenceFile is a type of Scheme, which is a flat file consisting of binary key/value pairs. This is a space and time efficient means to store data.

See Also:

Serialized Form

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	SequenceFile() Protected for use by TempDfs and other subclasses.
	SequenceFile(Fields fields) Creates a new SequenceFile instance that stores the given field names.

Method Summary

Methods

Modifier and Type	Method and Description
void	sink(FlowProcess<? extends Configuration> flowProcess, SinkCall<java.lang.Void,OutputCollector> sinkCall)
void	sinkConfInit(FlowProcess<? extends Configuration> flowProcess, Tap<Configuration,RecordReader,OutputCollector> tap, Configuration conf)
boolean	source(FlowProcess<? extends Configuration> flowProcess, SourceCall<java.lang.Object[],RecordReader> sourceCall)
void	sourceCleanup(FlowProcess<? extends Configuration> flowProcess, SourceCall<java.lang.Object[],RecordReader> sourceCall)
void	sourceConfInit(FlowProcess<? extends Configuration> flowProcess, Tap<Configuration,RecordReader,OutputCollector> tap, Configuration conf)
void	sourcePrepare(FlowProcess<? extends Configuration> flowProcess, SourceCall<java.lang.Object[],RecordReader> sourceCall)

Methods inherited from class cascading.scheme.Scheme

equals, getNumSinkParts, getSinkFields, getSourceFields, getTrace, hashCode, isSink, isSource, isSymmetrical, presentSinkFields, presentSinkFieldsInternal, presentSourceFields, presentSourceFieldsInternal, retrieveSinkFields, retrieveSourceFields, setNumSinkParts, setSinkFields, setSourceFields, sinkCleanup, sinkPrepare, sourceRePrepare, toString

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

SequenceFile

protected SequenceFile()

Protected for use by TempDfs and other subclasses. Not for general consumption.

SequenceFile

@ConstructorProperties(value="fields")
public SequenceFile(Fields fields)

Creates a new SequenceFile instance that stores the given field names.

Parameters:

fields -

Method Detail

sourceConfInit

public void sourceConfInit(FlowProcess<? extends Configuration> flowProcess,
                  Tap<Configuration,RecordReader,OutputCollector> tap,
                  Configuration conf)

Specified by:

sourceConfInit in class Scheme<Configuration,RecordReader,OutputCollector,java.lang.Object[],java.lang.Void>

sinkConfInit

public void sinkConfInit(FlowProcess<? extends Configuration> flowProcess,
                Tap<Configuration,RecordReader,OutputCollector> tap,
                Configuration conf)

Specified by:

sinkConfInit in class Scheme<Configuration,RecordReader,OutputCollector,java.lang.Object[],java.lang.Void>

sourcePrepare

public void sourcePrepare(FlowProcess<? extends Configuration> flowProcess,
                 SourceCall<java.lang.Object[],RecordReader> sourceCall)

Overrides:

sourcePrepare in class Scheme<Configuration,RecordReader,OutputCollector,java.lang.Object[],java.lang.Void>

source

public boolean source(FlowProcess<? extends Configuration> flowProcess,
             SourceCall<java.lang.Object[],RecordReader> sourceCall)
               throws java.io.IOException

Specified by:

source in class Scheme<Configuration,RecordReader,OutputCollector,java.lang.Object[],java.lang.Void>

Throws:

java.io.IOException

sourceCleanup

public void sourceCleanup(FlowProcess<? extends Configuration> flowProcess,
                 SourceCall<java.lang.Object[],RecordReader> sourceCall)

Overrides:

sourceCleanup in class Scheme<Configuration,RecordReader,OutputCollector,java.lang.Object[],java.lang.Void>

sink

public void sink(FlowProcess<? extends Configuration> flowProcess,
        SinkCall<java.lang.Void,OutputCollector> sinkCall)
          throws java.io.IOException

Specified by:

sink in class Scheme<Configuration,RecordReader,OutputCollector,java.lang.Object[],java.lang.Void>

Throws:

java.io.IOException