cascading.tap.partition

Class BasePartitionTap.PartitionScheme<Config,Input,Output>

java.lang.Object

cascading.scheme.Scheme<Config,Input,Output,java.lang.Void,java.lang.Void>

cascading.tap.partition.BasePartitionTap.PartitionScheme<Config,Input,Output>

All Implemented Interfaces:

Traceable, java.io.Serializable

Enclosing class:

BasePartitionTap<Config,Input,Output>

public static class BasePartitionTap.PartitionScheme<Config,Input,Output>
extends Scheme<Config,Input,Output,java.lang.Void,java.lang.Void>

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
PartitionScheme(Scheme scheme)
PartitionScheme(Scheme scheme, Fields partitionFields)

Method Summary

Modifier and Type	Method and Description
int	getNumSinkParts() Method getNumSinkParts returns the numSinkParts of this Scheme object.
Fields	getSinkFields() Method getSinkFields returns the sinkFields of this Scheme object.
Fields	getSourceFields() Method getSourceFields returns the sourceFields of this Scheme object.
Fields	retrieveSinkFields(FlowProcess<? extends Config> flowProcess, Tap tap) Method retrieveSinkFields notifies a Scheme when it is appropriate to dynamically update the fields it sources.
Fields	retrieveSourceFields(FlowProcess<? extends Config> flowProcess, Tap tap) Method retrieveSourceFields notifies a Scheme when it is appropriate to dynamically update the fields it sources.
void	setNumSinkParts(int numSinkParts) Method setNumSinkParts sets the numSinkParts of this Scheme object.
void	setSinkFields(Fields sinkFields) Method setSinkFields sets the sinkFields of this Scheme object.
void	setSourceFields(Fields sourceFields) Method setSourceFields sets the sourceFields of this Scheme object.
void	sink(FlowProcess<? extends Config> flowProcess, SinkCall<java.lang.Void,Output> sinkCall) Method sink writes out the given Tuple found on SinkCall.getOutgoingEntry() to the SinkCall.getOutput().
void	sinkCleanup(FlowProcess<? extends Config> flowProcess, SinkCall<java.lang.Void,Output> sinkCall) Method sinkCleanup is used to destroy resources created by Scheme.sinkPrepare(cascading.flow.FlowProcess, SinkCall).
void	sinkConfInit(FlowProcess<? extends Config> flowProcess, Tap<Config,Input,Output> tap, Config conf) Method sinkInit initializes this instance as a sink.
void	sinkPrepare(FlowProcess<? extends Config> flowProcess, SinkCall<java.lang.Void,Output> sinkCall) Method sinkPrepare is used to initialize resources needed during each call of Scheme.sink(cascading.flow.FlowProcess, SinkCall).
boolean	source(FlowProcess<? extends Config> flowProcess, SourceCall<java.lang.Void,Input> sourceCall) Method source will read a new "record" or value from SourceCall.getInput() and populate the available Tuple via SourceCall.getIncomingEntry() and return true on success or false if no more values available.
void	sourceCleanup(FlowProcess<? extends Config> flowProcess, SourceCall<java.lang.Void,Input> sourceCall) Method sourceCleanup is used to destroy resources created by Scheme.sourcePrepare(cascading.flow.FlowProcess, SourceCall).
void	sourceConfInit(FlowProcess<? extends Config> flowProcess, Tap<Config,Input,Output> tap, Config conf) Method sourceInit initializes this instance as a source.
void	sourcePrepare(FlowProcess<? extends Config> flowProcess, SourceCall<java.lang.Void,Input> sourceCall) Method sourcePrepare is used to initialize resources needed during each call of Scheme.source(cascading.flow.FlowProcess, SourceCall).

Methods inherited from class cascading.scheme.Scheme

equals, getTrace, hashCode, isSink, isSource, isSymmetrical, presentSinkFields, presentSinkFieldsInternal, presentSourceFields, presentSourceFieldsInternal, sinkWrap, sourceRePrepare, sourceWrap, toString

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

PartitionScheme

public PartitionScheme(Scheme scheme)

PartitionScheme

public PartitionScheme(Scheme scheme,
                       Fields partitionFields)

Method Detail

getSinkFields

public Fields getSinkFields()

Description copied from class: Scheme

Method getSinkFields returns the sinkFields of this Scheme object.

Overrides:

getSinkFields in class Scheme<Config,Input,Output,java.lang.Void,java.lang.Void>

Returns:

the sinkFields (type Fields) of this Scheme object.

setSinkFields

public void setSinkFields(Fields sinkFields)

Description copied from class: Scheme

Method setSinkFields sets the sinkFields of this Scheme object.

Overrides:

setSinkFields in class Scheme<Config,Input,Output,java.lang.Void,java.lang.Void>

Parameters:

sinkFields - the sinkFields of this Scheme object.

retrieveSourceFields

public Fields retrieveSourceFields(FlowProcess<? extends Config> flowProcess,
                                   Tap tap)

Description copied from class: Scheme

Method retrieveSourceFields notifies a Scheme when it is appropriate to dynamically update the fields it sources. By default the current declared fields are returned.

The FlowProcess presents all known properties resolved by the current planner.

The tap instance is the parent Tap for this Scheme instance.

Overrides:

retrieveSourceFields in class Scheme<Config,Input,Output,java.lang.Void,java.lang.Void>

Parameters:

flowProcess - of type FlowProcess

tap - of type Tap

Returns:

Fields

retrieveSinkFields

public Fields retrieveSinkFields(FlowProcess<? extends Config> flowProcess,
                                 Tap tap)

Description copied from class: Scheme

Method retrieveSinkFields notifies a Scheme when it is appropriate to dynamically update the fields it sources. By default the current declared fields are returned.

The FlowProcess presents all known properties resolved by the current planner.

The tap instance is the parent Tap for this Scheme instance.

Overrides:

retrieveSinkFields in class Scheme<Config,Input,Output,java.lang.Void,java.lang.Void>

Parameters:

flowProcess - of type FlowProcess

tap - of type Tap

Returns:

Fields

getSourceFields

public Fields getSourceFields()

Description copied from class: Scheme

Method getSourceFields returns the sourceFields of this Scheme object.

Overrides:

getSourceFields in class Scheme<Config,Input,Output,java.lang.Void,java.lang.Void>

Returns:

the sourceFields (type Fields) of this Scheme object.

setSourceFields

public void setSourceFields(Fields sourceFields)

Description copied from class: Scheme

Method setSourceFields sets the sourceFields of this Scheme object.

Overrides:

setSourceFields in class Scheme<Config,Input,Output,java.lang.Void,java.lang.Void>

Parameters:

sourceFields - the sourceFields of this Scheme object.

getNumSinkParts

public int getNumSinkParts()

Description copied from class: Scheme

Method getNumSinkParts returns the numSinkParts of this Scheme object.

Overrides:

getNumSinkParts in class Scheme<Config,Input,Output,java.lang.Void,java.lang.Void>

Returns:

the numSinkParts (type int) of this Scheme object.

setNumSinkParts

public void setNumSinkParts(int numSinkParts)

Description copied from class: Scheme

Method setNumSinkParts sets the numSinkParts of this Scheme object.

Overrides:

setNumSinkParts in class Scheme<Config,Input,Output,java.lang.Void,java.lang.Void>

Parameters:

numSinkParts - the numSinkParts of this Scheme object.

sourceConfInit

public void sourceConfInit(FlowProcess<? extends Config> flowProcess,
                           Tap<Config,Input,Output> tap,
                           Config conf)

Description copied from class: Scheme

Method sourceInit initializes this instance as a source.

This method is executed client side as a means to provide necessary configuration parameters used by the underlying platform.

It is not intended to initialize resources that would be necessary during the execution of this class, like a "formatter" or "parser".

See Scheme.sourcePrepare(cascading.flow.FlowProcess, SourceCall) if resources much be initialized before use. And Scheme.sourceCleanup(cascading.flow.FlowProcess, SourceCall) if resources must be destroyed after use.

Specified by:

sourceConfInit in class Scheme<Config,Input,Output,java.lang.Void,java.lang.Void>

Parameters:

flowProcess - of type FlowProcess

tap - of type Tap

conf - of type Config

sourcePrepare

public void sourcePrepare(FlowProcess<? extends Config> flowProcess,
                          SourceCall<java.lang.Void,Input> sourceCall)
                   throws java.io.IOException

Description copied from class: Scheme

Method sourcePrepare is used to initialize resources needed during each call of Scheme.source(cascading.flow.FlowProcess, SourceCall).

This method is guaranteed to be called once before the first invocation of Scheme.source(FlowProcess, SourceCall).

Be sure to place any initialized objects in the SourceContext so each instance will remain thread-safe.

Overrides:

sourcePrepare in class Scheme<Config,Input,Output,java.lang.Void,java.lang.Void>

Parameters:

flowProcess - of type FlowProcess

sourceCall - of type SourceCall

Throws:

java.io.IOException

source

public boolean source(FlowProcess<? extends Config> flowProcess,
                      SourceCall<java.lang.Void,Input> sourceCall)
               throws java.io.IOException

Description copied from class: Scheme

Method source will read a new "record" or value from SourceCall.getInput() and populate the available Tuple via SourceCall.getIncomingEntry() and return true on success or false if no more values available.

It's ok to set a new Tuple instance on the incomingEntry TupleEntry, or to simply re-use the existing instance.

Note this is only time it is safe to modify a Tuple instance handed over via a method call.

This method may optionally throw a TapException if it cannot process a particular instance of data. If the payload Tuple is set on the TapException, that Tuple will be written to any applicable failure trap Tap.

Specified by:

source in class Scheme<Config,Input,Output,java.lang.Void,java.lang.Void>

Parameters:

flowProcess - of type FlowProcess

sourceCall - of SourceCall

Returns:

returns true when a Tuple was successfully read

Throws:

java.io.IOException

sourceCleanup

public void sourceCleanup(FlowProcess<? extends Config> flowProcess,
                          SourceCall<java.lang.Void,Input> sourceCall)
                   throws java.io.IOException

Description copied from class: Scheme

Method sourceCleanup is used to destroy resources created by Scheme.sourcePrepare(cascading.flow.FlowProcess, SourceCall).

Overrides:

sourceCleanup in class Scheme<Config,Input,Output,java.lang.Void,java.lang.Void>

Parameters:

flowProcess - of Process

sourceCall - of type SourceCall

Throws:

java.io.IOException

sinkConfInit

public void sinkConfInit(FlowProcess<? extends Config> flowProcess,
                         Tap<Config,Input,Output> tap,
                         Config conf)

Description copied from class: Scheme

Method sinkInit initializes this instance as a sink.

This method is executed client side as a means to provide necessary configuration parameters used by the underlying platform.

It is not intended to initialize resources that would be necessary during the execution of this class, like a "formatter" or "parser".

See Scheme.sinkPrepare(cascading.flow.FlowProcess, SinkCall) if resources much be initialized before use. And Scheme.sinkCleanup(cascading.flow.FlowProcess, SinkCall) if resources must be destroyed after use.

Specified by:

sinkConfInit in class Scheme<Config,Input,Output,java.lang.Void,java.lang.Void>

Parameters:

flowProcess - of type FlowProcess

tap - of type Tap

conf - of type Config

sinkPrepare

public void sinkPrepare(FlowProcess<? extends Config> flowProcess,
                        SinkCall<java.lang.Void,Output> sinkCall)
                 throws java.io.IOException

Description copied from class: Scheme

Method sinkPrepare is used to initialize resources needed during each call of Scheme.sink(cascading.flow.FlowProcess, SinkCall).

This method is guaranteed to be called once before the first invocation of Scheme.sink(FlowProcess, SinkCall).

Be sure to place any initialized objects in the SinkContext so each instance will remain threadsafe.

Overrides:

sinkPrepare in class Scheme<Config,Input,Output,java.lang.Void,java.lang.Void>

Parameters:

flowProcess - of type FlowProcess

sinkCall - of type SinkCall

Throws:

java.io.IOException

sink

public void sink(FlowProcess<? extends Config> flowProcess,
                 SinkCall<java.lang.Void,Output> sinkCall)
          throws java.io.IOException

Description copied from class: Scheme

Method sink writes out the given Tuple found on SinkCall.getOutgoingEntry() to the SinkCall.getOutput().

This method may optionally throw a TapException if it cannot process a particular instance of data. If the payload Tuple is set on the TapException, that Tuple will be written to any applicable failure trap Tap. If not set, the incoming Tuple will be written instead.

Specified by:

sink in class Scheme<Config,Input,Output,java.lang.Void,java.lang.Void>

Parameters:

flowProcess - of Process

sinkCall - of SinkCall

Throws:

java.io.IOException

sinkCleanup

public void sinkCleanup(FlowProcess<? extends Config> flowProcess,
                        SinkCall<java.lang.Void,Output> sinkCall)
                 throws java.io.IOException

Description copied from class: Scheme

Method sinkCleanup is used to destroy resources created by Scheme.sinkPrepare(cascading.flow.FlowProcess, SinkCall).

Overrides:

sinkCleanup in class Scheme<Config,Input,Output,java.lang.Void,java.lang.Void>

Parameters:

flowProcess - of type FlowProcess

sinkCall - of type SinkCall

Throws:

java.io.IOException