cascading.tap

Class MultiSinkTap<Child extends Tap,Config,Output>

java.lang.Object

cascading.tap.Tap<Config,java.lang.Void,Output>

cascading.tap.SinkTap<Config,Output>

cascading.tap.MultiSinkTap<Child,Config,Output>

All Implemented Interfaces:

FlowElement, ScopedElement, CompositeTap<Child>, Traceable, java.io.Serializable

public class MultiSinkTap<Child extends Tap,Config,Output>
extends SinkTap<Config,Output>
implements CompositeTap<Child>

Class MultiSinkTap is both a CompositeTap and SinkTap that can write to multiple child Tap instances simultaneously.

It is the counterpart to MultiSourceTap.

Note all child Tap instances may or may not have the same declared Fields. In the case they do not, all sink fields will be merged into a single Fields instance via Fields.merge(cascading.tuple.Fields...).

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor and Description
MultiSinkTap(Child... taps) Constructor MultiSinkTap creates a new MultiSinkTap instance.

Method Summary

Modifier and Type	Method and Description
boolean	commitResource(Config conf) Method commitResource allows the underlying resource to be notified when all write processing is successful so that any additional cleanup or processing may be completed.
boolean	createResource(Config conf) Method createResource creates the underlying resource.
boolean	deleteResource(Config conf) Method deleteResource deletes the resource represented by this instance.
boolean	equals(java.lang.Object o)
java.util.Iterator<Child>	getChildTaps()
java.lang.String	getIdentifier() Method getIdentifier returns a String representing the resource this Tap instance represents.
long	getModifiedTime(Config conf) Method getModifiedTime returns the date this resource was last modified.
long	getNumChildTaps()
Scheme	getScheme() Method getScheme returns the scheme of this Tap object.
protected Child[]	getTaps()
int	hashCode()
TupleEntryCollector	openForWrite(FlowProcess<? extends Config> flowProcess, Output output) Method openForWrite opens the resource represented by this Tap instance for writing.
void	presentSinkFields(FlowProcess<? extends Config> flowProcess, Fields fields)
boolean	resourceExists(Config conf) Method resourceExists returns true if the path represented by this instance exists.
boolean	rollbackResource(Config conf) Method rollbackResource allows the underlying resource to be notified when any write processing has failed or was stopped so that any cleanup may be started.
void	sinkConfInit(FlowProcess<? extends Config> flowProcess, Config conf) Method sinkConfInit initializes this instance as a sink.
java.lang.String	toString()

Methods inherited from class cascading.tap.SinkTap

getSourceFields, isSource, openForRead, prepareResourceForRead, sourceConfInit

Methods inherited from class cascading.tap.Tap

createResource, deleteResource, entryStream, entryStream, entryStreamCopy, entryStreamCopy, flowConfInit, getConfigDef, getFullIdentifier, getFullIdentifier, getModifiedTime, getNodeConfigDef, getSinkFields, getSinkMode, getStepConfigDef, getTrace, hasConfigDef, hasNodeConfigDef, hasStepConfigDef, id, isKeep, isReplace, isSink, isTemporary, isUpdate, openForRead, openForReadUnchecked, openForWrite, outgoingScopeFor, prepareResourceForWrite, presentSourceFields, resolveIncomingOperationArgumentFields, resolveIncomingOperationPassThroughFields, resourceExists, retrieveSinkFields, retrieveSourceFields, setScheme, spliterator, splititerator, taps, tupleStream, tupleStream, tupleStreamCopy, tupleStreamCopy

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

MultiSinkTap

@ConstructorProperties(value="taps")
public MultiSinkTap(Child... taps)

Constructor MultiSinkTap creates a new MultiSinkTap instance.

Parameters:

taps - of type Tap...

Method Detail

getTaps

protected Child[] getTaps()

getChildTaps

public java.util.Iterator<Child> getChildTaps()

Specified by:

getChildTaps in interface CompositeTap<Child extends Tap>

getNumChildTaps

public long getNumChildTaps()

Specified by:

getNumChildTaps in interface CompositeTap<Child extends Tap>

getIdentifier

public java.lang.String getIdentifier()

Description copied from class: Tap

Method getIdentifier returns a String representing the resource this Tap instance represents.

Often, if the tap accesses a filesystem, the identifier is nothing more than the path to the file or directory. In other cases it may be a an URL or URI representing a connection string or remote resource.

Any two Tap instances having the same value for the identifier are considered equal.

Specified by:

getIdentifier in class Tap<Config,java.lang.Void,Output>

Returns:

String

presentSinkFields

public void presentSinkFields(FlowProcess<? extends Config> flowProcess,
                              Fields fields)

Overrides:

presentSinkFields in class Tap<Config,java.lang.Void,Output>

openForWrite

public TupleEntryCollector openForWrite(FlowProcess<? extends Config> flowProcess,
                                        Output output)
                                 throws java.io.IOException

Description copied from class: Tap

Method openForWrite opens the resource represented by this Tap instance for writing.

This method is used internally and does not honor the SinkMode setting. If SinkMode is SinkMode.REPLACE, this call may fail. See Tap.openForWrite(cascading.flow.FlowProcess).

output value may be null, if so, sub-classes must inquire with the underlying Scheme via Scheme.sinkConfInit(cascading.flow.FlowProcess, Tap, Object) to get the proper output type and instantiate it before calling super.openForWrite().

Specified by:

openForWrite in class Tap<Config,java.lang.Void,Output>

Parameters:

flowProcess - of type FlowProcess

output - of type Output

Returns:

TupleEntryCollector

Throws:

java.io.IOException - when the resource cannot be opened

sinkConfInit

public void sinkConfInit(FlowProcess<? extends Config> flowProcess,
                         Config conf)

Description copied from class: Tap

Method sinkConfInit initializes this instance as a sink.

This method maybe called more than once if this Tap instance is used outside the scope of a Flow instance or if it participates in multiple times in a given Flow or across different Flows in a Cascade.

Note this method will be called in context of this Tap being used as a traditional 'sink' and as a 'trap'.

In the context of a Flow, it will be called after FlowListener.onStarting(cascading.flow.Flow)

Note that no resources or services should be modified by this method. If this Tap instance returns true for Tap.isReplace(), then Tap.deleteResource(Object) will be called by the parent Flow.

Overrides:

sinkConfInit in class Tap<Config,java.lang.Void,Output>

Parameters:

flowProcess - of type FlowProcess

conf - of type Config

createResource

public boolean createResource(Config conf)
                       throws java.io.IOException

Description copied from class: Tap

Method createResource creates the underlying resource.

Specified by:

createResource in class Tap<Config,java.lang.Void,Output>

Parameters:

conf - of type Config

Returns:

boolean

Throws:

java.io.IOException - when there is an error making directories

deleteResource

public boolean deleteResource(Config conf)
                       throws java.io.IOException

Description copied from class: Tap

Method deleteResource deletes the resource represented by this instance.

Specified by:

deleteResource in class Tap<Config,java.lang.Void,Output>

Parameters:

conf - of type Config

Returns:

boolean

Throws:

java.io.IOException - when the resource cannot be deleted

commitResource

public boolean commitResource(Config conf)
                       throws java.io.IOException

Description copied from class: Tap

Method commitResource allows the underlying resource to be notified when all write processing is successful so that any additional cleanup or processing may be completed.

See Tap.rollbackResource(Object) to handle cleanup in the face of failures.

This method is invoked once client side and not in the cluster, if any.

If other sink Tap instance in a given Flow fail on commitResource after called on this instance, rollbackResource will not be called.

Overrides:

commitResource in class Tap<Config,java.lang.Void,Output>

Parameters:

conf - of type Config

Returns:

returns true if successful

Throws:

java.io.IOException

rollbackResource

public boolean rollbackResource(Config conf)
                         throws java.io.IOException

Description copied from class: Tap

Method rollbackResource allows the underlying resource to be notified when any write processing has failed or was stopped so that any cleanup may be started.

See Tap.commitResource(Object) to handle cleanup when the write has successfully completed.

This method is invoked once client side and not in the cluster, if any.

Overrides:

rollbackResource in class Tap<Config,java.lang.Void,Output>

Parameters:

conf - of type Config

Returns:

returns true if successful

Throws:

java.io.IOException

resourceExists

public boolean resourceExists(Config conf)
                       throws java.io.IOException

Description copied from class: Tap

Method resourceExists returns true if the path represented by this instance exists.

Specified by:

resourceExists in class Tap<Config,java.lang.Void,Output>

Parameters:

conf - of type Config

Returns:

true if the underlying resource already exists

Throws:

java.io.IOException - when the status cannot be determined

getModifiedTime

public long getModifiedTime(Config conf)
                     throws java.io.IOException

Description copied from class: Tap

Method getModifiedTime returns the date this resource was last modified.

If the resource does not exist, returns zero (0).

If the resource is continuous, returns Long.MAX_VALUE.

Specified by:

getModifiedTime in class Tap<Config,java.lang.Void,Output>

Parameters:

conf - of type Config

Returns:

The date this resource was last modified.

Throws:

java.io.IOException

getScheme

public Scheme getScheme()

Description copied from class: Tap

Method getScheme returns the scheme of this Tap object.

Overrides:

getScheme in class Tap<Config,java.lang.Void,Output>

Returns:

the scheme (type Scheme) of this Tap object.

toString

public java.lang.String toString()

Overrides:

toString in class Tap<Config,java.lang.Void,Output>

equals

public boolean equals(java.lang.Object o)

Overrides:

equals in class Tap<Config,java.lang.Void,Output>

hashCode

public int hashCode()

Overrides:

hashCode in class Tap<Config,java.lang.Void,Output>