cascading.tap
Class BaseTemplateTap<Config,Output>

java.lang.Object
  cascading.tap.Tap<Config,Void,Output>
      cascading.tap.SinkTap<Config,Output>
          cascading.tap.BaseTemplateTap<Config,Output>

All Implemented Interfaces:

FlowElement, Traceable, Serializable

public abstract class BaseTemplateTap<Config,Output>
extends SinkTap<Config,Output>

See Also:

Serialized Form

Nested Class Summary

static class	BaseTemplateTap.Counters
static class	BaseTemplateTap.TemplateScheme<Config,Output>

Field Summary

protected boolean	keepParentOnDelete Field keepParentOnDelete
protected static int	OPEN_TAPS_THRESHOLD_DEFAULT Field OPEN_FILES_THRESHOLD_DEFAULT
protected int	openTapsThreshold Field openTapsThreshold
protected Tap	parent Field parent
protected String	pathTemplate Field pathTemplate

Constructor Summary

protected	BaseTemplateTap(Scheme<Config,?,Output,?,?> scheme)
protected	BaseTemplateTap(Scheme<Config,?,Output,?,?> scheme, SinkMode sinkMode)
protected	BaseTemplateTap(Tap parent, String pathTemplate, Fields pathFields, int openTapsThreshold)
protected	BaseTemplateTap(Tap parent, String pathTemplate, Fields pathFields, SinkMode sinkMode)
protected	BaseTemplateTap(Tap parent, String pathTemplate, Fields pathFields, SinkMode sinkMode, boolean keepParentOnDelete, int openTapsThreshold)
protected	BaseTemplateTap(Tap parent, String pathTemplate, int openTapsThreshold)
protected	BaseTemplateTap(Tap parent, String pathTemplate, SinkMode sinkMode)
protected	BaseTemplateTap(Tap parent, String pathTemplate, SinkMode sinkMode, boolean keepParentOnDelete, int openTapsThreshold)

Method Summary

boolean	commitResource(Config conf) Method commitResource allows the underlying resource to be notified when all write processing is successful so that any additional cleanup or processing may be completed.
boolean	createResource(Config conf) Method createResource creates the underlying resource.
protected abstract TupleEntrySchemeCollector	createTupleEntrySchemeCollector(FlowProcess<Config> flowProcess, Tap parent, String path)
boolean	deleteResource(Config conf) Method deleteResource deletes the resource represented by this instance.
boolean	equals(Object object)
String	getIdentifier() Method getIdentifier returns a String representing the resource this Tap instance represents.
long	getModifiedTime(Config conf) Method getModifiedTime returns the date this resource was last modified.
int	getOpenTapsThreshold() Method getOpenTapsThreshold returns the openTapsThreshold of this TemplateTap object.
Tap	getParent() Method getParent returns the parent Tap of this TemplateTap object.
String	getPathTemplate() Method getPathTemplate returns the pathTemplate Formatter format String of this TemplateTap object.
int	hashCode()
TupleEntryCollector	openForWrite(FlowProcess<Config> flowProcess, Output output) Method openForWrite opens the resource represented by this Tap instance for writing.
boolean	prepareResourceForRead(Config conf) Method prepareResourceForRead allows the underlying resource to be notified when reading will begin.
boolean	prepareResourceForWrite(Config conf) Method prepareResourceForWrite allows the underlying resource to be notified when writing will begin.
boolean	resourceExists(Config conf) Method resourceExists returns true if the path represented by this instance exists.
boolean	rollbackResource(Config conf) Method rollbackResource allows the underlying resource to be notified when any write processing has failed or was stopped so that any cleanup may be started.
String	toString()

Methods inherited from class cascading.tap.SinkTap

getSourceFields, isSource, openForRead, sourceConfInit

Methods inherited from class cascading.tap.Tap

createResource, deleteResource, flowConfInit, getConfigDef, getFullIdentifier, getFullIdentifier, getModifiedTime, getScheme, getSinkFields, getSinkMode, getStepConfigDef, getTrace, hasConfigDef, hasStepConfigDef, id, isEquivalentTo, isKeep, isReplace, isSink, isTemporary, isUpdate, openForRead, openForWrite, outgoingScopeFor, presentSinkFields, presentSourceFields, resolveIncomingOperationArgumentFields, resolveIncomingOperationPassThroughFields, resourceExists, retrieveSinkFields, retrieveSourceFields, setScheme, sinkConfInit, taps

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

OPEN_TAPS_THRESHOLD_DEFAULT

protected static final int OPEN_TAPS_THRESHOLD_DEFAULT

Field OPEN_FILES_THRESHOLD_DEFAULT

See Also:

Constant Field Values

parent

protected Tap parent

Field parent

pathTemplate

protected String pathTemplate

Field pathTemplate

keepParentOnDelete

protected boolean keepParentOnDelete

Field keepParentOnDelete

openTapsThreshold

protected int openTapsThreshold

Field openTapsThreshold

Constructor Detail

BaseTemplateTap

protected BaseTemplateTap(Tap parent,
                          String pathTemplate,
                          int openTapsThreshold)

BaseTemplateTap

protected BaseTemplateTap(Tap parent,
                          String pathTemplate,
                          SinkMode sinkMode)

BaseTemplateTap

protected BaseTemplateTap(Tap parent,
                          String pathTemplate,
                          SinkMode sinkMode,
                          boolean keepParentOnDelete,
                          int openTapsThreshold)

BaseTemplateTap

protected BaseTemplateTap(Tap parent,
                          String pathTemplate,
                          Fields pathFields,
                          int openTapsThreshold)

BaseTemplateTap

protected BaseTemplateTap(Tap parent,
                          String pathTemplate,
                          Fields pathFields,
                          SinkMode sinkMode)

BaseTemplateTap

protected BaseTemplateTap(Tap parent,
                          String pathTemplate,
                          Fields pathFields,
                          SinkMode sinkMode,
                          boolean keepParentOnDelete,
                          int openTapsThreshold)

BaseTemplateTap

protected BaseTemplateTap(Scheme<Config,?,Output,?,?> scheme,
                          SinkMode sinkMode)

BaseTemplateTap

protected BaseTemplateTap(Scheme<Config,?,Output,?,?> scheme)

Method Detail

createTupleEntrySchemeCollector

protected abstract TupleEntrySchemeCollector createTupleEntrySchemeCollector(FlowProcess<Config> flowProcess,
                                                                             Tap parent,
                                                                             String path)
                                                                      throws IOException

Throws:

IOException

getParent

public Tap getParent()

Method getParent returns the parent Tap of this TemplateTap object.

Returns:

the parent (type Tap) of this TemplateTap object.

getPathTemplate

public String getPathTemplate()

Method getPathTemplate returns the pathTemplate Formatter format String of this TemplateTap object.

Returns:

the pathTemplate (type String) of this TemplateTap object.

getIdentifier

public String getIdentifier()

Description copied from class: Tap

Method getIdentifier returns a String representing the resource this Tap instance represents.

Often, if the tap accesses a filesystem, the identifier is nothing more than the path to the file or directory. In other cases it may be a an URL or URI representing a connection string or remote resource.

Any two Tap instances having the same value for the identifier are considered equal.

Specified by:

getIdentifier in class Tap<Config,Void,Output>

Returns:

String

getOpenTapsThreshold

public int getOpenTapsThreshold()

Method getOpenTapsThreshold returns the openTapsThreshold of this TemplateTap object.

Returns:

the openTapsThreshold (type int) of this TemplateTap object.

openForWrite

public TupleEntryCollector openForWrite(FlowProcess<Config> flowProcess,
                                        Output output)
                                 throws IOException

Description copied from class: Tap

Method openForWrite opens the resource represented by this Tap instance for writing.

This method is used internally and does not honor the SinkMode setting. If SinkMode is SinkMode.REPLACE, this call may fail. See Tap.openForWrite(cascading.flow.FlowProcess).

output value may be null, if so, sub-classes must inquire with the underlying Scheme via Scheme.sinkConfInit(cascading.flow.FlowProcess, Tap, Object) to get the proper output type and instantiate it before calling super.openForWrite().

Specified by:

openForWrite in class Tap<Config,Void,Output>

Parameters:

flowProcess - of type FlowProcess

output - of type Output

Returns:

TupleEntryCollector

Throws:

IOException - when the resource cannot be opened

createResource

public boolean createResource(Config conf)
                       throws IOException

Description copied from class: Tap

Method createResource creates the underlying resource.

Specified by:

createResource in class Tap<Config,Void,Output>

Parameters:

conf - of type Config

Returns:

boolean

Throws:

IOException - when there is an error making directories

See Also:

Tap.createResource(Object)

deleteResource

public boolean deleteResource(Config conf)
                       throws IOException

Description copied from class: Tap

Method deleteResource deletes the resource represented by this instance.

Specified by:

deleteResource in class Tap<Config,Void,Output>

Parameters:

conf - of type Config

Returns:

boolean

Throws:

IOException - when the resource cannot be deleted

See Also:

Tap.deleteResource(Object)

prepareResourceForRead

public boolean prepareResourceForRead(Config conf)
                               throws IOException

Description copied from class: Tap

Method prepareResourceForRead allows the underlying resource to be notified when reading will begin.

This method will be called client side so that any remote or external resources can be initialized.

If this method returns false, an exception will be thrown halting the current Flow.

In most cases, resource initialization should happen in the Tap.openForRead(FlowProcess, Object) method.

This allows for initialization of cluster side resources, like a JDBC driver used to read data from a database, that cannot be passed client to cluster.

Overrides:

prepareResourceForRead in class SinkTap<Config,Output>

Parameters:

conf - of type Config

Returns:

returns true if successful

Throws:

IOException

prepareResourceForWrite

public boolean prepareResourceForWrite(Config conf)
                                throws IOException

Description copied from class: Tap

Method prepareResourceForWrite allows the underlying resource to be notified when writing will begin.

This method will be called once client side so that any remote or external resources can be initialized.

If this method returns false, an exception will be thrown halting the current Flow.

In most cases, resource initialization should happen in the Tap.openForWrite(FlowProcess, Object) method.

This allows for initialization of cluster side resources, like a JDBC driver used to write data to a database, that cannot be passed client to cluster.

In the above JDBC example, overriding this method will allow for testing for the existence of and/or creating a remote table used by all individual cluster side tasks.

Overrides:

prepareResourceForWrite in class Tap<Config,Void,Output>

Parameters:

conf - of type Config

Returns:

returns true if successful

Throws:

IOException

commitResource

public boolean commitResource(Config conf)
                       throws IOException

Description copied from class: Tap

Method commitResource allows the underlying resource to be notified when all write processing is successful so that any additional cleanup or processing may be completed.

See Tap.rollbackResource(Object) to handle cleanup in the face of failures.

This method is invoked once client side and not in the cluster, if any.

If other sink Tap instance in a given Flow fail on commitResource after called on this instance, rollbackResource will not be called.

Overrides:

commitResource in class Tap<Config,Void,Output>

Parameters:

conf - of type Config

Returns:

returns true if successful

Throws:

IOException

rollbackResource

public boolean rollbackResource(Config conf)
                         throws IOException

Description copied from class: Tap

Method rollbackResource allows the underlying resource to be notified when any write processing has failed or was stopped so that any cleanup may be started.

See Tap.commitResource(Object) to handle cleanup when the write has successfully completed.

This method is invoked once client side and not in the cluster, if any.

Overrides:

rollbackResource in class Tap<Config,Void,Output>

Parameters:

conf - of type Config

Returns:

returns true if successful

Throws:

IOException

resourceExists

public boolean resourceExists(Config conf)
                       throws IOException

Description copied from class: Tap

Method resourceExists returns true if the path represented by this instance exists.

Specified by:

resourceExists in class Tap<Config,Void,Output>

Parameters:

conf - of type Config

Returns:

true if the underlying resource already exists

Throws:

IOException - when the status cannot be determined

See Also:

Tap.resourceExists(Object)

getModifiedTime

public long getModifiedTime(Config conf)
                     throws IOException

Description copied from class: Tap

Method getModifiedTime returns the date this resource was last modified.

Specified by:

getModifiedTime in class Tap<Config,Void,Output>

Parameters:

conf - of type Config

Returns:

The date this resource was last modified.

Throws:

IOException

See Also:

Tap.getModifiedTime(Object)

equals

public boolean equals(Object object)

Overrides:

equals in class Tap<Config,Void,Output>

hashCode

public int hashCode()

Overrides:

hashCode in class Tap<Config,Void,Output>

toString

public String toString()

Overrides:

toString in class Tap<Config,Void,Output>