cascading.flow

Class FlowDef

java.lang.Object

cascading.property.UnitOfWorkDef<FlowDef>

cascading.flow.FlowDef

public class FlowDef
extends UnitOfWorkDef<FlowDef>

Class FlowDef is a fluent interface for defining a Flow.

This allows for ad-hoc building of Flow data and meta-data, like tags.

Instead of calling one of the FlowConnector connect methods, FlowConnector.connect(FlowDef) can be called.

Field Summary

Fields

Modifier and Type	Field and Description
protected java.util.List<AssemblyPlanner>	assemblyPlanners
protected AssertionLevel	assertionLevel
protected java.util.Map<java.lang.String,Tap>	checkpoints
protected java.util.List<java.lang.String>	classPath
protected DebugLevel	debugLevel
protected java.util.HashMap<java.lang.String,java.lang.String>	flowDescriptor
protected java.lang.String	runID
protected java.util.Map<java.lang.String,Tap>	sinks
protected java.util.Map<java.lang.String,Tap>	sources
protected java.util.List<Pipe>	tails
protected java.util.Map<java.lang.String,Tap>	traps

Fields inherited from class cascading.property.UnitOfWorkDef

name, tags

Constructor Summary

Constructors

Modifier	Constructor and Description
	FlowDef() Constructor FlowDef creates a new FlowDef instance.
protected	FlowDef(FlowDef flowDef, java.util.Map<java.lang.String,Tap> sources, java.util.Map<java.lang.String,Tap> sinks, java.util.Map<java.lang.String,Tap> traps, java.util.Map<java.lang.String,Tap> checkpoints)

Method Summary

Modifier and Type	Method and Description
FlowDef	addAssemblyPlanner(AssemblyPlanner assemblyPlanner) Method addAssemblyPlanner adds new AssemblyPlanner instances to be evaluated.
FlowDef	addCheckpoint(Checkpoint pipe, Tap checkpoint) Method addCheckpoint adds a new checkpoint Tap named after the given Checkpoint for use in the resulting Flow.
FlowDef	addCheckpoint(java.lang.String name, Tap checkpoint) Method addCheckpoint adds a new named checkpoint Tap for use in the resulting Flow.
FlowDef	addCheckpoints(java.util.Map<java.lang.String,Tap> checkpoints) Method addCheckpoints adds a Map of the names and Tap pairs.
FlowDef	addDescription(java.lang.String description) Method addDescription adds a user readable description to the flowDescriptor.
FlowDef	addDescription(java.lang.String key, java.lang.String value) Method addDescription adds a description to the flowDescriptor.
FlowDef	addDescriptions(java.util.Map<java.lang.String,java.lang.String> descriptions) Method addProperties adds all properties in the given map in order to the flowDescriptor.
FlowDef	addSink(Pipe tail, Tap sink) Method addSink adds a new sink Tap named after the given Pipe for use in the resulting Flow.
FlowDef	addSink(java.lang.String name, Tap sink) Method addSink adds a new named sink Tap for use in the resulting Flow.
FlowDef	addSinks(java.util.Map<java.lang.String,Tap> sinks) Method addSinks adds a Map of the named and Tap pairs.
FlowDef	addSource(Pipe pipe, Tap source) Method addSource adds a new source Tap named after the given Pipe for use in the resulting Flow.
FlowDef	addSource(java.lang.String name, Tap source) Method addSource adds a new named source Tap for use in the resulting Flow.
FlowDef	addSources(java.util.Map<java.lang.String,Tap> sources) Method addSources adds a map of name and Tap pairs.
FlowDef	addTail(Pipe tail) Method addTail adds a new Pipe to this FlowDef that represents a tail in a pipe assembly.
FlowDef	addTails(java.util.Collection<Pipe> tails) Method addTails adds a Collection of tails.
FlowDef	addTails(Pipe... tails) Method addTails adds an array of tails.
FlowDef	addTailSink(Pipe tail, Tap sink) Method addTailSink adds the tail Pipe and sink Tap to this FlowDef.
FlowDef	addToClassPath(java.lang.String artifact) Adds each given artifact to the classpath the assembly will execute under allowing Operators to dynamically load classes and resources from a ClassLoader.
FlowDef	addTrap(Pipe pipe, Tap trap) Method addTrap adds a new trap Tap named after the given Pipe for use in the resulting Flow.
FlowDef	addTrap(java.lang.String name, Tap trap) Method addTrap adds a new named trap Tap for use in the resulting Flow.
FlowDef	addTraps(java.util.Map<java.lang.String,Tap> traps) Method addTraps adds a Map of the names and Tap pairs.
static FlowDef	flowDef() Creates a new instance of a FlowDef.
java.util.List<AssemblyPlanner>	getAssemblyPlanners() Method getAssemblyPlanners returns the current registered AssemblyPlanners.
AssertionLevel	getAssertionLevel()
java.util.Map<java.lang.String,Tap>	getCheckpoints() Method getCheckpoints returns the checkpoint taps of this FlowDef object.
java.util.Map<java.lang.String,Tap>	getCheckpointsCopy() Method getCheckpointsCopy returns a copy of the checkpoint tap Map.
java.util.List<java.lang.String>	getClassPath()
DebugLevel	getDebugLevel()
java.util.HashMap<java.lang.String,java.lang.String>	getFlowDescriptor() Method getFlowDescriptor returns the flowDescriptor of this FlowDef.
java.lang.String	getRunID()
java.util.Map<java.lang.String,Tap>	getSinks() Method getSinks returns the sinks of this FlowDef object.
java.util.Map<java.lang.String,Tap>	getSinksCopy() Method getSinksCopy returns a copy of the sink Map.
java.util.Map<java.lang.String,Tap>	getSources() Method getSources returns the sources of this FlowDef object.
java.util.Map<java.lang.String,Tap>	getSourcesCopy() Method getSourcesCopy returns a copy of the sources Map.
java.util.List<Pipe>	getTails() Method getTails returns all the current pipe assembly tails the FlowDef holds.
Pipe[]	getTailsArray() Method getTailsArray returns all the current pipe assembly tails the FlowDef holds.
java.util.Map<java.lang.String,Tap>	getTraps() Method getTraps returns the traps of this FlowDef object.
java.util.Map<java.lang.String,Tap>	getTrapsCopy() Method getTrapsCopy returns a copy of the trap Map.
FlowDef	setAssertionLevel(AssertionLevel assertionLevel)
FlowDef	setDebugLevel(DebugLevel debugLevel)
FlowDef	setRunID(java.lang.String runID) Method setRunID sets the checkpoint run or execution ID to be used to find prior failed runs against this runID.

Methods inherited from class cascading.property.UnitOfWorkDef

addTag, addTags, getName, getTags, setName

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

sources

protected java.util.Map<java.lang.String,Tap> sources

sinks

protected java.util.Map<java.lang.String,Tap> sinks

traps

protected java.util.Map<java.lang.String,Tap> traps

checkpoints

protected java.util.Map<java.lang.String,Tap> checkpoints

classPath

protected java.util.List<java.lang.String> classPath

tails

protected java.util.List<Pipe> tails

assemblyPlanners

protected java.util.List<AssemblyPlanner> assemblyPlanners

flowDescriptor

protected java.util.HashMap<java.lang.String,java.lang.String> flowDescriptor

assertionLevel

protected AssertionLevel assertionLevel

debugLevel

protected DebugLevel debugLevel

runID

protected java.lang.String runID

Constructor Detail

FlowDef

public FlowDef()

Constructor FlowDef creates a new FlowDef instance.

FlowDef

protected FlowDef(FlowDef flowDef,
                  java.util.Map<java.lang.String,Tap> sources,
                  java.util.Map<java.lang.String,Tap> sinks,
                  java.util.Map<java.lang.String,Tap> traps,
                  java.util.Map<java.lang.String,Tap> checkpoints)

Method Detail

flowDef

public static FlowDef flowDef()

Creates a new instance of a FlowDef.

Returns:

a FlowDef

getAssemblyPlanners

public java.util.List<AssemblyPlanner> getAssemblyPlanners()

Method getAssemblyPlanners returns the current registered AssemblyPlanners.

Returns:

a List of AssemblyPlanner instances

addAssemblyPlanner

public FlowDef addAssemblyPlanner(AssemblyPlanner assemblyPlanner)

Method addAssemblyPlanner adds new AssemblyPlanner instances to be evaluated.

Parameters:

assemblyPlanner - of type AssemblyPlanner

Returns:

a FlowDef

getSources

public java.util.Map<java.lang.String,Tap> getSources()

Method getSources returns the sources of this FlowDef object.

Returns:

the sources (type Map) of this FlowDef object.

getSourcesCopy

public java.util.Map<java.lang.String,Tap> getSourcesCopy()

Method getSourcesCopy returns a copy of the sources Map.

Returns:

the sourcesCopy (type Map) of this FlowDef object.

getFlowDescriptor

public java.util.HashMap<java.lang.String,java.lang.String> getFlowDescriptor()

Method getFlowDescriptor returns the flowDescriptor of this FlowDef.

Returns:

the flowDescriptor of this FlowDef object.

addSource

public FlowDef addSource(java.lang.String name,
                         Tap source)

Method addSource adds a new named source Tap for use in the resulting Flow.

Parameters:

name - of String

source - of Tap

Returns:

FlowDef

addSource

public FlowDef addSource(Pipe pipe,
                         Tap source)

Method addSource adds a new source Tap named after the given Pipe for use in the resulting Flow.

If the given pipe is not a head pipe, it will be resolved. If more than one is found, an IllegalArgumentException will be thrown.

Parameters:

pipe - of Pipe

source - of Tap

Returns:

FlowDef

addSources

public FlowDef addSources(java.util.Map<java.lang.String,Tap> sources)

Method addSources adds a map of name and Tap pairs.

Parameters:

sources - of Map

Returns:

FlowDef

addDescription

public FlowDef addDescription(java.lang.String description)

Method addDescription adds a user readable description to the flowDescriptor.

This uses the FlowDescriptors.DESCRIPTION key.

addDescription

public FlowDef addDescription(java.lang.String key,
                              java.lang.String value)

Method addDescription adds a description to the flowDescriptor.

Flow descriptions provide meta-data to monitoring systems describing the workload a given Flow represents. For known description types, see FlowDescriptors.

If an existing key exists, it will be appended to the original value using FlowDescriptors.VALUE_SEPARATOR.

Parameters:

key - The key as a String.

value - The value as a String.

Returns:

FlowDef

addDescriptions

public FlowDef addDescriptions(java.util.Map<java.lang.String,java.lang.String> descriptions)

Method addProperties adds all properties in the given map in order to the flowDescriptor. If the given Map has an explicit order, it will be preserved.

Flow descriptions provide meta-data to monitoring systems describing the workload a given Flow represents. For known description types, see FlowDescriptors.

Parameters:

descriptions - The properties to be added to the map.

Returns:

FlowDef

getSinks

public java.util.Map<java.lang.String,Tap> getSinks()

Method getSinks returns the sinks of this FlowDef object.

Returns:

the sinks (type Map) of this FlowDef object.

getSinksCopy

public java.util.Map<java.lang.String,Tap> getSinksCopy()

Method getSinksCopy returns a copy of the sink Map.

Returns:

the sinksCopy (type Map) of this FlowDef object.

addSink

public FlowDef addSink(java.lang.String name,
                       Tap sink)

Method addSink adds a new named sink Tap for use in the resulting Flow.

Parameters:

name - of String

sink - of Tap

Returns:

FlowDef

addSink

public FlowDef addSink(Pipe tail,
                       Tap sink)

Method addSink adds a new sink Tap named after the given Pipe for use in the resulting Flow.

Parameters:

tail - of Pipe

sink - of Tap

Returns:

FlowDef

addTailSink

public FlowDef addTailSink(Pipe tail,
                           Tap sink)

Method addTailSink adds the tail Pipe and sink Tap to this FlowDef.

This is a convenience method for adding both a tail and sink simultaneously. There isn't a similar method for heads and sources as the head Pipe can always be derived.

Parameters:

tail - of Pipe

sink - of Tap

Returns:

FlowDef

addSinks

public FlowDef addSinks(java.util.Map<java.lang.String,Tap> sinks)

Method addSinks adds a Map of the named and Tap pairs.

Parameters:

sinks - of Map

Returns:

FlowDef

getTraps

public java.util.Map<java.lang.String,Tap> getTraps()

Method getTraps returns the traps of this FlowDef object.

Returns:

the traps (type Map) of this FlowDef object.

getTrapsCopy

public java.util.Map<java.lang.String,Tap> getTrapsCopy()

Method getTrapsCopy returns a copy of the trap Map.

Returns:

the trapsCopy (type Map) of this FlowDef object.

addTrap

public FlowDef addTrap(java.lang.String name,
                       Tap trap)

Method addTrap adds a new named trap Tap for use in the resulting Flow.

Parameters:

name - of String

trap - of Tap

Returns:

FlowDef

addTrap

public FlowDef addTrap(Pipe pipe,
                       Tap trap)

Method addTrap adds a new trap Tap named after the given Pipe for use in the resulting Flow.

Parameters:

pipe - of Pipe

trap - of Tap

Returns:

FlowDef

addTraps

public FlowDef addTraps(java.util.Map<java.lang.String,Tap> traps)

Method addTraps adds a Map of the names and Tap pairs.

Parameters:

traps - of Map

Returns:

FlowDef

getCheckpoints

public java.util.Map<java.lang.String,Tap> getCheckpoints()

Method getCheckpoints returns the checkpoint taps of this FlowDef object.

Returns:

the checkpoints (type Map) of this FlowDef object.

getCheckpointsCopy

public java.util.Map<java.lang.String,Tap> getCheckpointsCopy()

Method getCheckpointsCopy returns a copy of the checkpoint tap Map.

Returns:

the checkpointsCopy (type Map) of this FlowDef object.

addCheckpoint

public FlowDef addCheckpoint(java.lang.String name,
                             Tap checkpoint)

Method addCheckpoint adds a new named checkpoint Tap for use in the resulting Flow.

Parameters:

name - of String

checkpoint - of Tap

Returns:

FlowDef

addCheckpoint

public FlowDef addCheckpoint(Checkpoint pipe,
                             Tap checkpoint)

Method addCheckpoint adds a new checkpoint Tap named after the given Checkpoint for use in the resulting Flow.

Parameters:

pipe - of Pipe

checkpoint - of Tap

Returns:

FlowDef

addCheckpoints

public FlowDef addCheckpoints(java.util.Map<java.lang.String,Tap> checkpoints)

Method addCheckpoints adds a Map of the names and Tap pairs.

Parameters:

checkpoints - of Map

Returns:

FlowDef

getTails

public java.util.List<Pipe> getTails()

Method getTails returns all the current pipe assembly tails the FlowDef holds.

Returns:

the tails (type List) of this FlowDef object.

getTailsArray

public Pipe[] getTailsArray()

Method getTailsArray returns all the current pipe assembly tails the FlowDef holds.

Returns:

the tailsArray (type Pipe[]) of this FlowDef object.

addTail

public FlowDef addTail(Pipe tail)

Method addTail adds a new Pipe to this FlowDef that represents a tail in a pipe assembly.

Be sure to add a sink tap that has the same name as this tail.

Parameters:

tail - of Pipe

Returns:

FlowDef

addTails

public FlowDef addTails(java.util.Collection<Pipe> tails)

Method addTails adds a Collection of tails.

Parameters:

tails - of Collection

Returns:

FlowDef

addTails

public FlowDef addTails(Pipe... tails)

Method addTails adds an array of tails.

Parameters:

tails - of Pipe...

Returns:

FlowDef

setAssertionLevel

public FlowDef setAssertionLevel(AssertionLevel assertionLevel)

getAssertionLevel

public AssertionLevel getAssertionLevel()

setDebugLevel

public FlowDef setDebugLevel(DebugLevel debugLevel)

getDebugLevel

public DebugLevel getDebugLevel()

setRunID

public FlowDef setRunID(java.lang.String runID)

Method setRunID sets the checkpoint run or execution ID to be used to find prior failed runs against this runID.

When given, and a Flow fails to execute, a subsequent attempt to run the same Flow with the same runID will allow the Flow instance to start where it left off.

Not all planners support this feature.

A Flow name is required when using a runID.

Parameters:

runID - of type String

Returns:

FlowDef

getRunID

public java.lang.String getRunID()

getClassPath

public java.util.List<java.lang.String> getClassPath()

addToClassPath

public FlowDef addToClassPath(java.lang.String artifact)

Adds each given artifact to the classpath the assembly will execute under allowing Operators to dynamically load classes and resources from a ClassLoader.

Parameters:

artifact - a jar or other file String path

Returns:

FlowDef