cascading.flow

Interface FlowNode

All Superinterfaces:

ProcessModel

All Known Implementing Classes:

BaseFlowNode

public interface FlowNode
extends ProcessModel

Class FlowNode represents the smallest parallelizable unit of work. It is a child to a FlowStep and may have many siblings within the FlowStep.

A FlowNode is commonly executed as one or more slices, where a slice is a JVM executing against a portion of data.

Most slices within a FlowNode are identical, except for the sub-set of data they will be processing against.

But on some platforms, like MapReduce, a slice is executing a single flow pipeline. Thus a FlowNode may consist of some set of pipelines (or pipeline graph). One pipeline per 'streamed' input source Tap.

In a MapReduce model (like Apache Hadoop MapReduce) a FlowNode can by the Map or Reduce side of a job (where a job is a FlowStep).

In a DAG model (like Apache Tez), a FlowNode is a 'vertex', and the 'DAG' is a FlowStep.

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	CASCADING_FLOW_NODE

Method Summary

Methods

Modifier and Type	Method and Description
java.util.Collection<? extends FlowElement>	getFlowElementsFor(java.lang.Enum annotation)
java.util.Map<java.lang.String,java.lang.String>	getFlowNodeDescriptor() Returns an immutable map of properties giving more details about the FlowNode object.
FlowNodeStats	getFlowNodeStats()
FlowStep	getFlowStep()
java.lang.String	getID()
java.util.Collection<? extends Scope>	getNextScopes(FlowElement flowElement)
ElementGraph	getPipelineGraphFor(FlowElement streamedSource)
java.util.List<? extends ElementGraph>	getPipelineGraphs()
java.util.Collection<? extends Scope>	getPreviousScopes(FlowElement flowElement)
java.util.Set<java.lang.String>	getSinkElementNames()
java.util.Set<? extends FlowElement>	getSinkElements(java.lang.Enum annotation)
java.util.Set<java.lang.String>	getSinkTapNames(Tap flowElement)
java.util.Set<java.lang.String>	getSourceElementNames()
java.util.Set<? extends FlowElement>	getSourceElements(java.lang.Enum annotation)
java.util.Set<java.lang.String>	getSourceTapNames(Tap flowElement)
Tap	getTrap(java.lang.String branchName)
java.util.Collection<? extends Tap>	getTraps()

Methods inherited from interface cascading.flow.planner.process.ProcessModel

addProcessAnnotation, addProcessAnnotation, getElementGraph, getGroups, getName, getOrdinal, getProcessAnnotations, getSinkElements, getSinkTaps, getSourceElements, getSourceTaps, getSubmitPriority, getTrapMap

Field Detail

CASCADING_FLOW_NODE

static final java.lang.String CASCADING_FLOW_NODE

See Also:

Constant Field Values

Method Detail

getID

java.lang.String getID()

Specified by:

getID in interface ProcessModel

getFlowNodeDescriptor

java.util.Map<java.lang.String,java.lang.String> getFlowNodeDescriptor()

Returns an immutable map of properties giving more details about the FlowNode object.

FlowNode descriptions provide meta-data to monitoring systems describing the workload a given FlowNode represents. For known description types, see FlowNodeDescriptors.

Returns:

Map

getFlowNodeStats

FlowNodeStats getFlowNodeStats()

getFlowStep

FlowStep getFlowStep()

getFlowElementsFor

java.util.Collection<? extends FlowElement> getFlowElementsFor(java.lang.Enum annotation)

getSourceElements

java.util.Set<? extends FlowElement> getSourceElements(java.lang.Enum annotation)

getSinkElements

java.util.Set<? extends FlowElement> getSinkElements(java.lang.Enum annotation)

getSourceElementNames

java.util.Set<java.lang.String> getSourceElementNames()

getSinkElementNames

java.util.Set<java.lang.String> getSinkElementNames()

getSourceTapNames

java.util.Set<java.lang.String> getSourceTapNames(Tap flowElement)

getSinkTapNames

java.util.Set<java.lang.String> getSinkTapNames(Tap flowElement)

getTrap

Tap getTrap(java.lang.String branchName)

getTraps

java.util.Collection<? extends Tap> getTraps()

getPreviousScopes

java.util.Collection<? extends Scope> getPreviousScopes(FlowElement flowElement)

getNextScopes

java.util.Collection<? extends Scope> getNextScopes(FlowElement flowElement)

getPipelineGraphs

java.util.List<? extends ElementGraph> getPipelineGraphs()

getPipelineGraphFor

ElementGraph getPipelineGraphFor(FlowElement streamedSource)