cascading.flow.hadoop
Class ProcessFlow<P>

java.lang.Object
  cascading.flow.BaseFlow<JobConf>
      cascading.flow.hadoop.HadoopFlow
          cascading.flow.hadoop.ProcessFlow<P>

All Implemented Interfaces:

Flow<JobConf>, UnitOfWork<FlowStats>

public class ProcessFlow<P>
extends HadoopFlow

Class ProcessFlow is a Flow subclass that supports custom Riffle jobs.

Use this class to allow custom Riffle jobs to participate in the Cascade scheduler. If other Flow instances in the Cascade share resources with this Flow instance, all participants will be scheduled according to their dependencies (topologically).

Though this class sub-classes HadoopFlow, it does not support all the methods available or features.

Currently FlowListeners are supported but the FlowListener.onThrowable(cascading.flow.Flow, Throwable) event is not.

Nested Class Summary

Nested classes/interfaces inherited from class cascading.flow.BaseFlow

BaseFlow.FlowHolder

Field Summary

Fields inherited from class cascading.flow.BaseFlow

flowStats, sinks, sources, stop, stopJobsOnExit, thread

Fields inherited from interface cascading.flow.Flow

CASCADING_FLOW_ID

Constructor Summary

ProcessFlow(Map<Object,Object> properties, String name, P process)
Constructor ProcessFlow creates a new ProcessFlow instance.

ProcessFlow(String name, P process)
Constructor ProcessFlow creates a new ProcessFlow instance.

Method Summary

void	cleanup()
void	complete()
P	getProcess() Method getProcess returns the process of this ProcessFlow object.
void	prepare()
void	setTapFromProcess() Method setTapFromProcess build Tap instance for the give process incoming and outgoing dependencies.
void	start()
void	stop()
String	toString()

Methods inherited from class cascading.flow.hadoop.HadoopFlow

getConfig, getConfigAsProperties, getConfigCopy, getFlowProcess, getMaxNumParallelSteps, getProperty, initConfig, initFromProperties, internalClean, internalShutdown, internalStart, isPreserveTemporaryFiles, newConfig, setConfigProperty, stepsAreLocal

Methods inherited from class cascading.flow.BaseFlow

addListener, addStepListener, areSinksStale, areSourcesNewer, createConfig, createFlowThread, deleteCheckpointsIfNotUpdate, deleteCheckpointsIfReplace, deleteSinks, deleteSinksIfNotUpdate, deleteSinksIfReplace, deleteTrapsIfNotUpdate, deleteTrapsIfReplace, fireOnCompleted, fireOnStarting, fireOnStopping, fireOnThrowable, getCascadeID, getCascadingServices, getCheckpointNames, getCheckpoints, getCheckpointsCollection, getClassPath, getFieldsFor, getFlowSession, getFlowSkipStrategy, getFlowStats, getFlowSteps, getFlowStepStrategy, getHolder, getID, getName, getPlatformInfo, getRunID, getSink, getSink, getSinkModified, getSinkNames, getSinks, getSinksCollection, getSource, getSourceNames, getSources, getSourcesCollection, getSpawnStrategy, getStats, getSubmitPriority, getTags, getTrapNames, getTraps, getTrapsCollection, handleExecutorShutdown, hasListeners, hasStepListeners, initialize, initializeNewJobsMap, initSteps, internalStopAllJobs, isSkipFlow, isStopJobsOnExit, logInfo, openSink, openSink, openSource, openSource, openTapForRead, openTapForWrite, openTrap, openTrap, presentSinkFields, presentSourceFields, registerShutdownHook, removeListener, removeStepListener, resourceExists, retrieveSinkFields, retrieveSourceFields, setCascade, setCheckpoints, setFlowSkipStrategy, setFlowStepGraph, setFlowStepStrategy, setName, setSinks, setSources, setSpawnStrategy, setSubmitPriority, setTraps, updateSchemes, writeDOT, writeStepsDOT

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

ProcessFlow

@ConstructorProperties(value={"name","process"})
public ProcessFlow(String name,
                                              P process)

Constructor ProcessFlow creates a new ProcessFlow instance.

Parameters:

name - of type String

process - of type JobConf

ProcessFlow

@ConstructorProperties(value={"properties","name","process"})
public ProcessFlow(Map<Object,Object> properties,
                                              String name,
                                              P process)

Constructor ProcessFlow creates a new ProcessFlow instance.

Parameters:

properties - of type Map

name - of type String

process - of type P

Method Detail

setTapFromProcess

public void setTapFromProcess()

Method setTapFromProcess build Tap instance for the give process incoming and outgoing dependencies.

This method may be called repeatedly to re-configure the source and sink taps.

getProcess

public P getProcess()

Method getProcess returns the process of this ProcessFlow object.

Returns:

the process (type P) of this ProcessFlow object.

prepare

public void prepare()

Specified by:

prepare in interface Flow<JobConf>

Specified by:

prepare in interface UnitOfWork<FlowStats>

Overrides:

prepare in class BaseFlow<JobConf>

start

public void start()

Specified by:

start in interface Flow<JobConf>

Specified by:

start in interface UnitOfWork<FlowStats>

Overrides:

start in class BaseFlow<JobConf>

stop

public void stop()

Specified by:

stop in interface Flow<JobConf>

Specified by:

stop in interface UnitOfWork<FlowStats>

Overrides:

stop in class BaseFlow<JobConf>

complete

public void complete()

Specified by:

complete in interface Flow<JobConf>

Specified by:

complete in interface UnitOfWork<FlowStats>

Overrides:

complete in class BaseFlow<JobConf>

cleanup

public void cleanup()

Specified by:

cleanup in interface Flow<JobConf>

Specified by:

cleanup in interface UnitOfWork<FlowStats>

Overrides:

cleanup in class BaseFlow<JobConf>

toString

public String toString()

Overrides:

toString in class BaseFlow<JobConf>