cascading.flow.hadoop
Class ProcessFlow<P>
java.lang.Object
cascading.flow.BaseFlow <JobConf >
cascading.flow.hadoop.HadoopFlow
cascading.flow.hadoop.ProcessFlow<P>
All Implemented Interfaces: Flow <JobConf >, UnitOfWork <FlowStats >
public class ProcessFlow<P> extends HadoopFlow
Class ProcessFlow is a Flow
subclass that supports custom Riffle jobs.
Use this class to allow custom Riffle jobs to participate in the Cascade
scheduler. If
other Flow instances in the Cascade share resources with this Flow instance, all participants will be scheduled
according to their dependencies (topologically).
Though this class sub-classes HadoopFlow
, it does not support all the methods available or features.
Currently FlowListener
s are supported but the
FlowListener.onThrowable(cascading.flow.Flow, Throwable)
event is not.
Methods inherited from class cascading.flow.hadoop.HadoopFlow
getConfig , getConfigAsProperties , getConfigCopy , getFlowProcess , getMaxNumParallelSteps , getProperty , initConfig , initFromProperties , internalClean , internalShutdown , internalStart , isPreserveTemporaryFiles , newConfig , setConfigProperty , stepsAreLocal
Methods inherited from class cascading.flow.BaseFlow
addListener , addStepListener , areSinksStale , areSourcesNewer , createConfig , createFlowThread , deleteCheckpointsIfNotUpdate , deleteCheckpointsIfReplace , deleteSinks , deleteSinksIfNotUpdate , deleteSinksIfReplace , deleteTrapsIfNotUpdate , deleteTrapsIfReplace , fireOnCompleted , fireOnStarting , fireOnStopping , fireOnThrowable , getCascadeID , getCascadingServices , getCheckpointNames , getCheckpoints , getCheckpointsCollection , getClassPath , getFieldsFor , getFlowDescriptor , getFlowSession , getFlowSkipStrategy , getFlowStats , getFlowSteps , getFlowStepStrategy , getHolder , getID , getName , getPlatformInfo , getRunID , getSink , getSink , getSinkModified , getSinkNames , getSinks , getSinksCollection , getSource , getSourceNames , getSources , getSourcesCollection , getSpawnStrategy , getStats , getSubmitPriority , getTags , getTrapNames , getTraps , getTrapsCollection , handleExecutorShutdown , hasListeners , hasStepListeners , initialize , initializeNewJobsMap , initSteps , internalStopAllJobs , isSkipFlow , isStopJobsOnExit , logInfo , openSink , openSink , openSource , openSource , openTapForRead , openTapForWrite , openTrap , openTrap , presentSinkFields , presentSourceFields , registerShutdownHook , removeListener , removeStepListener , resourceExists , retrieveSinkFields , retrieveSourceFields , setCascade , setCheckpoints , setFlowSkipStrategy , setFlowStepGraph , setFlowStepStrategy , setName , setSinks , setSources , setSpawnStrategy , setSubmitPriority , setTraps , updateSchemes , writeDOT , writeStepsDOT
ProcessFlow
@ConstructorProperties (value ={"name","process"})
public ProcessFlow (String name,
P process)
Constructor ProcessFlow creates a new ProcessFlow instance.
Parameters: name
- of type Stringprocess
- of type JobConf
ProcessFlow
@ConstructorProperties (value ={"properties","name","process"})
public ProcessFlow (Map <Object ,Object > properties,
String name,
P process)
Constructor ProcessFlow creates a new ProcessFlow instance.
Parameters: properties
- of type Mapname
- of type Stringprocess
- of type P
ProcessFlow
@ConstructorProperties (value ={"properties","name","process","flowDescriptor"})
public ProcessFlow (Map <Object ,Object > properties,
String name,
P process,
Map <String ,String > flowDescriptor)
Constructor ProcessFlow creates a new ProcessFlow instance.
Parameters: properties
- of type Mapname
- of type Stringprocess
- of type PflowDescriptor
- pf type LinkedHashMap
setTapFromProcess
public void setTapFromProcess ()
Method setTapFromProcess build Tap
instance for the give process incoming and outgoing dependencies.
This method may be called repeatedly to re-configure the source and sink taps.
getProcess
public P getProcess ()
Method getProcess returns the process of this ProcessFlow object.
Returns: the process (type P) of this ProcessFlow object.
prepare
public void prepare ()
Specified by: prepare
in interface Flow <JobConf >
Specified by: prepare
in interface UnitOfWork <FlowStats >
Overrides: prepare
in class BaseFlow <JobConf >
start
public void start ()
Specified by: start
in interface Flow <JobConf >
Specified by: start
in interface UnitOfWork <FlowStats >
Overrides: start
in class BaseFlow <JobConf >
stop
public void stop ()
Specified by: stop
in interface Flow <JobConf >
Specified by: stop
in interface UnitOfWork <FlowStats >
Overrides: stop
in class BaseFlow <JobConf >
complete
public void complete ()
Specified by: complete
in interface Flow <JobConf >
Specified by: complete
in interface UnitOfWork <FlowStats >
Overrides: complete
in class BaseFlow <JobConf >
cleanup
public void cleanup ()
Specified by: cleanup
in interface Flow <JobConf >
Specified by: cleanup
in interface UnitOfWork <FlowStats >
Overrides: cleanup
in class BaseFlow <JobConf >
toString
public String toString ()
Overrides: toString
in class BaseFlow <JobConf >
Copyright © 2007-2014 Concurrent, Inc. All Rights Reserved.