public class FlowRuntimeProps extends Props
Flow
specific runtime properties through
a FlowConnector
.
These properties apply to the cluster or remote side of the Flow execution. For client (or local) side properties
see FlowProps
.
Available properties are:
FlowNode
.
In MapReduce this is the number of reducers. In Tez DAG this is the scatter gather parallelization.Modifier and Type | Field and Description |
---|---|
static String |
COMBINE_SPLITS |
static String |
GATHER_PARTITIONS |
static String |
LOG_COUNTERS |
Constructor and Description |
---|
FlowRuntimeProps() |
Modifier and Type | Method and Description |
---|---|
FlowRuntimeProps |
addLogCounter(Enum counter)
Method addLogCounter adds a new counter to be logged when a cluster side slice completes.
|
FlowRuntimeProps |
addLogCounter(String group,
String counter)
Method addLogCounter adds a new counter to be logged when a cluster side slice completes.
|
protected void |
addPropertiesTo(Properties properties) |
static FlowRuntimeProps |
flowRuntimeProps() |
Boolean |
getCombineSplits() |
int |
getGatherPartitions()
Method getGatherPartitions returns the number of gather partitions
|
FlowRuntimeProps |
setCombineSplits(Boolean combineSplits)
Method setCombineSplits will enable or disable combining of 'splits' on sources.
|
FlowRuntimeProps |
setGatherPartitions(int gatherPartitions)
Method setGatherPartitions sets the default number of gather partitions each
FlowNode
should use. |
buildProperties, buildProperties, buildProperties, buildProperties, setProperties, setProperties
public static final String GATHER_PARTITIONS
public static final String LOG_COUNTERS
public static final String COMBINE_SPLITS
public FlowRuntimeProps()
public static FlowRuntimeProps flowRuntimeProps()
public int getGatherPartitions()
public FlowRuntimeProps setGatherPartitions(int gatherPartitions)
FlowNode
should use.gatherPartitions
- number of gather partitions to use per nodepublic FlowRuntimeProps addLogCounter(Enum counter)
counter
- the Enum counter to logpublic FlowRuntimeProps addLogCounter(String group, String counter)
group
- the String counter group to logcounter
- the String counter name to logpublic Boolean getCombineSplits()
public FlowRuntimeProps setCombineSplits(Boolean combineSplits)
Tap
source resource. Combining
small splits into larger ones both reduce parallelism, but also reduce overhead of starting
work on a very small data set.
This is commonly done when sourcing large numbers of very small files.
Setting this value will change the default, which is a platform dependent value.combineSplits
- protected void addPropertiesTo(Properties properties)
addPropertiesTo
in class Props
Copyright © 2007-2015 Concurrent, Inc. All Rights Reserved.