public class Hadoop2MR1Planner extends HadoopPlanner
Hadoop2MR1FlowConnector
.
Notes:
Custom JobConf propertiesHadoopPlanner.copyJobConf(java.util.Map, org.apache.hadoop.mapred.JobConf)
on a map properties object before constructing a new Hadoop2MR1FlowConnector
.
A better practice would be to set Hadoop properties directly on the map properties object handed to the FlowConnector.
All values in the map will be passed to a new default JobConf instance to be used as defaults for all resulting
Flow instances.
For example, properties.set("mapred.child.java.opts","-Xmx512m");
would convince Hadoop
to spawn all child jvms with a heap of 512MB.FlowPlanner.TempTapElementFactory
Modifier and Type | Field and Description |
---|---|
static String |
PLATFORM_NAME |
checkpointTapRootPath, defaultAssertionLevel, defaultDebugLevel, defaultProperties, TRACE_PLAN_PATH, TRACE_PLAN_TRANSFORM_PATH, TRACE_STATS_PATH
Constructor and Description |
---|
Hadoop2MR1Planner() |
Modifier and Type | Method and Description |
---|---|
protected void |
checkPlatform(Configuration conf) |
static void |
copyConfiguration(Map<Object,Object> properties,
Configuration configuration)
Method copyJobConf adds the given JobConf values to the given properties object.
|
static void |
copyProperties(Configuration configuration,
Map<Object,Object> properties)
Method copyProperties adds the given Map values to the given JobConf object.
|
PlannerInfo |
getPlannerInfo(String registryName) |
configRuleRegistryDefaults, copyJobConf, copyProperties, createFlow, createFlowStep, createJobConf, getDefaultConfig, getDefaultURIScheme, getPlatformInfo, getURIScheme, initialize, makeTempTap
buildFlow, createFlowElementGraph, createFlowNode, getAssertionLevel, getDebugLevel, getDefaultProperties, handleExceptionDuringPlanning, makeFlowNodeName, makeFlowStepName, makeTempTap, resolveAssemblyPlanners, resolveTails, verifyAllTaps, verifyAssembly, verifyCheckpoints, verifyPipeAssemblyEndPoints, verifyResult, verifyResultInternal, verifySourceNotSinks, verifyTaps, verifyTraps
public static final String PLATFORM_NAME
public Hadoop2MR1Planner()
public static void copyConfiguration(Map<Object,Object> properties, Configuration configuration)
properties
- of type Mapconfiguration
- of type JobConfpublic static void copyProperties(Configuration configuration, Map<Object,Object> properties)
configuration
- of type JobConfproperties
- of type Mappublic PlannerInfo getPlannerInfo(String registryName)
getPlannerInfo
in class HadoopPlanner
protected void checkPlatform(Configuration conf)
checkPlatform
in class HadoopPlanner
Copyright © 2007-2015 Concurrent, Inc. All Rights Reserved.