public class HfsProps extends Props
Flow
may or may not be required to have set. These properties are typically passed to a Flow
via a FlowConnector
.Modifier and Type | Field and Description |
---|---|
static String |
COMBINE_INPUT_FILES
Field COMBINE_INPUT_FILES
|
static String |
COMBINE_INPUT_FILES_SAFE_MODE
Field COMBINE_INPUT_FILES_SAFEMODE
|
static String |
COMBINE_INPUT_FILES_SIZE_MAX
Field COMBINE_INPUT_FILES_SIZE_MAX
|
protected Long |
combinedInputMaxSize |
protected Boolean |
combinedInputSafeMode |
static String |
LOCAL_MODE_SCHEME
Fields LOCAL_MODE_SCHEME *
|
protected String |
localModeScheme |
static String |
TEMPORARY_DIRECTORY
Field TEMPORARY_DIRECTORY
|
protected String |
temporaryDirectory |
protected Boolean |
useCombinedInput |
Constructor and Description |
---|
HfsProps() |
Modifier and Type | Method and Description |
---|---|
protected void |
addPropertiesTo(Properties properties) |
Long |
getCombinedInputMaxSize() |
String |
getLocalModeScheme() |
String |
getTemporaryDirectory() |
static HfsProps |
hfsProps()
Creates a new HfsProps instance.
|
boolean |
isUseCombinedInput() |
boolean |
isUseCombinedInputSafeMode() |
HfsProps |
setCombinedInputMaxSize(long combinedInputMaxSize)
Method setCombinedInputMaxSize sets the maximum input split size to be used.
|
static void |
setCombinedInputMaxSize(Map<Object,Object> properties,
Long size)
Method setCombinedInputMaxSize sets the maximum input split size to be used.
|
static void |
setLocalModeScheme(Map<Object,Object> properties,
String scheme)
Method setLocalModeScheme provides a means to change the scheme value used to detect when a
MapReduce job should be run in Hadoop local mode.
|
HfsProps |
setLocalModeScheme(String localModeScheme)
Method setLocalModeScheme provides a means to change the scheme value used to detect when a
MapReduce job should be run in Hadoop local mode.
|
static void |
setTemporaryDirectory(Map<Object,Object> properties,
String temporaryDirectory)
Method setTemporaryDirectory sets the temporary directory on the given properties object.
|
HfsProps |
setTemporaryDirectory(String temporaryDirectory)
Method setTemporaryDirectory sets the temporary directory for use on the underlying filesystem.
|
HfsProps |
setUseCombinedInput(boolean useCombinedInput)
Method setUseCombinedInput provides a means to indicate whether to leverage
CombineFileInputFormat for the input format. |
static void |
setUseCombinedInput(Map<Object,Object> properties,
Boolean combine)
Method setUseCombinedInput provides a means to indicate whether to leverage
CombineFileInputFormat for the input format. |
HfsProps |
setUseCombinedInputSafeMode(boolean combinedInputSafeMode)
Method setUseCombinedInputSafeMode toggles safe mode when using
CombineFileInputFormat . |
static void |
setUseCombinedInputSafeMode(Map<Object,Object> properties,
Boolean safeMode)
Method setUseCombinedInputSafeMode toggles safe mode when using
CombineFileInputFormat . |
buildProperties, buildProperties, buildProperties, buildProperties, setProperties, setProperties
public static final String TEMPORARY_DIRECTORY
public static final String LOCAL_MODE_SCHEME
public static final String COMBINE_INPUT_FILES
public static final String COMBINE_INPUT_FILES_SAFE_MODE
public static final String COMBINE_INPUT_FILES_SIZE_MAX
protected String temporaryDirectory
protected String localModeScheme
protected Boolean useCombinedInput
protected Long combinedInputMaxSize
protected Boolean combinedInputSafeMode
public HfsProps()
public static void setTemporaryDirectory(Map<Object,Object> properties, String temporaryDirectory)
properties
- of type MaptemporaryDirectory
- of type Stringpublic static void setLocalModeScheme(Map<Object,Object> properties, String scheme)
"file"
, set to
"none"
to disable entirely.properties
- of type Mapscheme
- a Stringpublic static void setUseCombinedInput(Map<Object,Object> properties, Boolean combine)
CombineFileInputFormat
for the input format. By default it is false.
Use setCombinedInputMaxSize(long)
to set the max split/combined input size. Other specific
properties must be specified directly if needed. Specifically "mapred.min.split.size.per.node" and
"mapred.min.split.size.per.rack", which are 0 by default.properties
- of type Mapcombine
- a booleanpublic static void setUseCombinedInputSafeMode(Map<Object,Object> properties, Boolean safeMode)
CombineFileInputFormat
. Safe mode will throw an exception if the underlying
InputFormat is not of type FileInputFormat
. If safeMode is off a warning will
be logged instead. safeMode is on by default.
Setting this property when not setting setUseCombinedInput(boolean)
to true has no effect.properties
- of type MapsafeMode
- a booleanpublic static void setCombinedInputMaxSize(Map<Object,Object> properties, Long size)
properties
- of type Mapsize
- of type longpublic static HfsProps hfsProps()
public String getTemporaryDirectory()
public HfsProps setTemporaryDirectory(String temporaryDirectory)
temporaryDirectory
- of type Stringpublic String getLocalModeScheme()
public HfsProps setLocalModeScheme(String localModeScheme)
"file"
, set to
"none"
to disable entirely.localModeScheme
- of type Stringpublic boolean isUseCombinedInput()
public HfsProps setUseCombinedInput(boolean useCombinedInput)
CombineFileInputFormat
for the input format. By default it is false.useCombinedInput
- booleanpublic Long getCombinedInputMaxSize()
public HfsProps setCombinedInputMaxSize(long combinedInputMaxSize)
setUseCombinedInput(boolean)
is true
.combinedInputMaxSize
- of type longpublic boolean isUseCombinedInputSafeMode()
public HfsProps setUseCombinedInputSafeMode(boolean combinedInputSafeMode)
CombineFileInputFormat
. Safe mode will throw an exception if the underlying
InputFormat is not of type FileInputFormat
. If safeMode is off a warning will
be logged instead. safeMode is on by default.
Setting this property when not setting setUseCombinedInput(boolean)
to true has no effect.combinedInputSafeMode
- booleanprotected void addPropertiesTo(Properties properties)
addPropertiesTo
in class Props
Copyright © 2007-2015 Concurrent, Inc. All Rights Reserved.