public class HfsProps extends Props
Flow
may or may not be required to have set. These properties are typically passed to a Flow
via a FlowConnector
.Modifier and Type | Field and Description |
---|---|
static java.lang.String |
COMBINE_INPUT_FILES
Field COMBINE_INPUT_FILES
|
static java.lang.String |
COMBINE_INPUT_FILES_SAFE_MODE
Field COMBINE_INPUT_FILES_SAFEMODE
|
static java.lang.String |
COMBINE_INPUT_FILES_SIZE_MAX
Field COMBINE_INPUT_FILES_SIZE_MAX
|
protected java.lang.Long |
combinedInputMaxSize |
protected java.lang.Boolean |
combinedInputSafeMode |
static java.lang.String |
LOCAL_MODE_SCHEME
Fields LOCAL_MODE_SCHEME *
|
protected java.lang.String |
localModeScheme |
static java.lang.String |
TEMPORARY_DIRECTORY
Field TEMPORARY_DIRECTORY
|
protected java.lang.String |
temporaryDirectory |
protected java.lang.Boolean |
useCombinedInput |
Constructor and Description |
---|
HfsProps() |
Modifier and Type | Method and Description |
---|---|
protected void |
addPropertiesTo(java.util.Properties properties) |
java.lang.Long |
getCombinedInputMaxSize() |
java.lang.String |
getLocalModeScheme() |
java.lang.String |
getTemporaryDirectory() |
static HfsProps |
hfsProps()
Creates a new HfsProps instance.
|
boolean |
isUseCombinedInput() |
boolean |
isUseCombinedInputSafeMode() |
HfsProps |
setCombinedInputMaxSize(long combinedInputMaxSize)
Method setCombinedInputMaxSize sets the maximum input split size to be used.
|
static void |
setCombinedInputMaxSize(java.util.Map<java.lang.Object,java.lang.Object> properties,
java.lang.Long size)
Method setCombinedInputMaxSize sets the maximum input split size to be used.
|
static void |
setLocalModeScheme(java.util.Map<java.lang.Object,java.lang.Object> properties,
java.lang.String scheme)
Method setLocalModeScheme provides a means to change the scheme value used to detect when a
MapReduce job should be run in Hadoop local mode.
|
HfsProps |
setLocalModeScheme(java.lang.String localModeScheme)
Method setLocalModeScheme provides a means to change the scheme value used to detect when a
MapReduce job should be run in Hadoop local mode.
|
static void |
setTemporaryDirectory(java.util.Map<java.lang.Object,java.lang.Object> properties,
java.lang.String temporaryDirectory)
Method setTemporaryDirectory sets the temporary directory on the given properties object.
|
HfsProps |
setTemporaryDirectory(java.lang.String temporaryDirectory)
Method setTemporaryDirectory sets the temporary directory for use on the underlying filesystem.
|
HfsProps |
setUseCombinedInput(boolean useCombinedInput)
Method setUseCombinedInput provides a means to indicate whether to leverage
CombineFileInputFormat for the input format. |
static void |
setUseCombinedInput(java.util.Map<java.lang.Object,java.lang.Object> properties,
java.lang.Boolean combine)
Method setUseCombinedInput provides a means to indicate whether to leverage
CombineFileInputFormat for the input format. |
HfsProps |
setUseCombinedInputSafeMode(boolean combinedInputSafeMode)
Method setUseCombinedInputSafeMode toggles safe mode when using
CombineFileInputFormat . |
static void |
setUseCombinedInputSafeMode(java.util.Map<java.lang.Object,java.lang.Object> properties,
java.lang.Boolean safeMode)
Method setUseCombinedInputSafeMode toggles safe mode when using
CombineFileInputFormat . |
buildProperties, buildProperties, buildProperties, buildProperties, setProperties, setProperties
public static final java.lang.String TEMPORARY_DIRECTORY
public static final java.lang.String LOCAL_MODE_SCHEME
public static final java.lang.String COMBINE_INPUT_FILES
public static final java.lang.String COMBINE_INPUT_FILES_SAFE_MODE
public static final java.lang.String COMBINE_INPUT_FILES_SIZE_MAX
protected java.lang.String temporaryDirectory
protected java.lang.String localModeScheme
protected java.lang.Boolean useCombinedInput
protected java.lang.Long combinedInputMaxSize
protected java.lang.Boolean combinedInputSafeMode
public HfsProps()
public static void setTemporaryDirectory(java.util.Map<java.lang.Object,java.lang.Object> properties, java.lang.String temporaryDirectory)
properties
- of type MaptemporaryDirectory
- of type Stringpublic static void setLocalModeScheme(java.util.Map<java.lang.Object,java.lang.Object> properties, java.lang.String scheme)
"file"
, set to
"none"
to disable entirely.properties
- of type Mapscheme
- a Stringpublic static void setUseCombinedInput(java.util.Map<java.lang.Object,java.lang.Object> properties, java.lang.Boolean combine)
CombineFileInputFormat
for the input format. By default it is false.
Use setCombinedInputMaxSize(long)
to set the max split/combined input size. Other specific
properties must be specified directly if needed. Specifically "mapred.min.split.size.per.node" and
"mapred.min.split.size.per.rack", which are 0 by default.properties
- of type Mapcombine
- a booleanpublic static void setUseCombinedInputSafeMode(java.util.Map<java.lang.Object,java.lang.Object> properties, java.lang.Boolean safeMode)
CombineFileInputFormat
. Safe mode will throw an exception if the underlying
InputFormat is not of type FileInputFormat
. If safeMode is off a warning will
be logged instead. safeMode is on by default.
Setting this property when not setting setUseCombinedInput(boolean)
to true has no effect.properties
- of type MapsafeMode
- a booleanpublic static void setCombinedInputMaxSize(java.util.Map<java.lang.Object,java.lang.Object> properties, java.lang.Long size)
properties
- of type Mapsize
- of type longpublic static HfsProps hfsProps()
public java.lang.String getTemporaryDirectory()
public HfsProps setTemporaryDirectory(java.lang.String temporaryDirectory)
temporaryDirectory
- of type Stringpublic java.lang.String getLocalModeScheme()
public HfsProps setLocalModeScheme(java.lang.String localModeScheme)
"file"
, set to
"none"
to disable entirely.localModeScheme
- of type Stringpublic boolean isUseCombinedInput()
public HfsProps setUseCombinedInput(boolean useCombinedInput)
CombineFileInputFormat
for the input format. By default it is false.useCombinedInput
- booleanpublic java.lang.Long getCombinedInputMaxSize()
public HfsProps setCombinedInputMaxSize(long combinedInputMaxSize)
setUseCombinedInput(boolean)
is true
.combinedInputMaxSize
- of type longpublic boolean isUseCombinedInputSafeMode()
public HfsProps setUseCombinedInputSafeMode(boolean combinedInputSafeMode)
CombineFileInputFormat
. Safe mode will throw an exception if the underlying
InputFormat is not of type FileInputFormat
. If safeMode is off a warning will
be logged instead. safeMode is on by default.
Setting this property when not setting setUseCombinedInput(boolean)
to true has no effect.combinedInputSafeMode
- booleanprotected void addPropertiesTo(java.util.Properties properties)
addPropertiesTo
in class Props
Copyright © 2007-2015 Xplenty, Inc. All Rights Reserved.