cascading.tuple.collect

Class SpillableProps

java.lang.Object

cascading.property.Props

cascading.tuple.collect.SpillableProps

public class SpillableProps
extends Props

Class SpillableProps is a fluent interface for building properties to be passed to a FlowConnector before creating new Flow instances.

See Also:

SpillableTupleList, SpillableTupleMap

Field Summary

Fields

Modifier and Type	Field and Description
static int	defaultListThreshold
static int	defaultMapInitialCapacity
static float	defaultMapLoadFactor
static int	defaultMapThreshold
static String	LIST_THRESHOLD Number of tuples to hold in memory before spilling them to disk.
static String	MAP_CAPACITY The initial hash map capacity.
static String	MAP_LOADFACTOR The initial hash map load factor.
static String	MAP_THRESHOLD The total number of tuple values (not keys) to attempt to keep in memory.
static String	SPILL_CODECS A comma delimited list of possible codecs to try.
static String	SPILL_COMPRESS Whether to enable compression of the spills or not, on by default.

Constructor Summary

Constructors

Constructor and Description
SpillableProps()

Method Summary

Methods

Modifier and Type	Method and Description
SpillableProps	addCodec(String codec) Method addCodec adds a codec class names to use.
SpillableProps	addCodecs(List<String> codecs) Method addCodecs adds a list of possible codec class names to use.
protected void	addPropertiesTo(Properties properties)
List<String>	getCodecs()
int	getListSpillThreshold()
int	getMapInitialCapacity()
float	getMapLoadFactor()
int	getMapSpillThreshold()
boolean	isCompressSpill()
SpillableProps	setCodecs(List<String> codecs) Method setCodecs sets list of possible codec class names to use.
SpillableProps	setCompressSpill(boolean compressSpill) Method setCompressSpill either enables or disables spill compression.
SpillableProps	setListSpillThreshold(int listSpillThreshold) Method setListSpillThreshold sets the number of tuples to hold in memory before spilling them to disk.
SpillableProps	setMapInitialCapacity(int mapInitialCapacity) Method setMapInitialCapacity sets the default capacity to be used by the backing Map implementation.
SpillableProps	setMapLoadFactor(float mapLoadFactor) Method setMapLoadFactor sets the default load factor to be used by the backing Map implementation.
SpillableProps	setMapSpillThreshold(int mapSpillThreshold) Method setMapSpillThreshold the total number of tuple values (not keys) to attempt to keep in memory.
static SpillableProps	spillableProps() Creates a new SpillableProps instance.

Methods inherited from class cascading.property.Props

buildProperties, buildProperties, buildProperties, buildProperties, setProperties, setProperties

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

SPILL_COMPRESS

public static final String SPILL_COMPRESS

Whether to enable compression of the spills or not, on by default.

See Also:

Boolean.parseBoolean(String), Constant Field Values

SPILL_CODECS

public static final String SPILL_CODECS

A comma delimited list of possible codecs to try. This is platform dependent.

See Also:

Constant Field Values

LIST_THRESHOLD

public static final String LIST_THRESHOLD

Number of tuples to hold in memory before spilling them to disk.

See Also:

Constant Field Values

MAP_THRESHOLD

public static final String MAP_THRESHOLD

The total number of tuple values (not keys) to attempt to keep in memory.

See Also:

Constant Field Values

MAP_CAPACITY

public static final String MAP_CAPACITY

The initial hash map capacity.

See Also:

HashMap, Constant Field Values

MAP_LOADFACTOR

public static final String MAP_LOADFACTOR

The initial hash map load factor.

See Also:

HashMap, Constant Field Values

defaultListThreshold

public static final int defaultListThreshold

See Also:

Constant Field Values

defaultMapThreshold

public static final int defaultMapThreshold

See Also:

Constant Field Values

defaultMapInitialCapacity

public static final int defaultMapInitialCapacity

See Also:

Constant Field Values

defaultMapLoadFactor

public static final float defaultMapLoadFactor

See Also:

Constant Field Values

Constructor Detail

SpillableProps

public SpillableProps()

Method Detail

spillableProps

public static SpillableProps spillableProps()

Creates a new SpillableProps instance.

Returns:

SpillableProps instance

isCompressSpill

public boolean isCompressSpill()

setCompressSpill

public SpillableProps setCompressSpill(boolean compressSpill)

Method setCompressSpill either enables or disables spill compression. Enabled by default.

Spill compression relies on properly configured and available codecs. See setCodecs(java.util.List).

Parameters:

compressSpill - type boolean

Returns:

this

getCodecs

public List<String> getCodecs()

setCodecs

public SpillableProps setCodecs(List<String> codecs)

Method setCodecs sets list of possible codec class names to use. They will be loaded in order, if available.

This is platform dependent.

Parameters:

codecs - type list

Returns:

this

addCodecs

public SpillableProps addCodecs(List<String> codecs)

Method addCodecs adds a list of possible codec class names to use. They will be loaded in order, if available.

This is platform dependent.

Parameters:

codecs - type list

addCodec

public SpillableProps addCodec(String codec)

Method addCodec adds a codec class names to use.

This is platform dependent.

Parameters:

codec - type String

getListSpillThreshold

public int getListSpillThreshold()

setListSpillThreshold

public SpillableProps setListSpillThreshold(int listSpillThreshold)

Method setListSpillThreshold sets the number of tuples to hold in memory before spilling them to disk.

Parameters:

listSpillThreshold - of type int

Returns:

this

getMapSpillThreshold

public int getMapSpillThreshold()

setMapSpillThreshold

public SpillableProps setMapSpillThreshold(int mapSpillThreshold)

Method setMapSpillThreshold the total number of tuple values (not keys) to attempt to keep in memory.

The default implementation cannot spill Map keys to disk.

Parameters:

mapSpillThreshold - of type int

Returns:

this

getMapInitialCapacity

public int getMapInitialCapacity()

setMapInitialCapacity

public SpillableProps setMapInitialCapacity(int mapInitialCapacity)

Method setMapInitialCapacity sets the default capacity to be used by the backing Map implementation.

Parameters:

mapInitialCapacity - type int

Returns:

this

getMapLoadFactor

public float getMapLoadFactor()

setMapLoadFactor

public SpillableProps setMapLoadFactor(float mapLoadFactor)

Method setMapLoadFactor sets the default load factor to be used by the backing Map implementation.

Parameters:

mapLoadFactor - type float

Returns:

this

addPropertiesTo

protected void addPropertiesTo(Properties properties)

Specified by:

addPropertiesTo in class Props