cascading.tuple.hadoop.collect
Class HadoopSpillableTupleList
java.lang.Object
cascading.tuple.collect.SpillableTupleList
cascading.tuple.hadoop.collect.HadoopSpillableTupleList
- All Implemented Interfaces:
- Spillable, Iterable<Tuple>, Collection<Tuple>
public class HadoopSpillableTupleList
- extends SpillableTupleList
SpillableTupleList is a simple Iterable
object that can store an unlimited number of Tuple
instances by spilling
excess to a temporary disk file.
Spills will automatically be compressed using the defaultCodecs
values. To disable compression or
change the codecs, see SpillableProps.SPILL_COMPRESS
and SpillableProps.SPILL_CODECS
.
It is recommended to add Lzo if available.
"org.apache.hadoop.io.compress.LzoCodec,org.apache.hadoop.io.compress.GzipCodec,org.apache.hadoop.io.compress.DefaultCodec"
Methods inherited from class cascading.tuple.collect.SpillableTupleList |
add, addAll, clear, contains, containsAll, getCodecClass, getGrouping, getThreshold, isEmpty, iterator, remove, removeAll, retainAll, setGrouping, setSpillListener, setSpillStrategy, size, spillCount, toArray, toArray |
Methods inherited from class java.lang.Object |
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
defaultCodecs
public static final String defaultCodecs
- See Also:
- Constant Field Values
HadoopSpillableTupleList
public HadoopSpillableTupleList(int threshold,
CompressionCodec codec,
JobConf jobConf)
- Constructor SpillableTupleList creates a new SpillableTupleList instance using the given threshold value, and
the first available compression codec, if any.
- Parameters:
threshold
- of type longcodec
- of type CompressionCodec
HadoopSpillableTupleList
public HadoopSpillableTupleList(int threshold,
TupleSerialization tupleSerialization,
CompressionCodec codec)
getCodec
public static CompressionCodec getCodec(FlowProcess flowProcess,
String defaultCodecs)
createTupleOutputStream
protected TupleOutputStream createTupleOutputStream(File file)
- Specified by:
createTupleOutputStream
in class SpillableTupleList
createTupleInputStream
protected TupleInputStream createTupleInputStream(File file)
- Specified by:
createTupleInputStream
in class SpillableTupleList
Copyright © 2007-2014 Concurrent, Inc. All Rights Reserved.