|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object cascading.operation.BaseOperation<LinkedHashMap<Tuple,Object>> cascading.pipe.assembly.Unique.FilterPartialDuplicates
public static class Unique.FilterPartialDuplicates
Class FilterPartialDuplicates is a Filter
that is used to remove observed duplicates from the tuple stream.
First
Aggregator
in order to improve de-duping performance by removing as many values
as possible before the intermediate GroupBy
operator.
The threshold
value is used to maintain a LRU of a constant size. If more than threshold unique values
are seen, the oldest cached values will be removed from the cache.
Unique
,
Serialized FormField Summary |
---|
Fields inherited from class cascading.operation.BaseOperation |
---|
fieldDeclaration, numArgs, trace |
Fields inherited from interface cascading.operation.Operation |
---|
ANY |
Constructor Summary | |
---|---|
Unique.FilterPartialDuplicates()
Constructor FilterPartialDuplicates creates a new FilterPartialDuplicates instance. |
|
Unique.FilterPartialDuplicates(int threshold)
Constructor FilterPartialDuplicates creates a new FilterPartialDuplicates instance. |
|
Unique.FilterPartialDuplicates(Unique.Include include,
int threshold)
Constructor FilterPartialDuplicates creates a new FilterPartialDuplicates instance. |
Method Summary | |
---|---|
void |
cleanup(FlowProcess flowProcess,
OperationCall<LinkedHashMap<Tuple,Object>> operationCall)
Method cleanup does nothing, and may safely be overridden. |
boolean |
equals(Object object)
|
int |
hashCode()
|
boolean |
isRemove(FlowProcess flowProcess,
FilterCall<LinkedHashMap<Tuple,Object>> filterCall)
Method isRemove returns true if input should be removed from the tuple stream. |
void |
prepare(FlowProcess flowProcess,
OperationCall<LinkedHashMap<Tuple,Object>> operationCall)
Method prepare does nothing, and may safely be overridden. |
Methods inherited from class cascading.operation.BaseOperation |
---|
flush, getFieldDeclaration, getNumArgs, getTrace, isSafe, printOperationInternal, toString, toStringInternal |
Methods inherited from class java.lang.Object |
---|
clone, finalize, getClass, notify, notifyAll, wait, wait, wait |
Methods inherited from interface cascading.operation.Operation |
---|
flush, getFieldDeclaration, getNumArgs, isSafe |
Constructor Detail |
---|
public Unique.FilterPartialDuplicates()
@ConstructorProperties(value="threshold") public Unique.FilterPartialDuplicates(int threshold)
threshold
- of type int@ConstructorProperties(value={"include","threshold"}) public Unique.FilterPartialDuplicates(Unique.Include include, int threshold)
threshold
- of type intMethod Detail |
---|
public void prepare(FlowProcess flowProcess, OperationCall<LinkedHashMap<Tuple,Object>> operationCall)
BaseOperation
prepare
in interface Operation<LinkedHashMap<Tuple,Object>>
prepare
in class BaseOperation<LinkedHashMap<Tuple,Object>>
public boolean isRemove(FlowProcess flowProcess, FilterCall<LinkedHashMap<Tuple,Object>> filterCall)
Filter
isRemove
in interface Filter<LinkedHashMap<Tuple,Object>>
flowProcess
- of type FlowProcessfilterCall
- of type FilterCall
public void cleanup(FlowProcess flowProcess, OperationCall<LinkedHashMap<Tuple,Object>> operationCall)
BaseOperation
cleanup
in interface Operation<LinkedHashMap<Tuple,Object>>
cleanup
in class BaseOperation<LinkedHashMap<Tuple,Object>>
public boolean equals(Object object)
equals
in class BaseOperation<LinkedHashMap<Tuple,Object>>
public int hashCode()
hashCode
in class BaseOperation<LinkedHashMap<Tuple,Object>>
|
|||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | ||||||||
SUMMARY: NESTED | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |