public static class Unique.FilterPartialDuplicates extends BaseOperation<CascadingCache<Tuple,Object>> implements Filter<CascadingCache<Tuple,Object>>
Filter
that is used to remove observed duplicates from the tuple stream.
Use this class typically in tandem with a First
Aggregator
in order to improve de-duping performance by removing as many values
as possible before the intermediate GroupBy
operator.
The capacity
value is used to maintain a LRU of a constant size. If more than capacity unique values
are seen, the oldest cached values will be removed from the cache.Unique
,
Serialized FormfieldDeclaration, numArgs, trace
Constructor and Description |
---|
Unique.FilterPartialDuplicates()
Constructor FilterPartialDuplicates creates a new FilterPartialDuplicates instance.
|
Unique.FilterPartialDuplicates(int capacity)
Constructor FilterPartialDuplicates creates a new FilterPartialDuplicates instance.
|
Unique.FilterPartialDuplicates(Unique.Include include,
int capacity)
Constructor FilterPartialDuplicates creates a new FilterPartialDuplicates instance.
|
Unique.FilterPartialDuplicates(Unique.Include include,
int capacity,
TupleHasher tupleHasher)
Constructor FilterPartialDuplicates creates a new FilterPartialDuplicates instance.
|
Modifier and Type | Method and Description |
---|---|
void |
cleanup(FlowProcess flowProcess,
OperationCall<CascadingCache<Tuple,Object>> operationCall)
Method cleanup does nothing, and may safely be overridden.
|
boolean |
equals(Object object) |
int |
hashCode() |
boolean |
isRemove(FlowProcess flowProcess,
FilterCall<CascadingCache<Tuple,Object>> filterCall)
Method isRemove returns true if input should be removed from the tuple stream.
|
void |
prepare(FlowProcess flowProcess,
OperationCall<CascadingCache<Tuple,Object>> operationCall)
Method prepare does nothing, and may safely be overridden.
|
flush, getFieldDeclaration, getNumArgs, getTrace, isSafe, printOperationInternal, toString, toStringInternal
clone, finalize, getClass, notify, notifyAll, wait, wait, wait
flush, getFieldDeclaration, getNumArgs, isSafe
public Unique.FilterPartialDuplicates()
@ConstructorProperties(value="capacity") public Unique.FilterPartialDuplicates(int capacity)
capacity
- of type int@ConstructorProperties(value={"include","capacity"}) public Unique.FilterPartialDuplicates(Unique.Include include, int capacity)
include
- of type Includecapacity
- of type int@ConstructorProperties(value={"include","capacity","tupleHasher"}) public Unique.FilterPartialDuplicates(Unique.Include include, int capacity, TupleHasher tupleHasher)
capacity
- of type intinclude
- of type IncludetupleHasher
- of type TupleHasherpublic void prepare(FlowProcess flowProcess, OperationCall<CascadingCache<Tuple,Object>> operationCall)
BaseOperation
prepare
in interface Operation<CascadingCache<Tuple,Object>>
prepare
in class BaseOperation<CascadingCache<Tuple,Object>>
public boolean isRemove(FlowProcess flowProcess, FilterCall<CascadingCache<Tuple,Object>> filterCall)
Filter
isRemove
in interface Filter<CascadingCache<Tuple,Object>>
flowProcess
- of type FlowProcessfilterCall
- of type FilterCallpublic void cleanup(FlowProcess flowProcess, OperationCall<CascadingCache<Tuple,Object>> operationCall)
BaseOperation
cleanup
in interface Operation<CascadingCache<Tuple,Object>>
cleanup
in class BaseOperation<CascadingCache<Tuple,Object>>
public boolean equals(Object object)
equals
in class BaseOperation<CascadingCache<Tuple,Object>>
public int hashCode()
hashCode
in class BaseOperation<CascadingCache<Tuple,Object>>
Copyright © 2007-2015 Concurrent, Inc. All Rights Reserved.