cascading.pipe.assembly
Class MaxBy

java.lang.Object
  extended by cascading.pipe.Pipe
      extended by cascading.pipe.SubAssembly
          extended by cascading.pipe.assembly.AggregateBy
              extended by cascading.pipe.assembly.MaxBy
All Implemented Interfaces:
FlowElement, Traceable, Serializable

public class MaxBy
extends AggregateBy

Class MaxBy is used to find the maximum value in a grouping.

Typically finding the max value of a field in a tuple stream relies on a GroupBy and a MaxValue Aggregator operation.

This SubAssembly also uses the MaxBy.MaxPartials AggregateBy.Functor to track the maximum value before the GroupBy operator to reduce IO over the network.

This strategy is similar to using combiners, except no sorting or serialization is invoked and results in a much simpler mechanism.

The threshold value tells the underlying MaxPartials functions how many unique key sums to accumulate in the LRU cache, before emitting the least recently used entry. This accumulation happens map-side, and thus is bounded by the size of your map task JVM and the typical size of each group key.

By default, either the value of AggregateByProps.AGGREGATE_BY_CAPACITY System property or AggregateByProps.AGGREGATE_BY_DEFAULT_CAPACITY will be used.

See Also:
AggregateBy, Serialized Form

Nested Class Summary
static class MaxBy.MaxPartials
           
 
Nested classes/interfaces inherited from class cascading.pipe.assembly.AggregateBy
AggregateBy.Cache, AggregateBy.CompositeFunction, AggregateBy.Flush, AggregateBy.Functor
 
Field Summary
static int DEFAULT_THRESHOLD
          Deprecated. 
 
Fields inherited from class cascading.pipe.assembly.AggregateBy
AGGREGATE_BY_THRESHOLD, USE_DEFAULT_THRESHOLD
 
Fields inherited from class cascading.pipe.Pipe
configDef, parent, stepConfigDef
 
Constructor Summary
MaxBy(Fields valueField, Fields maxField)
          Constructor MaxBy creates a new MaxBy instance.
MaxBy(Pipe[] pipes, Fields groupingFields, Fields valueField, Fields maxField)
          Constructor MaxBy creates a new MaxBy instance.
MaxBy(Pipe[] pipes, Fields groupingFields, Fields valueField, Fields maxField, int threshold)
          Constructor MaxBy creates a new MaxBy instance.
MaxBy(Pipe pipe, Fields groupingFields, Fields valueField, Fields maxField)
          Constructor MaxBy creates a new MaxBy instance.
MaxBy(Pipe pipe, Fields groupingFields, Fields valueField, Fields maxField, int threshold)
          Constructor MaxBy creates a new MaxBy instance.
MaxBy(String name, Pipe[] pipes, Fields groupingFields, Fields valueField, Fields maxField)
          Constructor MaxBy creates a new MaxBy instance.
MaxBy(String name, Pipe[] pipes, Fields groupingFields, Fields valueField, Fields maxField, int threshold)
          Constructor MaxBy creates a new MaxBy instance.
MaxBy(String name, Pipe pipe, Fields groupingFields, Fields valueField, Fields maxField)
          Constructor MaxBy creates a new MaxBy instance.
MaxBy(String name, Pipe pipe, Fields groupingFields, Fields valueField, Fields maxField, int threshold)
          Constructor MaxBy creates a new MaxBy instance.
 
Method Summary
 
Methods inherited from class cascading.pipe.assembly.AggregateBy
getAggregators, getArgumentFields, getCapacity, getFieldDeclarations, getFunctors, getGroupBy, getGroupingFields, getThreshold, initialize, initialize, verify
 
Methods inherited from class cascading.pipe.SubAssembly
getName, getPrevious, getTailNames, getTails, setPrevious, setTails, unwind
 
Methods inherited from class cascading.pipe.Pipe
equals, getConfigDef, getHeads, getParent, getStepConfigDef, getTrace, hasConfigDef, hashCode, hasStepConfigDef, id, isEquivalentTo, named, names, outgoingScopeFor, pipes, print, printInternal, resolveIncomingOperationArgumentFields, resolveIncomingOperationPassThroughFields, setParent, toString
 
Methods inherited from class java.lang.Object
clone, finalize, getClass, notify, notifyAll, wait, wait, wait
 

Field Detail

DEFAULT_THRESHOLD

@Deprecated
public static final int DEFAULT_THRESHOLD
Deprecated. 
DEFAULT_THRESHOLD

See Also:
Constant Field Values
Constructor Detail

MaxBy

@ConstructorProperties(value={"valueField","maxField"})
public MaxBy(Fields valueField,
                                        Fields maxField)
Constructor MaxBy creates a new MaxBy instance. Use this constructor when used with a AggregateBy instance.

Parameters:
valueField - of type Fields
maxField - of type Fields

MaxBy

@ConstructorProperties(value={"pipe","groupingFields","valueField","maxField"})
public MaxBy(Pipe pipe,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields maxField)
Constructor MaxBy creates a new MaxBy instance.

Parameters:
pipe - of type Pipe
groupingFields - of type Fields
valueField - of type Fields
maxField - of type Fields

MaxBy

@ConstructorProperties(value={"pipe","groupingFields","valueField","maxField","threshold"})
public MaxBy(Pipe pipe,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields maxField,
                                        int threshold)
Constructor MaxBy creates a new MaxBy instance.

Parameters:
pipe - of type Pipe
groupingFields - of type Fields
valueField - of type Fields
maxField - of type Fields
threshold - of type int

MaxBy

@ConstructorProperties(value={"name","pipe","groupingFields","valueField","maxField"})
public MaxBy(String name,
                                        Pipe pipe,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields maxField)
Constructor MaxBy creates a new MaxBy instance.

Parameters:
name - of type String
pipe - of type Pipe
groupingFields - of type Fields
valueField - of type Fields
maxField - of type Fields

MaxBy

@ConstructorProperties(value={"name","pipe","groupingFields","valueField","maxField","threshold"})
public MaxBy(String name,
                                        Pipe pipe,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields maxField,
                                        int threshold)
Constructor MaxBy creates a new MaxBy instance.

Parameters:
name - of type String
pipe - of type Pipe
groupingFields - of type Fields
valueField - of type Fields
maxField - of type Fields
threshold - of type int

MaxBy

@ConstructorProperties(value={"pipes","groupingFields","valueField","maxField"})
public MaxBy(Pipe[] pipes,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields maxField)
Constructor MaxBy creates a new MaxBy instance.

Parameters:
pipes - of type Pipe[]
groupingFields - of type Fields
valueField - of type Fields
maxField - of type Fields

MaxBy

@ConstructorProperties(value={"pipes","groupingFields","valueField","maxField","threshold"})
public MaxBy(Pipe[] pipes,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields maxField,
                                        int threshold)
Constructor MaxBy creates a new MaxBy instance.

Parameters:
pipes - of type Pipe[]
groupingFields - of type Fields
valueField - of type Fields
maxField - of type Fields
threshold - of type int

MaxBy

@ConstructorProperties(value={"name","pipes","groupingFields","valueField","maxField"})
public MaxBy(String name,
                                        Pipe[] pipes,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields maxField)
Constructor MaxBy creates a new MaxBy instance.

Parameters:
name - of type String
pipes - of type Pipe[]
groupingFields - of type Fields
valueField - of type Fields
maxField - of type Fields

MaxBy

@ConstructorProperties(value={"name","pipes","groupingFields","valueField","maxField","threshold"})
public MaxBy(String name,
                                        Pipe[] pipes,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields maxField,
                                        int threshold)
Constructor MaxBy creates a new MaxBy instance.

Parameters:
name - of type String
pipes - of type Pipe[]
groupingFields - of type Fields
valueField - of type Fields
maxField - of type Fields
threshold - of type int


Copyright © 2007-2015 Concurrent, Inc. All Rights Reserved.