cascading.pipe.assembly
Class MinBy

java.lang.Object
  cascading.pipe.Pipe
      cascading.pipe.SubAssembly
          cascading.pipe.assembly.AggregateBy
              cascading.pipe.assembly.MinBy

All Implemented Interfaces:

FlowElement, Traceable, Serializable

public class MinBy
extends AggregateBy

Class MinBy is used to find the minimum value in a grouping.

Typically finding the min value of a field in a tuple stream relies on a GroupBy and a MinValue Aggregator operation.

This SubAssembly also uses the MinBy.MinPartials AggregateBy.Functor to track the minimum value before the GroupBy operator to reduce IO over the network.

This strategy is similar to using combiners, except no sorting or serialization is invoked and results in a much simpler mechanism.

The threshold value tells the underlying MinPartials functions how many unique key sums to accumulate in the LRU cache, before emitting the least recently used entry. This accumulation happens map-side, and thus is bounded by the size of your map task JVM and the typical size of each group key.

By default, either the value of AggregateByProps.AGGREGATE_BY_CAPACITY System property or AggregateByProps.AGGREGATE_BY_DEFAULT_CAPACITY will be used.

See Also:

AggregateBy, Serialized Form

Nested Class Summary

static class

MinBy.MinPartials

Nested classes/interfaces inherited from class cascading.pipe.assembly.AggregateBy

AggregateBy.Cache, AggregateBy.CompositeFunction, AggregateBy.Flush, AggregateBy.Functor

Field Summary

static int

DEFAULT_THRESHOLD Deprecated.

Fields inherited from class cascading.pipe.assembly.AggregateBy

AGGREGATE_BY_THRESHOLD, USE_DEFAULT_THRESHOLD

Fields inherited from class cascading.pipe.Pipe

configDef, parent, stepConfigDef

Constructor Summary

MinBy(Fields valueField, Fields minField)
Constructor MinBy creates a new MinBy instance.

MinBy(Pipe[] pipes, Fields groupingFields, Fields valueField, Fields minField)
Constructor MinBy creates a new MinBy instance.

MinBy(Pipe[] pipes, Fields groupingFields, Fields valueField, Fields minField, int threshold)
Constructor MinBy creates a new MinBy instance.

MinBy(Pipe pipe, Fields groupingFields, Fields valueField, Fields minField)
Constructor MinBy creates a new MinBy instance.

MinBy(Pipe pipe, Fields groupingFields, Fields valueField, Fields minField, int threshold)
Constructor MinBy creates a new MinBy instance.

MinBy(String name, Pipe[] pipes, Fields groupingFields, Fields valueField, Fields minField)
Constructor MinBy creates a new MinBy instance.

MinBy(String name, Pipe[] pipes, Fields groupingFields, Fields valueField, Fields minField, int threshold)
Constructor MinBy creates a new MinBy instance.

MinBy(String name, Pipe pipe, Fields groupingFields, Fields valueField, Fields minField)
Constructor MinBy creates a new MinBy instance.

MinBy(String name, Pipe pipe, Fields groupingFields, Fields valueField, Fields minField, int threshold)
Constructor MinBy creates a new MinBy instance.

Method Summary

Methods inherited from class cascading.pipe.assembly.AggregateBy

getAggregators, getArgumentFields, getCapacity, getFieldDeclarations, getFunctors, getGroupBy, getGroupingFields, getThreshold, initialize, initialize, verify

Methods inherited from class cascading.pipe.SubAssembly

getName, getPrevious, getTailNames, getTails, setPrevious, setTails, unwind

Methods inherited from class cascading.pipe.Pipe

equals, getConfigDef, getHeads, getParent, getStepConfigDef, getTrace, hasConfigDef, hashCode, hasStepConfigDef, id, isEquivalentTo, named, names, outgoingScopeFor, pipes, print, printInternal, resolveIncomingOperationArgumentFields, resolveIncomingOperationPassThroughFields, setParent, toString

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

DEFAULT_THRESHOLD

@Deprecated
public static final int DEFAULT_THRESHOLD

Deprecated.

DEFAULT_THRESHOLD

See Also:

Constant Field Values

Constructor Detail

MinBy

@ConstructorProperties(value={"valueField","minField"})
public MinBy(Fields valueField,
                                        Fields minField)

Constructor MinBy creates a new MinBy instance. Use this constructor when used with a AggregateBy instance.

Parameters:

valueField - of type Fields

minField - of type Fields

MinBy

@ConstructorProperties(value={"pipe","groupingFields","valueField","minField"})
public MinBy(Pipe pipe,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields minField)

Constructor MinBy creates a new MinBy instance.

Parameters:

pipe - of type Pipe

groupingFields - of type Fields

valueField - of type Fields

minField - of type Fields

MinBy

@ConstructorProperties(value={"pipe","groupingFields","valueField","minField","threshold"})
public MinBy(Pipe pipe,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields minField,
                                        int threshold)

Constructor MinBy creates a new MinBy instance.

Parameters:

pipe - of type Pipe

groupingFields - of type Fields

valueField - of type Fields

minField - of type Fields

threshold - of type int

MinBy

@ConstructorProperties(value={"name","pipe","groupingFields","valueField","minField"})
public MinBy(String name,
                                        Pipe pipe,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields minField)

Constructor MinBy creates a new MinBy instance.

Parameters:

name - of type String

pipe - of type Pipe

groupingFields - of type Fields

valueField - of type Fields

minField - of type Fields

MinBy

@ConstructorProperties(value={"name","pipe","groupingFields","valueField","minField","threshold"})
public MinBy(String name,
                                        Pipe pipe,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields minField,
                                        int threshold)

Constructor MinBy creates a new MinBy instance.

Parameters:

name - of type String

pipe - of type Pipe

groupingFields - of type Fields

valueField - of type Fields

minField - of type Fields

threshold - of type int

MinBy

@ConstructorProperties(value={"pipes","groupingFields","valueField","minField"})
public MinBy(Pipe[] pipes,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields minField)

Constructor MinBy creates a new MinBy instance.

Parameters:

pipes - of type Pipe[]

groupingFields - of type Fields

valueField - of type Fields

minField - of type Fields

MinBy

@ConstructorProperties(value={"pipes","groupingFields","valueField","minField","threshold"})
public MinBy(Pipe[] pipes,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields minField,
                                        int threshold)

Constructor MinBy creates a new MinBy instance.

Parameters:

pipes - of type Pipe[]

groupingFields - of type Fields

valueField - of type Fields

minField - of type Fields

threshold - of type int

MinBy

@ConstructorProperties(value={"name","pipes","groupingFields","valueField","minField"})
public MinBy(String name,
                                        Pipe[] pipes,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields minField)

Constructor MinBy creates a new MinBy instance.

Parameters:

name - of type String

pipes - of type Pipe[]

groupingFields - of type Fields

valueField - of type Fields

minField - of type Fields

MinBy

@ConstructorProperties(value={"name","pipes","groupingFields","valueField","minField","threshold"})
public MinBy(String name,
                                        Pipe[] pipes,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields minField,
                                        int threshold)

Constructor MinBy creates a new MinBy instance.

Parameters:

name - of type String

pipes - of type Pipe[]

groupingFields - of type Fields

valueField - of type Fields

minField - of type Fields

threshold - of type int