cascading.pipe.assembly
Class SumBy

java.lang.Object
  cascading.pipe.Pipe
      cascading.pipe.SubAssembly
          cascading.pipe.assembly.AggregateBy
              cascading.pipe.assembly.SumBy

All Implemented Interfaces:

FlowElement, Traceable, Serializable

public class SumBy
extends AggregateBy

Class SumBy is used to sum values associated with duplicate keys in a tuple stream.

Typically finding the sum of field in a tuple stream relies on a GroupBy and a Sum Aggregator operation.

If all the values to be summed are all null, the result value is a function of how null is coerced by the given sumType. If a primitive type, 0 will be returned. Otherwise null.

This SubAssembly also uses the SumBy.SumPartials AggregateBy.Functor to sum field values before the GroupBy operator to reduce IO over the network.

This strategy is similar to using combiners, except no sorting or serialization is invoked and results in a much simpler mechanism.

The threshold value tells the underlying SumPartials functions how many unique key sums to accumulate in the LRU cache, before emitting the least recently used entry.

By default, either the value of AggregateBy.AGGREGATE_BY_THRESHOLD System property or AggregateBy.DEFAULT_THRESHOLD will be used.

See Also:

AggregateBy, Serialized Form

Nested Class Summary

static class

SumBy.SumPartials Class SumPartials is a AggregateBy.Functor that is used to sum observed duplicates from the tuple stream.

Nested classes/interfaces inherited from class cascading.pipe.assembly.AggregateBy

AggregateBy.Cache, AggregateBy.CompositeFunction, AggregateBy.Flush, AggregateBy.Functor

Field Summary

static int

DEFAULT_THRESHOLD Deprecated.

Fields inherited from class cascading.pipe.assembly.AggregateBy

AGGREGATE_BY_THRESHOLD, USE_DEFAULT_THRESHOLD

Fields inherited from class cascading.pipe.Pipe

configDef, parent, stepConfigDef

Constructor Summary

SumBy(Fields valueField, Fields sumField)
Constructor SumBy creates a new SumBy instance.

SumBy(Fields valueField, Fields sumField, Class sumType)
Constructor SumBy creates a new SumBy instance.

SumBy(Pipe[] pipes, Fields groupingFields, Fields valueField, Fields sumField)
Constructor SumBy creates a new SumBy instance.

SumBy(Pipe[] pipes, Fields groupingFields, Fields valueField, Fields sumField, Class sumType)
Constructor SumBy creates a new SumBy instance.

SumBy(Pipe[] pipes, Fields groupingFields, Fields valueField, Fields sumField, Class sumType, int threshold)
Constructor SumBy creates a new SumBy instance.

SumBy(Pipe[] pipes, Fields groupingFields, Fields valueField, Fields sumField, int threshold)
Constructor SumBy creates a new SumBy instance.

SumBy(Pipe pipe, Fields groupingFields, Fields valueField, Fields sumField)
Constructor SumBy creates a new SumBy instance.

SumBy(Pipe pipe, Fields groupingFields, Fields valueField, Fields sumField, Class sumType)
Constructor SumBy creates a new SumBy instance.

SumBy(Pipe pipe, Fields groupingFields, Fields valueField, Fields sumField, Class sumType, int threshold)
Constructor SumBy creates a new SumBy instance.

SumBy(Pipe pipe, Fields groupingFields, Fields valueField, Fields sumField, int threshold)
Constructor SumBy creates a new SumBy instance.

SumBy(String name, Pipe[] pipes, Fields groupingFields, Fields valueField, Fields sumField)
Constructor SumBy creates a new SumBy instance.

SumBy(String name, Pipe[] pipes, Fields groupingFields, Fields valueField, Fields sumField, Class sumType)
Constructor SumBy creates a new SumBy instance.

SumBy(String name, Pipe[] pipes, Fields groupingFields, Fields valueField, Fields sumField, Class sumType, int threshold)
Constructor SumBy creates a new SumBy instance.

SumBy(String name, Pipe[] pipes, Fields groupingFields, Fields valueField, Fields sumField, int threshold)
Constructor SumBy creates a new SumBy instance.

SumBy(String name, Pipe pipe, Fields groupingFields, Fields valueField, Fields sumField)
Constructor SumBy creates a new SumBy instance.

SumBy(String name, Pipe pipe, Fields groupingFields, Fields valueField, Fields sumField, Class sumType)
Constructor SumBy creates a new SumBy instance.

SumBy(String name, Pipe pipe, Fields groupingFields, Fields valueField, Fields sumField, Class sumType, int threshold)
Constructor SumBy creates a new SumBy instance.

SumBy(String name, Pipe pipe, Fields groupingFields, Fields valueField, Fields sumField, int threshold)
Constructor SumBy creates a new SumBy instance.

Method Summary

Methods inherited from class cascading.pipe.assembly.AggregateBy

getAggregators, getArgumentFields, getFieldDeclarations, getFunctors, getGroupBy, getGroupingFields, getThreshold, initialize, initialize, verify

Methods inherited from class cascading.pipe.SubAssembly

getName, getPrevious, getTailNames, getTails, setPrevious, setTails, unwind

Methods inherited from class cascading.pipe.Pipe

equals, getConfigDef, getHeads, getParent, getStepConfigDef, getTrace, hasConfigDef, hashCode, hasStepConfigDef, id, isEquivalentTo, named, names, outgoingScopeFor, pipes, print, printInternal, resolveIncomingOperationArgumentFields, resolveIncomingOperationPassThroughFields, setParent, toString

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

DEFAULT_THRESHOLD

@Deprecated
public static final int DEFAULT_THRESHOLD

Deprecated.

DEFAULT_THRESHOLD

See Also:

Constant Field Values

Constructor Detail

SumBy

@ConstructorProperties(value={"valueField","sumField"})
public SumBy(Fields valueField,
                                        Fields sumField)

Constructor SumBy creates a new SumBy instance. Use this constructor when used with a AggregateBy instance.

Parameters:

valueField - of type Fields

sumField - of type Fields

SumBy

@ConstructorProperties(value={"pipe","groupingFields","valueField","sumField"})
public SumBy(Pipe pipe,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields sumField)

Constructor SumBy creates a new SumBy instance.

Parameters:

pipe - of type Pipe

groupingFields - of type Fields

valueField - of type Fields

sumField - of type Fields

SumBy

@ConstructorProperties(value={"pipe","groupingFields","valueField","sumField","threshold"})
public SumBy(Pipe pipe,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields sumField,
                                        int threshold)

Constructor SumBy creates a new SumBy instance.

Parameters:

pipe - of type Pipe

groupingFields - of type Fields

valueField - of type Fields

sumField - of type Fields

threshold - of type int

SumBy

@ConstructorProperties(value={"name","pipe","groupingFields","valueField","sumField"})
public SumBy(String name,
                                        Pipe pipe,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields sumField)

Constructor SumBy creates a new SumBy instance.

Parameters:

name - of type String

pipe - of type Pipe

groupingFields - of type Fields

valueField - of type Fields

sumField - of type Fields

SumBy

@ConstructorProperties(value={"name","pipe","groupingFields","valueField","sumField","threshold"})
public SumBy(String name,
                                        Pipe pipe,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields sumField,
                                        int threshold)

Constructor SumBy creates a new SumBy instance.

Parameters:

name - of type String

pipe - of type Pipe

groupingFields - of type Fields

valueField - of type Fields

sumField - of type Fields

threshold - of type int

SumBy

@ConstructorProperties(value={"pipes","groupingFields","valueField","sumField"})
public SumBy(Pipe[] pipes,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields sumField)

Constructor SumBy creates a new SumBy instance.

Parameters:

pipes - of type Pipe[]

groupingFields - of type Fields

valueField - of type Fields

sumField - of type Fields

SumBy

@ConstructorProperties(value={"pipes","groupingFields","valueField","sumField","threshold"})
public SumBy(Pipe[] pipes,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields sumField,
                                        int threshold)

Constructor SumBy creates a new SumBy instance.

Parameters:

pipes - of type Pipe[]

groupingFields - of type Fields

valueField - of type Fields

sumField - of type Fields

threshold - of type int

SumBy

@ConstructorProperties(value={"name","pipes","groupingFields","valueField","sumField"})
public SumBy(String name,
                                        Pipe[] pipes,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields sumField)

Constructor SumBy creates a new SumBy instance.

Parameters:

name - of type String

pipes - of type Pipe[]

groupingFields - of type Fields

valueField - of type Fields

sumField - of type Fields

SumBy

@ConstructorProperties(value={"name","pipes","groupingFields","valueField","sumField","threshold"})
public SumBy(String name,
                                        Pipe[] pipes,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields sumField,
                                        int threshold)

Constructor SumBy creates a new SumBy instance.

Parameters:

name - of type String

pipes - of type Pipe[]

groupingFields - of type Fields

valueField - of type Fields

sumField - of type Fields

threshold - of type int

SumBy

@ConstructorProperties(value={"valueField","sumField","sumType"})
public SumBy(Fields valueField,
                                        Fields sumField,
                                        Class sumType)

Constructor SumBy creates a new SumBy instance. Use this constructor when used with a AggregateBy instance.

Parameters:

valueField - of type Fields

sumField - of type Fields

sumType - of type Class

SumBy

@ConstructorProperties(value={"pipe","groupingFields","valueField","sumField","sumType"})
public SumBy(Pipe pipe,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields sumField,
                                        Class sumType)

Constructor SumBy creates a new SumBy instance.

Parameters:

pipe - of type Pipe

groupingFields - of type Fields

valueField - of type Fields

sumField - of type Fields

sumType - of type Class

SumBy

@ConstructorProperties(value={"pipe","groupingFields","valueField","sumField","sumType","threshold"})
public SumBy(Pipe pipe,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields sumField,
                                        Class sumType,
                                        int threshold)

Constructor SumBy creates a new SumBy instance.

Parameters:

pipe - of type Pipe

groupingFields - of type Fields

valueField - of type Fields

sumField - of type Fields

sumType - of type Class

threshold - of type int

SumBy

@ConstructorProperties(value={"name","pipe","groupingFields","valueField","sumField","sumType"})
public SumBy(String name,
                                        Pipe pipe,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields sumField,
                                        Class sumType)

Constructor SumBy creates a new SumBy instance.

Parameters:

name - of type String

pipe - of type Pipe

groupingFields - of type Fields

valueField - of type Fields

sumField - of type Fields

sumType - of type Class

SumBy

@ConstructorProperties(value={"name","pipe","groupingFields","valueField","sumField","sumType","threshold"})
public SumBy(String name,
                                        Pipe pipe,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields sumField,
                                        Class sumType,
                                        int threshold)

Constructor SumBy creates a new SumBy instance.

Parameters:

name - of type String

pipe - of type Pipe

groupingFields - of type Fields

valueField - of type Fields

sumField - of type Fields

sumType - of type Class

threshold - of type int

SumBy

@ConstructorProperties(value={"pipes","groupingFields","valueField","sumField","sumType"})
public SumBy(Pipe[] pipes,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields sumField,
                                        Class sumType)

Constructor SumBy creates a new SumBy instance.

Parameters:

pipes - of type Pipe[]

groupingFields - of type Fields

valueField - of type Fields

sumField - of type Fields

sumType - of type Class

SumBy

@ConstructorProperties(value={"pipes","groupingFields","valueField","sumField","sumType","threshold"})
public SumBy(Pipe[] pipes,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields sumField,
                                        Class sumType,
                                        int threshold)

Constructor SumBy creates a new SumBy instance.

Parameters:

pipes - of type Pipe[]

groupingFields - of type Fields

valueField - of type Fields

sumField - of type Fields

sumType - of type Class

threshold - of type int

SumBy

@ConstructorProperties(value={"name","pipes","groupingFields","valueField","sumField","sumType"})
public SumBy(String name,
                                        Pipe[] pipes,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields sumField,
                                        Class sumType)

Constructor SumBy creates a new SumBy instance.

Parameters:

name - of type String

pipes - of type Pipe[]

groupingFields - of type Fields

valueField - of type Fields

sumField - of type Fields

sumType - of type Class

SumBy

@ConstructorProperties(value={"name","pipes","groupingFields","valueField","sumField","sumType","threshold"})
public SumBy(String name,
                                        Pipe[] pipes,
                                        Fields groupingFields,
                                        Fields valueField,
                                        Fields sumField,
                                        Class sumType,
                                        int threshold)

Constructor SumBy creates a new SumBy instance.

Parameters:

name - of type String

pipes - of type Pipe[]

groupingFields - of type Fields

valueField - of type Fields

sumField - of type Fields

sumType - of type Class

threshold - of type int