cascading.pipe.assembly
Class FirstBy

java.lang.Object
  cascading.pipe.Pipe
      cascading.pipe.SubAssembly
          cascading.pipe.assembly.AggregateBy
              cascading.pipe.assembly.FirstBy

All Implemented Interfaces:

FlowElement, Traceable, Serializable

public class FirstBy
extends AggregateBy

Class FirstBy is used to return the first encountered Tuple in a tuple stream grouping.

Typically finding the first Tuple in a tuple stream grouping relies on a GroupBy and a First Aggregator operation.

If the firstFields argument has custom Comparator instances, they will be used as the GroupBy sortFields.

This SubAssembly also uses the FirstBy.FirstPartials AggregateBy.Functor to collect field values before the GroupBy operator to reduce IO over the network.

This strategy is similar to using combiners, except no sorting or serialization is invoked and results in a much simpler mechanism.

The threshold value tells the underlying FirstPartials functions how many unique key counts to accumulate in the LRU cache, before emitting the least recently used entry. This accumulation happens map-side, and thus is bounded by the size of your map task JVM and the typical size of each group key.

By default, either the value of AggregateByProps.AGGREGATE_BY_CAPACITY System property or AggregateByProps.AGGREGATE_BY_DEFAULT_CAPACITY will be used.

See Also:

AggregateBy, Serialized Form

Nested Class Summary

static class

FirstBy.FirstPartials Class CountPartials is a AggregateBy.Functor that is used to count observed duplicates from the tuple stream.

Nested classes/interfaces inherited from class cascading.pipe.assembly.AggregateBy

AggregateBy.Cache, AggregateBy.CompositeFunction, AggregateBy.Flush, AggregateBy.Functor

Field Summary

Fields inherited from class cascading.pipe.assembly.AggregateBy

AGGREGATE_BY_THRESHOLD, DEFAULT_THRESHOLD, USE_DEFAULT_THRESHOLD

Fields inherited from class cascading.pipe.Pipe

configDef, parent, stepConfigDef

Constructor Summary

FirstBy(Fields firstFields)
Constructor FirstBy creates a new FirstBy instance.

FirstBy(Fields argumentFields, Fields firstFields)
Constructor FirstBy creates a new FirstBy instance.

FirstBy(Pipe[] pipes, Fields groupingFields, Fields firstFields)
Constructor FirstBy creates a new FirstBy instance.

FirstBy(Pipe[] pipes, Fields groupingFields, Fields firstFields, int threshold)
Constructor FirstBy creates a new FirstBy instance.

FirstBy(Pipe pipe, Fields groupingFields, Fields firstFields)
Constructor FirstBy creates a new FirstBy instance.

FirstBy(Pipe pipe, Fields groupingFields, Fields firstFields, int threshold)
Constructor FirstBy creates a new FirstBy instance.

FirstBy(String name, Pipe[] pipes, Fields groupingFields, Fields firstFields)
Constructor FirstBy creates a new FirstBy instance.

FirstBy(String name, Pipe[] pipes, Fields groupingFields, Fields firstFields, int threshold)
Constructor FirstBy creates a new FirstBy instance.

FirstBy(String name, Pipe pipe, Fields groupingFields, Fields firstFields)
Constructor FirstBy creates a new FirstBy instance.

FirstBy(String name, Pipe pipe, Fields groupingFields, Fields firstFields, int threshold)
Constructor FirstBy creates a new FirstBy instance.

Method Summary

Methods inherited from class cascading.pipe.assembly.AggregateBy

getAggregators, getArgumentFields, getCapacity, getFieldDeclarations, getFunctors, getGroupBy, getGroupingFields, getThreshold, initialize, initialize, verify

Methods inherited from class cascading.pipe.SubAssembly

getName, getPrevious, getTailNames, getTails, setPrevious, setTails, unwind

Methods inherited from class cascading.pipe.Pipe

equals, getConfigDef, getHeads, getParent, getStepConfigDef, getTrace, hasConfigDef, hashCode, hasStepConfigDef, id, isEquivalentTo, named, names, outgoingScopeFor, pipes, print, printInternal, resolveIncomingOperationArgumentFields, resolveIncomingOperationPassThroughFields, setParent, toString

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

FirstBy

@ConstructorProperties(value="firstFields")
public FirstBy(Fields firstFields)

Constructor FirstBy creates a new FirstBy instance. Use this constructor when used with a AggregateBy instance.

Parameters:

firstFields - of type Fields

FirstBy

@ConstructorProperties(value={"argumentFields","firstFields"})
public FirstBy(Fields argumentFields,
                                          Fields firstFields)

Constructor FirstBy creates a new FirstBy instance. Use this constructor when used with a AggregateBy instance.

Parameters:

firstFields - of type Fields

FirstBy

@ConstructorProperties(value={"pipe","groupingFields","firstFields"})
public FirstBy(Pipe pipe,
                                          Fields groupingFields,
                                          Fields firstFields)

Constructor FirstBy creates a new FirstBy instance.

Parameters:

pipe - of type Pipe

groupingFields - of type Fields

firstFields - of type Fields

FirstBy

@ConstructorProperties(value={"pipe","groupingFields","firstFields","threshold"})
public FirstBy(Pipe pipe,
                                          Fields groupingFields,
                                          Fields firstFields,
                                          int threshold)

Constructor FirstBy creates a new FirstBy instance.

Parameters:

pipe - of type Pipe

groupingFields - of type Fields

firstFields - fo type Fields

threshold - of type int

FirstBy

@ConstructorProperties(value={"name","pipe","groupingFields","firstFields"})
public FirstBy(String name,
                                          Pipe pipe,
                                          Fields groupingFields,
                                          Fields firstFields)

Constructor FirstBy creates a new FirstBy instance.

Parameters:

name - of type String

pipe - of type Pipe

groupingFields - of type Fields

firstFields - of type Fields

FirstBy

@ConstructorProperties(value={"name","pipe","groupingFields","firstFields","threshold"})
public FirstBy(String name,
                                          Pipe pipe,
                                          Fields groupingFields,
                                          Fields firstFields,
                                          int threshold)

Constructor FirstBy creates a new FirstBy instance.

Parameters:

name - of type String

pipe - of type Pipe

groupingFields - of type Fields

firstFields - of type Fields

threshold - of type int

FirstBy

@ConstructorProperties(value={"pipes","groupingFields","firstFields"})
public FirstBy(Pipe[] pipes,
                                          Fields groupingFields,
                                          Fields firstFields)

Constructor FirstBy creates a new FirstBy instance.

Parameters:

pipes - of type Pipe[]

groupingFields - of type Fields

firstFields - of type Fields

FirstBy

@ConstructorProperties(value={"pipes","groupingFields","firstFields","threshold"})
public FirstBy(Pipe[] pipes,
                                          Fields groupingFields,
                                          Fields firstFields,
                                          int threshold)

Constructor FirstBy creates a new FirstBy instance.

Parameters:

pipes - of type Pipe[]

groupingFields - of type Fields

firstFields - of type Fields

threshold - of type int

FirstBy

@ConstructorProperties(value={"name","pipes","groupingFields","firstFields"})
public FirstBy(String name,
                                          Pipe[] pipes,
                                          Fields groupingFields,
                                          Fields firstFields)

Constructor FirstBy creates a new FirstBy instance.

Parameters:

name - of type String

pipes - of type Pipe[]

groupingFields - of type Fields

firstFields - of type Fields

FirstBy

@ConstructorProperties(value={"name","pipes","groupingFields","firstFields","threshold"})
public FirstBy(String name,
                                          Pipe[] pipes,
                                          Fields groupingFields,
                                          Fields firstFields,
                                          int threshold)

Constructor FirstBy creates a new FirstBy instance.

Parameters:

name - of type String

pipes - of type Pipe[]

groupingFields - of type Fields

firstFields - of type Fields

threshold - of type int