cascading.stats

Class BaseCachedStepStats<Configuration,JobStatusClient,Counters>

java.lang.Object

cascading.stats.CascadingStats<FlowNodeStats>

cascading.stats.FlowStepStats

cascading.stats.BaseCachedStepStats<Configuration,JobStatusClient,Counters>

All Implemented Interfaces:

ProvidesCounters, java.io.Serializable

public abstract class BaseCachedStepStats<Configuration,JobStatusClient,Counters>
extends FlowStepStats

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class cascading.stats.CascadingStats

CascadingStats.Status, CascadingStats.Type

Field Summary

Fields

Modifier and Type	Field and Description
protected CounterCache<Configuration,JobStatusClient,Counters>	counterCache

Fields inherited from class cascading.stats.CascadingStats

clientState, finishedTime, lastCaptureDetail, listeners, name, pendingTime, runTime, startTime, STATS_COMPLETE_CHILD_DETAILS_BLOCK_DURATION, STATS_STORE_INTERVAL, status, submitTime, throwable, throwableTrace

Constructor Summary

Constructors

Constructor and Description
BaseCachedStepStats(FlowStep flowStep, ClientState clientState)

Method Summary

Modifier and Type	Method and Description
protected Counters	cachedCounters(boolean force)
java.util.Collection<java.lang.String>	getCounterGroups() Method getCounterGroups returns all of the Hadoop counter groups.
java.util.Collection<java.lang.String>	getCounterGroupsMatching(java.lang.String regex) Method getCounterGroupsMatching returns all the Hadoop counter groups that match the give regex pattern.
java.util.Collection<java.lang.String>	getCountersFor(java.lang.String group) Method getCountersFor returns the Hadoop counters for the given group.
long	getCounterValue(java.lang.Enum counter) Method getCounterValue returns the Hadoop counter value for the given counter enum.
long	getCounterValue(java.lang.String group, java.lang.String counter) Method getCounterValue returns the Hadoop counter value for the given group and counter name.
abstract JobStatusClient	getJobStatusClient() Method getRunningJob returns the Hadoop org.apache.hadoop.mapred.RunningJob managing this Hadoop job.
long	getLastSuccessfulCounterFetchTime() Method getLastSuccessfulCounterFetchTime returns the time, in millis, the last moment counters were successfully retrieved.
void	recordChildStats() Synchronized to prevent state changes mid record, #stop may be called out of band

Methods inherited from class cascading.stats.FlowStepStats

addNodeStats, getChildren, getChildWith, getFlowNodeIDs, getFlowNodeStats, getFlowNodeStatsMap, getFlowStep, getID, getNodesCount, getProcessLogger, getProcessStatusURL, getProcessStepID, getType, hasCapturedFinalDetail, recordInfo, toString

Methods inherited from class cascading.stats.CascadingStats

addListener, captureDetail, captureDetail, cleanup, fireListeners, getCountersFor, getCurrentDuration, getDuration, getFinishedTime, getName, getPendingTime, getPrefix, getRunTime, getStartTime, getStatsString, getStatus, getSubmitTime, getThrowable, getThrowableTrace, isDetailStale, isEngaged, isFailed, isFinished, isPending, isRunning, isSkipped, isStarted, isStopped, isSubmitted, isSuccessful, logDebug, logError, logError, logInfo, logWarn, markDetailCaptured, markFailed, markFailed, markFailed, markFailed, markFinishedTime, markPending, markPendingTime, markRunning, markRunTime, markSkipped, markStarted, markStartedThenRunning, markStartTime, markStartToRunTime, markStopped, markSubmitted, markSubmitTime, markSuccessful, prepare, recordStats, removeListener, setStatsStoreInterval

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

counterCache

protected CounterCache<Configuration,JobStatusClient,Counters> counterCache

Constructor Detail

BaseCachedStepStats

public BaseCachedStepStats(FlowStep flowStep,
                           ClientState clientState)

Method Detail

getJobStatusClient

public abstract JobStatusClient getJobStatusClient()

Method getRunningJob returns the Hadoop org.apache.hadoop.mapred.RunningJob managing this Hadoop job.

Returns:

the runningJob (type RunningJob) of this HadoopStepStats object.

getLastSuccessfulCounterFetchTime

public long getLastSuccessfulCounterFetchTime()

Description copied from interface: ProvidesCounters

Method getLastSuccessfulCounterFetchTime returns the time, in millis, the last moment counters were successfully retrieved.

If -1, counter values were never successfully retrieved.

If this return value is less than the CascadingStats.getFinishedTime() it is likely the counter service became unavailable.

Returns:

the moment counters were last successfully retrieved

getCounterGroups

public java.util.Collection<java.lang.String> getCounterGroups()

Method getCounterGroups returns all of the Hadoop counter groups.

Returns:

the counterGroups (type Collection) of this HadoopStepStats object.

getCounterGroupsMatching

public java.util.Collection<java.lang.String> getCounterGroupsMatching(java.lang.String regex)

Method getCounterGroupsMatching returns all the Hadoop counter groups that match the give regex pattern.

Specified by:

getCounterGroupsMatching in class CascadingStats<FlowNodeStats>

Parameters:

regex - of String

Returns:

Collection

getCountersFor

public java.util.Collection<java.lang.String> getCountersFor(java.lang.String group)

Method getCountersFor returns the Hadoop counters for the given group.

Parameters:

group - of String

Returns:

Collection

getCounterValue

public long getCounterValue(java.lang.Enum counter)

Method getCounterValue returns the Hadoop counter value for the given counter enum.

Parameters:

counter - of Enum

Returns:

long

getCounterValue

public long getCounterValue(java.lang.String group,
                            java.lang.String counter)

Method getCounterValue returns the Hadoop counter value for the given group and counter name.

Parameters:

group - of String

counter - of String

Returns:

long

cachedCounters

protected Counters cachedCounters(boolean force)

recordChildStats

public void recordChildStats()

Synchronized to prevent state changes mid record, #stop may be called out of band

Specified by:

recordChildStats in class FlowStepStats