cascading.stats.hadoop

Class HadoopStepStats

java.lang.Object

cascading.stats.CascadingStats<cascading.stats.FlowNodeStats>

cascading.stats.FlowStepStats

cascading.stats.BaseCachedStepStats<Configuration,RunningJob,Counters>

cascading.stats.hadoop.HadoopStepStats

All Implemented Interfaces:

cascading.stats.ProvidesCounters, java.io.Serializable

public abstract class HadoopStepStats
extends cascading.stats.BaseCachedStepStats<Configuration,RunningJob,Counters>

Class HadoopStepStats provides Hadoop specific statistics and methods to underlying Hadoop facilities.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class cascading.stats.CascadingStats

cascading.stats.CascadingStats.Status, cascading.stats.CascadingStats.Type

Field Summary

Fields inherited from class cascading.stats.BaseCachedStepStats

counterCache

Fields inherited from class cascading.stats.CascadingStats

clientState, finishedTime, lastCaptureDetail, listeners, name, pendingTime, runTime, startTime, STATS_COMPLETE_CHILD_DETAILS_BLOCK_DURATION, STATS_STORE_INTERVAL, status, submitTime, throwable, throwableTrace

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	HadoopStepStats(cascading.flow.FlowStep<JobConf> flowStep, cascading.management.state.ClientState clientState)

Method Summary

Modifier and Type	Method and Description
void	captureDetail(cascading.stats.CascadingStats.Type depth) Method captureDetail captures statistics task details and completion events.
protected static Job	getJob(RunningJob runningJob)
abstract JobClient	getJobClient() Method getJobClient returns the Hadoop JobClient managing this Hadoop job.
float	getMapProgress() Returns the underlying Map tasks progress percentage.
int	getNumMapTasks() Method getNumMapTasks returns the numMapTasks from the Hadoop job file.
int	getNumReduceTasks() Method getNumReduceTasks returns the numReducerTasks from the Hadoop job file.
java.lang.String	getProcessStatusURL()
java.lang.String	getProcessStepID()
float	getReduceProgress() Returns the underlying Reduce tasks progress percentage.
java.lang.String	getStatusURL() Deprecated. see getProcessStatusURL()

Methods inherited from class cascading.stats.BaseCachedStepStats

cachedCounters, getCounterGroups, getCounterGroupsMatching, getCountersFor, getCounterValue, getCounterValue, getJobStatusClient, getLastSuccessfulCounterFetchTime, recordChildStats

Methods inherited from class cascading.stats.FlowStepStats

addNodeStats, getChildren, getChildWith, getFlowNodeIDs, getFlowNodeStats, getFlowNodeStatsMap, getFlowStep, getID, getNodesCount, getProcessLogger, getType, hasCapturedFinalDetail, recordInfo, toString

Methods inherited from class cascading.stats.CascadingStats

addListener, captureDetail, cleanup, fireListeners, getCountersFor, getCurrentDuration, getDuration, getFinishedTime, getName, getPendingTime, getPrefix, getRunTime, getStartTime, getStatsString, getStatus, getSubmitTime, getThrowable, getThrowableTrace, isDetailStale, isEngaged, isFailed, isFinished, isPending, isRunning, isSkipped, isStarted, isStopped, isSubmitted, isSuccessful, logDebug, logError, logError, logInfo, logWarn, markDetailCaptured, markFailed, markFailed, markFailed, markFailed, markFinishedTime, markPending, markPendingTime, markRunning, markRunTime, markSkipped, markStarted, markStartedThenRunning, markStartTime, markStartToRunTime, markStopped, markSubmitted, markSubmitTime, markSuccessful, prepare, recordStats, removeListener, setStatsStoreInterval

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

HadoopStepStats

protected HadoopStepStats(cascading.flow.FlowStep<JobConf> flowStep,
                          cascading.management.state.ClientState clientState)

Method Detail

getJob

protected static Job getJob(RunningJob runningJob)

getNumMapTasks

public int getNumMapTasks()

Method getNumMapTasks returns the numMapTasks from the Hadoop job file.

Returns:

the numMapTasks (type int) of this HadoopStepStats object.

getNumReduceTasks

public int getNumReduceTasks()

Method getNumReduceTasks returns the numReducerTasks from the Hadoop job file.

Returns:

the numReducerTasks (type int) of this HadoopStepStats object.

getProcessStepID

public java.lang.String getProcessStepID()

Specified by:

getProcessStepID in class cascading.stats.FlowStepStats

getJobClient

public abstract JobClient getJobClient()

Method getJobClient returns the Hadoop JobClient managing this Hadoop job.

Returns:

the jobClient (type JobClient) of this HadoopStepStats object.

getMapProgress

public float getMapProgress()

Returns the underlying Map tasks progress percentage.

This method is experimental.

Returns:

float

getReduceProgress

public float getReduceProgress()

Returns the underlying Reduce tasks progress percentage.

This method is experimental.

Returns:

float

getProcessStatusURL

public java.lang.String getProcessStatusURL()

Overrides:

getProcessStatusURL in class cascading.stats.FlowStepStats

getStatusURL

@Deprecated
public java.lang.String getStatusURL()

Deprecated. see getProcessStatusURL()

captureDetail

public void captureDetail(cascading.stats.CascadingStats.Type depth)

Method captureDetail captures statistics task details and completion events.

Specified by:

captureDetail in class cascading.stats.CascadingStats<cascading.stats.FlowNodeStats>