org.apache.hadoop.hive.ql.parse
Class TaskCompiler

java.lang.Object
  org.apache.hadoop.hive.ql.parse.TaskCompiler

Direct Known Subclasses:

MapReduceCompiler, TezCompiler

public abstract class TaskCompiler
extends Object

TaskCompiler is a the base class for classes that compile operator pipelines into tasks.

Field Summary

protected HiveConf	conf
protected SessionState.LogHelper	console
protected Hive	db
protected org.apache.commons.logging.Log	LOG

Constructor Summary

TaskCompiler()

Method Summary

void	compile(ParseContext pCtx, List<Task<? extends Serializable>> rootTasks, HashSet<ReadEntity> inputs, HashSet<WriteEntity> outputs)
protected abstract void	decideExecMode(List<Task<? extends Serializable>> rootTasks, Context ctx, GlobalLimitCtx globalLimitCtx)
protected void	genColumnStatsTask(QB qb, List<LoadTableDesc> loadTableWork, List<LoadFileDesc> loadFileWork, List<Task<? extends Serializable>> rootTasks) A helper function to generate a column stats task on top of map-red task.
protected abstract void	generateTaskTree(List<Task<? extends Serializable>> rootTasks, ParseContext pCtx, List<Task<MoveWork>> mvTask, Set<ReadEntity> inputs, Set<WriteEntity> outputs)
protected void	getLeafTasks(List<Task<? extends Serializable>> rootTasks, HashSet<Task<? extends Serializable>> leaves) Find all leaf tasks of the list of root tasks.
ParseContext	getParseContext(ParseContext pCtx, List<Task<? extends Serializable>> rootTasks) Create a clone of the parse context
void	init(HiveConf conf, SessionState.LogHelper console, Hive db)
protected void	optimizeOperatorPlan(ParseContext pCtxSet, Set<ReadEntity> inputs, Set<WriteEntity> outputs)
protected abstract void	optimizeTaskPlan(List<Task<? extends Serializable>> rootTasks, ParseContext pCtx, Context ctx)
protected abstract void	setInputFormat(Task<? extends Serializable> rootTask)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

protected final org.apache.commons.logging.Log LOG

db

protected Hive db

console

protected SessionState.LogHelper console

conf

protected HiveConf conf

Constructor Detail

TaskCompiler

public TaskCompiler()

Method Detail

init

public void init(HiveConf conf,
                 SessionState.LogHelper console,
                 Hive db)

compile

public void compile(ParseContext pCtx,
                    List<Task<? extends Serializable>> rootTasks,
                    HashSet<ReadEntity> inputs,
                    HashSet<WriteEntity> outputs)
             throws SemanticException

Throws:

SemanticException

genColumnStatsTask

protected void genColumnStatsTask(QB qb,
                                  List<LoadTableDesc> loadTableWork,
                                  List<LoadFileDesc> loadFileWork,
                                  List<Task<? extends Serializable>> rootTasks)

A helper function to generate a column stats task on top of map-red task. The column stats task fetches from the output of the map-red task, constructs the column stats object and persists it to the metastore. This method generates a plan with a column stats task on top of map-red task and sets up the appropriate metadata to be used during execution.

Parameters:

qb -

getLeafTasks

protected void getLeafTasks(List<Task<? extends Serializable>> rootTasks,
                            HashSet<Task<? extends Serializable>> leaves)

Find all leaf tasks of the list of root tasks.

decideExecMode

protected abstract void decideExecMode(List<Task<? extends Serializable>> rootTasks,
                                       Context ctx,
                                       GlobalLimitCtx globalLimitCtx)
                                throws SemanticException

Throws:

SemanticException

optimizeOperatorPlan

protected void optimizeOperatorPlan(ParseContext pCtxSet,
                                    Set<ReadEntity> inputs,
                                    Set<WriteEntity> outputs)
                             throws SemanticException

Throws:

SemanticException

optimizeTaskPlan

protected abstract void optimizeTaskPlan(List<Task<? extends Serializable>> rootTasks,
                                         ParseContext pCtx,
                                         Context ctx)
                                  throws SemanticException

Throws:

SemanticException

setInputFormat

protected abstract void setInputFormat(Task<? extends Serializable> rootTask)

generateTaskTree

protected abstract void generateTaskTree(List<Task<? extends Serializable>> rootTasks,
                                         ParseContext pCtx,
                                         List<Task<MoveWork>> mvTask,
                                         Set<ReadEntity> inputs,
                                         Set<WriteEntity> outputs)
                                  throws SemanticException

Throws:

SemanticException

getParseContext

public ParseContext getParseContext(ParseContext pCtx,
                                    List<Task<? extends Serializable>> rootTasks)

Create a clone of the parse context