org.apache.hadoop.hive.ql.parse
Class SemanticAnalyzer

java.lang.Object
  org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer
      org.apache.hadoop.hive.ql.parse.SemanticAnalyzer

Direct Known Subclasses:

ColumnStatsSemanticAnalyzer

public class SemanticAnalyzer
extends BaseSemanticAnalyzer

Implementation of the semantic analyzer. It generates the query plan. There are other specific semantic analyzers for some hive operations such as DDLSemanticAnalyzer for ddl operations.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer

BaseSemanticAnalyzer.tableSpec

Field Summary

static String	DUMMY_DATABASE
static String	DUMMY_TABLE
protected boolean	noscan
protected boolean	partialscan

Fields inherited from class org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer

columnAccessInfo, conf, console, ctx, db, fetchTask, HIVE_COLUMN_ORDER_ASC, HIVE_COLUMN_ORDER_DESC, idToTableNameMap, inputs, linfo, LOG, ORCFILE_INPUT, ORCFILE_OUTPUT, ORCFILE_SERDE, outputs, PARQUETFILE_INPUT, PARQUETFILE_OUTPUT, PARQUETFILE_SERDE, queryProperties, RCFILE_INPUT, RCFILE_OUTPUT, rootTasks, SEQUENCEFILE_INPUT, SEQUENCEFILE_OUTPUT, tableAccessInfo, TEXTFILE_INPUT, TEXTFILE_OUTPUT

Constructor Summary

SemanticAnalyzer(HiveConf conf)

Method Summary

void	analyzeInternal(ASTNode ast)
boolean	doPhase1(ASTNode ast, QB qb, org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.Phase1Ctx ctx_1) Phase 1: (including, but not limited to): 1.
void	doPhase1QBExpr(ASTNode ast, QBExpr qbexpr, String id, String alias)
Map<ASTNode,ExprNodeDesc>	genAllExprNodeDesc(ASTNode expr, RowResolver input) Generates an expression node descriptors for the expression and children of it with default TypeCheckCtx.
Map<ASTNode,ExprNodeDesc>	genAllExprNodeDesc(ASTNode expr, RowResolver input, TypeCheckCtx tcCtx) Generates all of the expression node descriptors for the expression and children of it passed in the arguments.
static String	generateErrorMessage(ASTNode ast, String message)
ExprNodeDesc	genExprNodeDesc(ASTNode expr, RowResolver input) Generates an expression node descriptor for the expression with TypeCheckCtx.
ExprNodeDesc	genExprNodeDesc(ASTNode expr, RowResolver input, TypeCheckCtx tcCtx) Returns expression node descriptor for the expression.
Operator	genPlan(QB qb)
static String	getColumnInternalName(int pos)
void	getMetaData(QB qb)
void	getMetaData(QB qb, ReadEntity parentInput)
ParseContext	getParseContext()
QB	getQB()
List<FieldSchema>	getResultSchema()
RowResolver	getRowResolver(Operator opt) Get the row resolver given an operator.
void	init()
void	initParseCtx(ParseContext pctx)
org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.Phase1Ctx	initPhase1Ctx()
static ArrayList<WindowingSpec.WindowExpressionSpec>	parseSelect(String selectExprStr)
protected void	processNoScanCommand(ASTNode tree) process analyze ...
protected void	processPartialScanCommand(ASTNode tree) process analyze ...
<T extends OperatorDesc> Operator<T>	putOpInsertMap(Operator<T> op, RowResolver rr)
protected void	reset()
void	setQB(QB qb)
void	validate()

Methods inherited from class org.apache.hadoop.hive.ql.parse.BaseSemanticAnalyzer

analyze, analyzeDDLSkewedValues, analyzeSkewedTablDDLColNames, analyzeStoredAdDirs, charSetString, constructListBucketingCtx, createHiveDB, extractPartitionSpecs, getColumnAccessInfo, getColumnNames, getColumnNamesOrder, getColumns, getColumns, getDatabase, getDatabase, getDb, getFetchTask, getIdToTableNameMap, getInputs, getLineageInfo, getOutputs, getPartition, getPartitions, getQueryProperties, getRootTasks, getSkewedValueFromASTNode, getSkewedValuesFromASTNode, getTable, getTable, getTable, getTableAccessInfo, getTableWithQN, getTypeStringFromAST, getUnescapedName, getUnescapedName, getUnescapedUnqualifiedTableName, handleGenericFileFormat, initCtx, isValidPrefixSpec, readProps, setColumnAccessInfo, setFetchTask, setLineageInfo, setTableAccessInfo, stripIdentifierQuotes, stripQuotes, toMessage, unescapeIdentifier, unescapeSQLString, validatePartSpec

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DUMMY_DATABASE

public static final String DUMMY_DATABASE

See Also:

Constant Field Values

DUMMY_TABLE

public static final String DUMMY_TABLE

See Also:

Constant Field Values

noscan

protected boolean noscan

partialscan

protected boolean partialscan

Constructor Detail

SemanticAnalyzer

public SemanticAnalyzer(HiveConf conf)
                 throws SemanticException

Throws:

SemanticException

Method Detail

reset

protected void reset()

Overrides:

reset in class BaseSemanticAnalyzer

initParseCtx

public void initParseCtx(ParseContext pctx)

getParseContext

public ParseContext getParseContext()

doPhase1QBExpr

public void doPhase1QBExpr(ASTNode ast,
                           QBExpr qbexpr,
                           String id,
                           String alias)
                    throws SemanticException

Throws:

SemanticException

generateErrorMessage

public static String generateErrorMessage(ASTNode ast,
                                          String message)

doPhase1

public boolean doPhase1(ASTNode ast,
                        QB qb,
                        org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.Phase1Ctx ctx_1)
                 throws SemanticException

Phase 1: (including, but not limited to): 1. Gets all the aliases for all the tables / subqueries and makes the appropriate mapping in aliasToTabs, aliasToSubq 2. Gets the location of the destination and names the clase "inclause" + i 3. Creates a map from a string representation of an aggregation tree to the actual aggregation AST 4. Creates a mapping from the clause name to the select expression AST in destToSelExpr 5. Creates a mapping from a table alias to the lateral view AST's in aliasToLateralViews

Parameters:

ast -

qb -

ctx_1 -

Throws:

SemanticException

getMetaData

public void getMetaData(QB qb)
                 throws SemanticException

Throws:

SemanticException

getMetaData

public void getMetaData(QB qb,
                        ReadEntity parentInput)
                 throws SemanticException

Throws:

SemanticException

putOpInsertMap

public <T extends OperatorDesc> Operator<T> putOpInsertMap(Operator<T> op,
                                                           RowResolver rr)

getColumnInternalName

public static String getColumnInternalName(int pos)

genPlan

public Operator genPlan(QB qb)
                 throws SemanticException

Throws:

SemanticException

initPhase1Ctx

public org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.Phase1Ctx initPhase1Ctx()

init

public void init()

Overrides:

init in class BaseSemanticAnalyzer

analyzeInternal

public void analyzeInternal(ASTNode ast)
                     throws SemanticException

Specified by:

analyzeInternal in class BaseSemanticAnalyzer

Throws:

SemanticException

getResultSchema

public List<FieldSchema> getResultSchema()

Overrides:

getResultSchema in class BaseSemanticAnalyzer

Returns:

the schema for the fields which will be produced when the statement is executed, or null if not known

genExprNodeDesc

public ExprNodeDesc genExprNodeDesc(ASTNode expr,
                                    RowResolver input)
                             throws SemanticException

Generates an expression node descriptor for the expression with TypeCheckCtx.

Throws:

SemanticException

genAllExprNodeDesc

public Map<ASTNode,ExprNodeDesc> genAllExprNodeDesc(ASTNode expr,
                                                    RowResolver input)
                                             throws SemanticException

Generates an expression node descriptors for the expression and children of it with default TypeCheckCtx.

Throws:

SemanticException

genExprNodeDesc

public ExprNodeDesc genExprNodeDesc(ASTNode expr,
                                    RowResolver input,
                                    TypeCheckCtx tcCtx)
                             throws SemanticException

Returns expression node descriptor for the expression. If it's evaluated already in previous operator, it can be retrieved from cache.

Throws:

SemanticException

genAllExprNodeDesc

public Map<ASTNode,ExprNodeDesc> genAllExprNodeDesc(ASTNode expr,
                                                    RowResolver input,
                                                    TypeCheckCtx tcCtx)
                                             throws SemanticException

Generates all of the expression node descriptors for the expression and children of it passed in the arguments. This function uses the row resolver and the metadata information that are passed as arguments to resolve the column names to internal names.

Parameters:

expr - The expression

input - The row resolver

tcCtx - Customized type-checking context

Returns:

expression to exprNodeDesc mapping

Throws:

SemanticException - Failed to evaluate expression

validate

public void validate()
              throws SemanticException

Overrides:

validate in class BaseSemanticAnalyzer

Throws:

SemanticException

getRowResolver

public RowResolver getRowResolver(Operator opt)

Get the row resolver given an operator.

processPartialScanCommand

protected void processPartialScanCommand(ASTNode tree)
                                  throws SemanticException

process analyze ... partial command separate it from noscan command process so that it provides us flexibility

Parameters:

tree -

Throws:

SemanticException

processNoScanCommand

protected void processNoScanCommand(ASTNode tree)
                             throws SemanticException

process analyze ... noscan command

Parameters:

tree -

Throws:

SemanticException

getQB

public QB getQB()

setQB

public void setQB(QB qb)

parseSelect

public static ArrayList<WindowingSpec.WindowExpressionSpec> parseSelect(String selectExprStr)
                                                                 throws SemanticException

Throws:

SemanticException