org.apache.hadoop.hive.ql.optimizer.spark

Class SparkSortMergeJoinFactory

java.lang.Object

org.apache.hadoop.hive.ql.optimizer.spark.SparkSortMergeJoinFactory

public final class SparkSortMergeJoinFactory
extends Object

Operator factory for Spark SMBJoin processing.

Method Summary

Methods

Modifier and Type	Method and Description
static void	annotateMapWork(GenSparkProcContext context, MapWork mapWork, SMBMapJoinOperator smbMapJoinOp, TableScanOperator ts, boolean local) Annotate MapWork, input is a SMBJoinOp that is part of a MapWork, and its root TS operator.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

annotateMapWork

public static void annotateMapWork(GenSparkProcContext context,
                   MapWork mapWork,
                   SMBMapJoinOperator smbMapJoinOp,
                   TableScanOperator ts,
                   boolean local)
                            throws SemanticException

Annotate MapWork, input is a SMBJoinOp that is part of a MapWork, and its root TS operator. 1. Initializes the MapWork's aliasToWork, pointing to big-table's TS. 2. Adds the bucketing information to the MapWork. 3. Adds localwork to the MapWork, with localWork's aliasToWork pointing to small-table's TS.

Parameters:

context - proc walker context

mapWork - mapwork to annotate

smbMapJoinOp - SMB Map Join operator to get data

ts - Table Scan operator to get data

local - Whether ts is from a 'local' source (small-table that will be loaded by SMBJoin 'local' task)

Throws:

SemanticException