org.apache.hadoop.hive.ql.exec.tez
Class RecordProcessor

java.lang.Object
  org.apache.hadoop.hive.ql.exec.tez.RecordProcessor

Direct Known Subclasses:

MapRecordProcessor, ReduceRecordProcessor

public abstract class RecordProcessor
extends Object

Process input from tez LogicalInput and write output It has different subclasses for map and reduce processing

Field Summary

protected String	CLASS_NAME
protected Map<String,org.apache.tez.runtime.api.LogicalInput>	inputs
protected boolean	isLogInfoEnabled
protected org.apache.hadoop.mapred.JobConf	jconf
static org.apache.commons.logging.Log	l4j
static MemoryMXBean	memoryMXBean
protected Map<String,org.apache.hadoop.mapred.OutputCollector>	outMap
protected Map<String,org.apache.tez.runtime.api.LogicalOutput>	outputs
protected PerfLogger	perfLogger
protected org.apache.tez.runtime.api.TezProcessorContext	processorContext
protected org.apache.tez.mapreduce.processor.MRTaskReporter	reporter

Constructor Summary

RecordProcessor()

Method Summary

protected void	createOutputMap()
protected void	logCloseInfo() Log information to be logged at the end
protected void	logProgress() Log number of records processed and memory used after processing many records

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

jconf

protected org.apache.hadoop.mapred.JobConf jconf

inputs

protected Map<String,org.apache.tez.runtime.api.LogicalInput> inputs

outputs

protected Map<String,org.apache.tez.runtime.api.LogicalOutput> outputs

outMap

protected Map<String,org.apache.hadoop.mapred.OutputCollector> outMap

processorContext

protected org.apache.tez.runtime.api.TezProcessorContext processorContext

l4j

public static final org.apache.commons.logging.Log l4j

memoryMXBean

public static MemoryMXBean memoryMXBean

isLogInfoEnabled

protected boolean isLogInfoEnabled

reporter

protected org.apache.tez.mapreduce.processor.MRTaskReporter reporter

perfLogger

protected PerfLogger perfLogger

CLASS_NAME

protected String CLASS_NAME

Constructor Detail

RecordProcessor

public RecordProcessor()

Method Detail

logCloseInfo

protected void logCloseInfo()

Log information to be logged at the end

logProgress

protected void logProgress()

Log number of records processed and memory used after processing many records

createOutputMap

protected void createOutputMap()