eu.stratosphere.hadoopcompatibility

Class HadoopOutputFormatWrapper<K,V>

java.lang.Object

eu.stratosphere.hadoopcompatibility.HadoopOutputFormatWrapper<K,V>

All Implemented Interfaces:

eu.stratosphere.api.common.io.OutputFormat<eu.stratosphere.types.Record>, Serializable

public class HadoopOutputFormatWrapper<K,V>
extends Object
implements eu.stratosphere.api.common.io.OutputFormat<eu.stratosphere.types.Record>

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
StratosphereTypeConverter<K,V>	converter
FileOutputCommitterWrapper	fileOutputCommitterWrapper
org.apache.hadoop.mapred.OutputFormat<K,V>	hadoopOutputFormat
org.apache.hadoop.mapred.JobConf	jobConf
org.apache.hadoop.mapred.RecordWriter<K,V>	recordWriter

Constructor Summary

Constructors

Constructor and Description
HadoopOutputFormatWrapper(org.apache.hadoop.mapred.OutputFormat<K,V> hadoopFormat, org.apache.hadoop.mapred.JobConf job, StratosphereTypeConverter<K,V> conv)

Method Summary

Modifier and Type	Method and Description
void	close() commit the task by moving the output file out from the temporary directory.
void	configure(eu.stratosphere.configuration.Configuration parameters)
org.apache.hadoop.mapred.OutputFormat<K,V>	getHadoopOutputFormat()
org.apache.hadoop.mapred.JobConf	getJobConf()
void	open(int taskNumber, int numTasks) create the temporary output file for hadoop RecordWriter.
void	setHadoopOutputFormat(org.apache.hadoop.mapred.OutputFormat<K,V> hadoopOutputFormat)
void	setJobConf(org.apache.hadoop.mapred.JobConf job)
void	writeRecord(eu.stratosphere.types.Record record)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

jobConf

public org.apache.hadoop.mapred.JobConf jobConf

hadoopOutputFormat

public org.apache.hadoop.mapred.OutputFormat<K,V> hadoopOutputFormat

recordWriter

public org.apache.hadoop.mapred.RecordWriter<K,V> recordWriter

converter

public StratosphereTypeConverter<K,V> converter

fileOutputCommitterWrapper

public FileOutputCommitterWrapper fileOutputCommitterWrapper

Constructor Detail

HadoopOutputFormatWrapper

public HadoopOutputFormatWrapper(org.apache.hadoop.mapred.OutputFormat<K,V> hadoopFormat,
                                 org.apache.hadoop.mapred.JobConf job,
                                 StratosphereTypeConverter<K,V> conv)

Method Detail

configure

public void configure(eu.stratosphere.configuration.Configuration parameters)

Specified by:

configure in interface eu.stratosphere.api.common.io.OutputFormat<eu.stratosphere.types.Record>

open

public void open(int taskNumber,
                 int numTasks)
          throws IOException

create the temporary output file for hadoop RecordWriter.

Specified by:

open in interface eu.stratosphere.api.common.io.OutputFormat<eu.stratosphere.types.Record>

Parameters:

taskNumber - The number of the parallel instance.

numTasks - The number of parallel tasks.

Throws:

IOException

writeRecord

public void writeRecord(eu.stratosphere.types.Record record)
                 throws IOException

Specified by:

writeRecord in interface eu.stratosphere.api.common.io.OutputFormat<eu.stratosphere.types.Record>

Throws:

IOException

close

public void close()
           throws IOException

commit the task by moving the output file out from the temporary directory.

Specified by:

close in interface eu.stratosphere.api.common.io.OutputFormat<eu.stratosphere.types.Record>

Throws:

IOException

setJobConf

public void setJobConf(org.apache.hadoop.mapred.JobConf job)

getJobConf

public org.apache.hadoop.mapred.JobConf getJobConf()

getHadoopOutputFormat

public org.apache.hadoop.mapred.OutputFormat<K,V> getHadoopOutputFormat()

setHadoopOutputFormat

public void setHadoopOutputFormat(org.apache.hadoop.mapred.OutputFormat<K,V> hadoopOutputFormat)