com.pivotal.gemfirexd.hadoop.mapred

Class RowOutputFormat<VALUE>

java.lang.Object

org.apache.hadoop.mapreduce.OutputFormat<Key,VALUE>

com.pivotal.gemfirexd.hadoop.mapreduce.RowOutputFormat<VALUE>

com.pivotal.gemfirexd.hadoop.mapred.RowOutputFormat<VALUE>

Type Parameters:

VALUE - User's data class containing setter methods for table columns

All Implemented Interfaces:

org.apache.hadoop.mapred.OutputFormat<Key,VALUE>

public class RowOutputFormat<VALUE>
extends RowOutputFormat<VALUE>
implements org.apache.hadoop.mapred.OutputFormat<Key,VALUE>

Map Reduce OutputFormat implementation for Gemfirexd. This class uses client connection to populate records in the DB. User is responsible for knowing name and type of the table columns. User provides a data class which contains setter methods for each column which needs to be populated. This class calls each setter method to populate the insert query.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
class	RowOutputFormat.MapRedGfxdRecordWriter RecordWriter implementation for Gemfirexd.

Nested classes/interfaces inherited from class com.pivotal.gemfirexd.hadoop.mapreduce.RowOutputFormat

RowOutputFormat.GfxdRecordWriter

Field Summary

Fields

Modifier and Type	Field and Description
static String	OUTPUT_BATCH_SIZE By default row insert commands are executed in batch mode.
static String	OUTPUT_TABLE The name of the table to output to, for example APP.CUSTOMERS.
static String	OUTPUT_URL JDBC url used to connect to a running gemfirexd instance.

Fields inherited from class com.pivotal.gemfirexd.hadoop.mapreduce.RowOutputFormat

OUTPUT_BATCH_SIZE_DEFAULT

Constructor Summary

Constructors

Constructor and Description
RowOutputFormat()

Method Summary

Methods

Modifier and Type	Method and Description
void	checkOutputSpecs(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.mapred.JobConf job)
org.apache.hadoop.mapred.RecordWriter<Key,VALUE>	getRecordWriter(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.mapred.JobConf job, String name, org.apache.hadoop.util.Progressable progress)

Methods inherited from class com.pivotal.gemfirexd.hadoop.mapreduce.RowOutputFormat

checkOutputSpecs, getDriver, getOutputCommitter, getRecordWriter, validateConfiguration

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

OUTPUT_TABLE

public static final String OUTPUT_TABLE

The name of the table to output to, for example APP.CUSTOMERS. This should match the table name used with the CREATE TABLE statement.

See Also:

Constant Field Values

OUTPUT_URL

public static final String OUTPUT_URL

JDBC url used to connect to a running gemfirexd instance. For example jdbc:gemfirexd://myHostName:1527/

See Also:

Constant Field Values

OUTPUT_BATCH_SIZE

public static final String OUTPUT_BATCH_SIZE

By default row insert commands are executed in batch mode. The batch size can be configured using this property. The default value is 10K commands. If the batch size is equal or less than 0, then the batch is when close operation is executed

See Also:

Constant Field Values

Constructor Detail

RowOutputFormat

public RowOutputFormat()

Method Detail

checkOutputSpecs

public void checkOutputSpecs(org.apache.hadoop.fs.FileSystem fs,
                    org.apache.hadoop.mapred.JobConf job)
                      throws IOException

Specified by:

checkOutputSpecs in interface org.apache.hadoop.mapred.OutputFormat<Key,VALUE>

Throws:

IOException

getRecordWriter

public org.apache.hadoop.mapred.RecordWriter<Key,VALUE> getRecordWriter(org.apache.hadoop.fs.FileSystem fs,
                                                               org.apache.hadoop.mapred.JobConf job,
                                                               String name,
                                                               org.apache.hadoop.util.Progressable progress)
                                                                 throws IOException

Specified by:

getRecordWriter in interface org.apache.hadoop.mapred.OutputFormat<Key,VALUE>

Throws:

IOException