com.pivotal.gemfirexd.hadoop.mapreduce

Class RowInputFormat

java.lang.Object

org.apache.hadoop.mapreduce.InputFormat<Key,Row>

com.pivotal.gemfirexd.hadoop.mapreduce.RowInputFormat

public class RowInputFormat
extends org.apache.hadoop.mapreduce.InputFormat<Key,Row>

This class extends from InputFormat to provide table data reading ability.

Field Summary

Fields

Modifier and Type	Field and Description
static java.lang.String	CHECKPOINT_MODE In this mode checkpoint data is provided to job.
protected org.apache.hadoop.conf.Configuration	conf
static java.lang.String	END_TIME_MILLIS If this filter is set, an event will be provided to map reduce job if it happened before this timestamp in milliseconds
static java.lang.String	HOME_DIR The home directory for gemfirexd data in HDFS.
static java.lang.String	INPUT_TABLE The name of the table to process, for example APP.CUSTOMERS.
static java.lang.String	PROPERTY_PREFIX A valid property starting with this prefix will be used to tune input format's behavior.
static java.lang.String	START_TIME_MILLIS If this filter is set, an event will be provided to map reduce job if it happened after this timestamp in milliseconds

Constructor Summary

Constructors

Constructor and Description
RowInputFormat()

Method Summary

Modifier and Type	Method and Description
org.apache.hadoop.mapreduce.RecordReader<Key,Row>	createRecordReader(org.apache.hadoop.mapreduce.InputSplit split, org.apache.hadoop.mapreduce.TaskAttemptContext context)
java.util.List<org.apache.hadoop.mapreduce.InputSplit>	createSplits(java.util.Collection<org.apache.hadoop.fs.FileStatus> hoplogs)
static java.lang.String	getFullyQualifiedTableName(java.lang.String name)
java.util.Collection<org.apache.hadoop.fs.FileStatus>	getHoplogs(org.apache.hadoop.conf.Configuration job) Identifies filters provided in the job configuration and creates a list of sorted hoplogs satisfying the filters.
java.util.List<org.apache.hadoop.mapreduce.InputSplit>	getSplits(org.apache.hadoop.mapreduce.JobContext job)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

HOME_DIR

public static final java.lang.String HOME_DIR

The home directory for gemfirexd data in HDFS. This should match the HOMEDIR setting that was used with the CREATE HDFSSTORE in statement.

See Also:

Constant Field Values

INPUT_TABLE

public static final java.lang.String INPUT_TABLE

The name of the table to process, for example APP.CUSTOMERS. This should match the table name used with the CREATE TABLE statement.

See Also:

Constant Field Values

START_TIME_MILLIS

public static final java.lang.String START_TIME_MILLIS

If this filter is set, an event will be provided to map reduce job if it happened after this timestamp in milliseconds

See Also:

Constant Field Values

END_TIME_MILLIS

public static final java.lang.String END_TIME_MILLIS

If this filter is set, an event will be provided to map reduce job if it happened before this timestamp in milliseconds

See Also:

Constant Field Values

CHECKPOINT_MODE

public static final java.lang.String CHECKPOINT_MODE

In this mode checkpoint data is provided to job. Mapper, instead of getting all the events that happened in the system, which also includes old values of a key, gets the latest value known at the time of creating the checkpoint only. Checkpoint may not include latest data present in the system when the job is executed. In the checkpoint mode, time filters are ignored

See Also:

Constant Field Values

PROPERTY_PREFIX

public static final java.lang.String PROPERTY_PREFIX

A valid property starting with this prefix will be used to tune input format's behavior.

See Also:

Constant Field Values

conf

protected org.apache.hadoop.conf.Configuration conf

Constructor Detail

RowInputFormat

public RowInputFormat()

Method Detail

createRecordReader

public org.apache.hadoop.mapreduce.RecordReader<Key,Row> createRecordReader(org.apache.hadoop.mapreduce.InputSplit split,
                                                                            org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                                     throws java.io.IOException,
                                                                            java.lang.InterruptedException

Specified by:

createRecordReader in class org.apache.hadoop.mapreduce.InputFormat<Key,Row>

Throws:

java.io.IOException

java.lang.InterruptedException

getSplits

public java.util.List<org.apache.hadoop.mapreduce.InputSplit> getSplits(org.apache.hadoop.mapreduce.JobContext job)
                                                                 throws java.io.IOException,
                                                                        java.lang.InterruptedException

Specified by:

getSplits in class org.apache.hadoop.mapreduce.InputFormat<Key,Row>

Throws:

java.io.IOException

java.lang.InterruptedException

getHoplogs

public java.util.Collection<org.apache.hadoop.fs.FileStatus> getHoplogs(org.apache.hadoop.conf.Configuration job)
                                                                 throws java.io.IOException

Identifies filters provided in the job configuration and creates a list of sorted hoplogs satisfying the filters. If there are no sorted hoplogs, checks if sequential hoplogs are present and executes the same process for sequential hoplogs

Returns:

list of hoplogs

Throws:

java.io.IOException

getFullyQualifiedTableName

public static java.lang.String getFullyQualifiedTableName(java.lang.String name)

createSplits

public java.util.List<org.apache.hadoop.mapreduce.InputSplit> createSplits(java.util.Collection<org.apache.hadoop.fs.FileStatus> hoplogs)
                                                                    throws java.io.IOException

Throws:

java.io.IOException