org.apache.hadoop.hive.hbase
Class HBaseSerDe

java.lang.Object
  org.apache.hadoop.hive.serde2.AbstractSerDe
      org.apache.hadoop.hive.hbase.HBaseSerDe

All Implemented Interfaces:

Deserializer, SerDe, Serializer

public class HBaseSerDe
extends AbstractSerDe

HBaseSerDe can be used to serialize object into an HBase table and deserialize objects from an HBase table.

Field Summary

static String	HBASE_COLUMNS_MAPPING
static String	HBASE_COLUMNS_REGEX_MATCHING Determines whether a regex matching should be done on the columns or not.
static String	HBASE_COMPOSITE_KEY_CLASS
static String	HBASE_KEY_COL
static String	HBASE_PUT_TIMESTAMP
static String	HBASE_SCAN_BATCH
static String	HBASE_SCAN_CACHE
static String	HBASE_SCAN_CACHEBLOCKS
static String	HBASE_TABLE_DEFAULT_STORAGE_TYPE
static String	HBASE_TABLE_NAME
static org.apache.commons.logging.Log	LOG

Constructor Summary

HBaseSerDe()

Method Summary

Object	deserialize(org.apache.hadoop.io.Writable result) Deserialize a row from the HBase Result writable to a LazyObject
ObjectInspector	getObjectInspector()
static int	getRowKeyColumnOffset(List<org.apache.hadoop.hive.hbase.HBaseSerDe.ColumnMapping> columnsMapping)
SerDeStats	getSerDeStats()
Class<? extends org.apache.hadoop.io.Writable>	getSerializedClass()
void	initialize(org.apache.hadoop.conf.Configuration conf, Properties tbl) Initialize the SerDe given parameters.
static boolean	isRowKeyColumn(String hbaseColumnName)
boolean	isUseJSONSerialize()
static List<org.apache.hadoop.hive.hbase.HBaseSerDe.ColumnMapping>	parseColumnsMapping(String columnsMappingSpec) Parses the HBase columns mapping specifier to identify the column families, qualifiers and also caches the byte arrays corresponding to them.
static List<org.apache.hadoop.hive.hbase.HBaseSerDe.ColumnMapping>	parseColumnsMapping(String columnsMappingSpec, boolean doColumnRegexMatching) Parses the HBase columns mapping specifier to identify the column families, qualifiers and also caches the byte arrays corresponding to them.
org.apache.hadoop.io.Writable	serialize(Object obj, ObjectInspector objInspector)
void	setUseJSONSerialize(boolean useJSONSerialize)
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

HBASE_COLUMNS_MAPPING

public static final String HBASE_COLUMNS_MAPPING

See Also:

Constant Field Values

HBASE_TABLE_NAME

public static final String HBASE_TABLE_NAME

See Also:

Constant Field Values

HBASE_TABLE_DEFAULT_STORAGE_TYPE

public static final String HBASE_TABLE_DEFAULT_STORAGE_TYPE

See Also:

Constant Field Values

HBASE_KEY_COL

public static final String HBASE_KEY_COL

See Also:

Constant Field Values

HBASE_PUT_TIMESTAMP

public static final String HBASE_PUT_TIMESTAMP

See Also:

Constant Field Values

HBASE_COMPOSITE_KEY_CLASS

public static final String HBASE_COMPOSITE_KEY_CLASS

See Also:

Constant Field Values

HBASE_SCAN_CACHE

public static final String HBASE_SCAN_CACHE

See Also:

Constant Field Values

HBASE_SCAN_CACHEBLOCKS

public static final String HBASE_SCAN_CACHEBLOCKS

See Also:

Constant Field Values

HBASE_SCAN_BATCH

public static final String HBASE_SCAN_BATCH

See Also:

Constant Field Values

HBASE_COLUMNS_REGEX_MATCHING

public static final String HBASE_COLUMNS_REGEX_MATCHING

Determines whether a regex matching should be done on the columns or not. Defaults to true. WARNING: Note that currently this only supports the suffix wildcard .*

See Also:

Constant Field Values

LOG

public static final org.apache.commons.logging.Log LOG

Constructor Detail

HBaseSerDe

public HBaseSerDe()
           throws SerDeException

Throws:

SerDeException

Method Detail

toString

public String toString()

Overrides:

toString in class Object

initialize

public void initialize(org.apache.hadoop.conf.Configuration conf,
                       Properties tbl)
                throws SerDeException

Initialize the SerDe given parameters.

Specified by:

initialize in interface Deserializer

Specified by:

initialize in interface Serializer

Specified by:

initialize in class AbstractSerDe

Throws:

SerDeException

See Also:

SerDe#initialize(Configuration, Properties)

parseColumnsMapping

public static List<org.apache.hadoop.hive.hbase.HBaseSerDe.ColumnMapping> parseColumnsMapping(String columnsMappingSpec)
                                                                                       throws SerDeException

Parses the HBase columns mapping specifier to identify the column families, qualifiers and also caches the byte arrays corresponding to them. One of the Hive table columns maps to the HBase row key, by default the first column.

Parameters:

columnsMappingSpec - string hbase.columns.mapping specified when creating table

Returns:

List which contains the column mapping information by position

Throws:

SerDeException

parseColumnsMapping

public static List<org.apache.hadoop.hive.hbase.HBaseSerDe.ColumnMapping> parseColumnsMapping(String columnsMappingSpec,
                                                                                              boolean doColumnRegexMatching)
                                                                                       throws SerDeException

Parses the HBase columns mapping specifier to identify the column families, qualifiers and also caches the byte arrays corresponding to them. One of the Hive table columns maps to the HBase row key, by default the first column.

Parameters:

columnsMappingSpec - string hbase.columns.mapping specified when creating table

doColumnRegexMatching - whether to do a regex matching on the columns or not

Returns:

List which contains the column mapping information by position

Throws:

SerDeException

isRowKeyColumn

public static boolean isRowKeyColumn(String hbaseColumnName)

deserialize

public Object deserialize(org.apache.hadoop.io.Writable result)
                   throws SerDeException

Deserialize a row from the HBase Result writable to a LazyObject

Specified by:

deserialize in interface Deserializer

Specified by:

deserialize in class AbstractSerDe

Parameters:

result - the HBase Result Writable containing the row

Returns:

the deserialized object

Throws:

SerDeException

See Also:

SerDe#deserialize(Writable)

getObjectInspector

public ObjectInspector getObjectInspector()
                                   throws SerDeException

Specified by:

getObjectInspector in interface Deserializer

Specified by:

getObjectInspector in class AbstractSerDe

Throws:

SerDeException

getSerializedClass

public Class<? extends org.apache.hadoop.io.Writable> getSerializedClass()

Specified by:

getSerializedClass in interface Serializer

Specified by:

getSerializedClass in class AbstractSerDe

serialize

public org.apache.hadoop.io.Writable serialize(Object obj,
                                               ObjectInspector objInspector)
                                        throws SerDeException

Specified by:

serialize in interface Serializer

Specified by:

serialize in class AbstractSerDe

Throws:

SerDeException

isUseJSONSerialize

public boolean isUseJSONSerialize()

Returns:

the useJSONSerialize

setUseJSONSerialize

public void setUseJSONSerialize(boolean useJSONSerialize)

Parameters:

useJSONSerialize - the useJSONSerialize to set

getSerDeStats

public SerDeStats getSerDeStats()

Specified by:

getSerDeStats in interface Deserializer

Specified by:

getSerDeStats in interface Serializer

Specified by:

getSerDeStats in class AbstractSerDe

getRowKeyColumnOffset

public static int getRowKeyColumnOffset(List<org.apache.hadoop.hive.hbase.HBaseSerDe.ColumnMapping> columnsMapping)
                                 throws SerDeException

Throws:

SerDeException