Package org.apache.hadoop.hive.ql.io.orc

The Optimized Row Columnar (ORC) File Format.

See:
Description

Interface Summary

BinaryColumnStatistics	Statistics for binary columns.
BooleanColumnStatistics	Statistics for boolean columns.
ColumnStatistics	Statistics that are available for all types of columns.
DateColumnStatistics	Statistics for DATE columns.
DecimalColumnStatistics	Statistics for decimal columns.
DirectDecompressionCodec
DoubleColumnStatistics	Statistics for float and double columns.
IntegerColumnStatistics	Statistics for all of the integer columns, such as byte, short, int, and long.
OrcFile.WriterCallback
OrcFile.WriterContext
OrcProto.BinaryStatisticsOrBuilder
OrcProto.BucketStatisticsOrBuilder
OrcProto.ColumnEncodingOrBuilder
OrcProto.ColumnStatisticsOrBuilder
OrcProto.DateStatisticsOrBuilder
OrcProto.DecimalStatisticsOrBuilder
OrcProto.DoubleStatisticsOrBuilder
OrcProto.FooterOrBuilder
OrcProto.IntegerStatisticsOrBuilder
OrcProto.MetadataOrBuilder
OrcProto.PostScriptOrBuilder
OrcProto.RowIndexEntryOrBuilder
OrcProto.RowIndexOrBuilder
OrcProto.StreamOrBuilder
OrcProto.StringStatisticsOrBuilder
OrcProto.StripeFooterOrBuilder
OrcProto.StripeInformationOrBuilder
OrcProto.StripeStatisticsOrBuilder
OrcProto.TypeOrBuilder
OrcProto.UserMetadataItemOrBuilder
Reader	The interface for reading ORC files.
RecordReader	A row-by-row iterator for ORC files.
StringColumnStatistics	Statistics for string columns.
StripeInformation	Information about the stripes in an ORC file that is provided by the Reader.
Writer	The interface for writing ORC files.

Class Summary

FileDump	A tool for printing out the file structure of ORC files.
Metadata
OrcFile	Contains factory methods to read or write ORC files.
OrcFile.ReaderOptions
OrcFile.WriterOptions	Options for creating ORC file writers.
OrcInputFormat	A MapReduce/Hive input format for ORC files.
OrcNewInputFormat	An InputFormat for ORC files.
OrcNewOutputFormat	An OutputFormat that writes ORC files.
OrcNewSplit	OrcFileSplit.
OrcOutputFormat	A Hive OutputFormat for ORC files.
OrcProto
OrcProto.BinaryStatistics	Protobuf type org.apache.hadoop.hive.ql.io.orc.BinaryStatistics
OrcProto.BinaryStatistics.Builder	Protobuf type org.apache.hadoop.hive.ql.io.orc.BinaryStatistics
OrcProto.BucketStatistics	Protobuf type org.apache.hadoop.hive.ql.io.orc.BucketStatistics
OrcProto.BucketStatistics.Builder	Protobuf type org.apache.hadoop.hive.ql.io.orc.BucketStatistics
OrcProto.ColumnEncoding	Protobuf type org.apache.hadoop.hive.ql.io.orc.ColumnEncoding
OrcProto.ColumnEncoding.Builder	Protobuf type org.apache.hadoop.hive.ql.io.orc.ColumnEncoding
OrcProto.ColumnStatistics	Protobuf type org.apache.hadoop.hive.ql.io.orc.ColumnStatistics
OrcProto.ColumnStatistics.Builder	Protobuf type org.apache.hadoop.hive.ql.io.orc.ColumnStatistics
OrcProto.DateStatistics	Protobuf type org.apache.hadoop.hive.ql.io.orc.DateStatistics
OrcProto.DateStatistics.Builder	Protobuf type org.apache.hadoop.hive.ql.io.orc.DateStatistics
OrcProto.DecimalStatistics	Protobuf type org.apache.hadoop.hive.ql.io.orc.DecimalStatistics
OrcProto.DecimalStatistics.Builder	Protobuf type org.apache.hadoop.hive.ql.io.orc.DecimalStatistics
OrcProto.DoubleStatistics	Protobuf type org.apache.hadoop.hive.ql.io.orc.DoubleStatistics
OrcProto.DoubleStatistics.Builder	Protobuf type org.apache.hadoop.hive.ql.io.orc.DoubleStatistics
OrcProto.Footer	Protobuf type org.apache.hadoop.hive.ql.io.orc.Footer
OrcProto.Footer.Builder	Protobuf type org.apache.hadoop.hive.ql.io.orc.Footer
OrcProto.IntegerStatistics	Protobuf type org.apache.hadoop.hive.ql.io.orc.IntegerStatistics
OrcProto.IntegerStatistics.Builder	Protobuf type org.apache.hadoop.hive.ql.io.orc.IntegerStatistics
OrcProto.Metadata	Protobuf type org.apache.hadoop.hive.ql.io.orc.Metadata
OrcProto.Metadata.Builder	Protobuf type org.apache.hadoop.hive.ql.io.orc.Metadata
OrcProto.PostScript	Protobuf type org.apache.hadoop.hive.ql.io.orc.PostScript
OrcProto.PostScript.Builder	Protobuf type org.apache.hadoop.hive.ql.io.orc.PostScript
OrcProto.RowIndex	Protobuf type org.apache.hadoop.hive.ql.io.orc.RowIndex
OrcProto.RowIndex.Builder	Protobuf type org.apache.hadoop.hive.ql.io.orc.RowIndex
OrcProto.RowIndexEntry	Protobuf type org.apache.hadoop.hive.ql.io.orc.RowIndexEntry
OrcProto.RowIndexEntry.Builder	Protobuf type org.apache.hadoop.hive.ql.io.orc.RowIndexEntry
OrcProto.Stream	Protobuf type org.apache.hadoop.hive.ql.io.orc.Stream
OrcProto.Stream.Builder	Protobuf type org.apache.hadoop.hive.ql.io.orc.Stream
OrcProto.StringStatistics	Protobuf type org.apache.hadoop.hive.ql.io.orc.StringStatistics
OrcProto.StringStatistics.Builder	Protobuf type org.apache.hadoop.hive.ql.io.orc.StringStatistics
OrcProto.StripeFooter	Protobuf type org.apache.hadoop.hive.ql.io.orc.StripeFooter
OrcProto.StripeFooter.Builder	Protobuf type org.apache.hadoop.hive.ql.io.orc.StripeFooter
OrcProto.StripeInformation	Protobuf type org.apache.hadoop.hive.ql.io.orc.StripeInformation
OrcProto.StripeInformation.Builder	Protobuf type org.apache.hadoop.hive.ql.io.orc.StripeInformation
OrcProto.StripeStatistics	Protobuf type org.apache.hadoop.hive.ql.io.orc.StripeStatistics
OrcProto.StripeStatistics.Builder	Protobuf type org.apache.hadoop.hive.ql.io.orc.StripeStatistics
OrcProto.Type	Protobuf type org.apache.hadoop.hive.ql.io.orc.Type
OrcProto.Type.Builder	Protobuf type org.apache.hadoop.hive.ql.io.orc.Type
OrcProto.UserMetadataItem	Protobuf type org.apache.hadoop.hive.ql.io.orc.UserMetadataItem
OrcProto.UserMetadataItem.Builder	Protobuf type org.apache.hadoop.hive.ql.io.orc.UserMetadataItem
OrcRawRecordMerger	Merges a base and a list of delta files together into a single stream of events.
OrcRecordUpdater	A RecordUpdater where the files are stored as ORC.
OrcRecordUpdater.OrcOptions	An extension to AcidOutputFormat that allows users to add additional options.
OrcSerde	A serde class for ORC.
OrcSplit	OrcFileSplit.
OrcStruct
Reader.Options	Options for creating a RecordReader.
StripeStatistics
VectorizedOrcInputFormat	A MapReduce/Hive input format for ORC files.
VectorizedOrcSerde	A serde class for ORC.

Enum Summary

CompressionKind	An enumeration that lists the generic compression algorithms that can be applied to ORC files.
OrcFile.OrcTableProperties	Enum container for all orc table properties.
OrcFile.Version	Create a version number for the ORC file format, so that we can add non-forward compatible changes in the future.
OrcProto.ColumnEncoding.Kind	Protobuf enum org.apache.hadoop.hive.ql.io.orc.ColumnEncoding.Kind
OrcProto.CompressionKind	Protobuf enum org.apache.hadoop.hive.ql.io.orc.CompressionKind
OrcProto.Stream.Kind	Protobuf enum org.apache.hadoop.hive.ql.io.orc.Stream.Kind
OrcProto.Type.Kind	Protobuf enum org.apache.hadoop.hive.ql.io.orc.Type.Kind

Package org.apache.hadoop.hive.ql.io.orc Description

The Optimized Row Columnar (ORC) File Format. This format:

• Decomposes complex column types into primitives
• Uses type-specific encoders for each column
  • Dictionary encodings for low cardinality columns
  • Run length encoding of data
  • variable length encoding of integers
• Divides file into large stripes
• Each stripe includes light-weight indexes that enable the reader to skip large sets of rows that don't satisfy the filter condition
• A file footer that contains meta-information about file
  • Precise byte range for each stripe
  • Type information for the file
  • Any user meta-information
• Seek to row number is implemented to support secondary indexes
• Support for additional generic compression: LZO, SNAPPY, ZLIB.

Format:

 HEADER (3 bytes) "ORC"
 STRIPE (0 or more stripes)
 FILE-FOOTER
 POST SCRIPT
 PS LENGTH (1 byte)
 
 

Stripe:

 INDEX-STREAM (0 or more)
 DATA-STREAM (0 or more)
 STRIPE-FOOTER