Package org.apache.pulsar.io.hdfs2.sink.seq

Class HdfsTextSink

java.lang.Object

org.apache.pulsar.io.hdfs2.AbstractHdfsConnector

org.apache.pulsar.io.hdfs2.sink.HdfsAbstractSink<K,V>

org.apache.pulsar.io.hdfs2.sink.seq.HdfsAbstractSequenceFileSink<java.lang.String,java.lang.String,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>

org.apache.pulsar.io.hdfs2.sink.seq.HdfsTextSink

All Implemented Interfaces:

java.lang.AutoCloseable, org.apache.pulsar.io.core.Sink<java.lang.String>

public class HdfsTextSink
extends HdfsAbstractSequenceFileSink<java.lang.String,java.lang.String,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>

A Simple Sink class for Hdfs Sequence File.

Field Summary

Fields inherited from class org.apache.pulsar.io.hdfs2.sink.seq.HdfsAbstractSequenceFileSink

counter, hdfsStream, writer

Fields inherited from class org.apache.pulsar.io.hdfs2.sink.HdfsAbstractSink

hdfsSinkConfig, syncThread, unackedRecords

Fields inherited from class org.apache.pulsar.io.hdfs2.AbstractHdfsConnector

compressionCodecFactory, connectorConfig, hdfsResources

Constructor Summary

Constructors

Constructor	Description
HdfsTextSink()

Method Summary

Modifier and Type	Method	Description
org.apache.pulsar.io.core.KeyValue<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>	convert(org.apache.pulsar.io.core.KeyValue<java.lang.String,java.lang.String> kv)
org.apache.pulsar.io.core.KeyValue<java.lang.String,java.lang.String>	extractKeyValue(org.apache.pulsar.functions.api.Record<java.lang.String> record)
protected java.util.List<org.apache.hadoop.io.SequenceFile.Writer.Option>	getOptions()

Methods inherited from class org.apache.pulsar.io.hdfs2.sink.seq.HdfsAbstractSequenceFileSink

close, createWriter, getWriter, write

Methods inherited from class org.apache.pulsar.io.hdfs2.sink.HdfsAbstractSink

connectToHdfs, getHdfsStream, getPath, launchSyncThread, open

Methods inherited from class org.apache.pulsar.io.hdfs2.AbstractHdfsConnector

checkHdfsUriForTimeout, getCompressionCodec, getCompressionCodecFactory, getConfiguration, getEncoding, getFileSystem, getFileSystem, getFileSystemAsUser, getUserGroupInformation, resetHDFSResources

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.pulsar.io.core.Sink

open

Constructor Detail

HdfsTextSink

public HdfsTextSink()

Method Detail

getOptions

protected java.util.List<org.apache.hadoop.io.SequenceFile.Writer.Option> getOptions()
                                                                              throws java.lang.IllegalArgumentException,
                                                                                     java.io.IOException

Overrides:

getOptions in class HdfsAbstractSequenceFileSink<java.lang.String,java.lang.String,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>

Throws:

java.lang.IllegalArgumentException

java.io.IOException

extractKeyValue

public org.apache.pulsar.io.core.KeyValue<java.lang.String,java.lang.String> extractKeyValue(org.apache.pulsar.functions.api.Record<java.lang.String> record)

Specified by:

extractKeyValue in class HdfsAbstractSink<java.lang.String,java.lang.String>

convert

public org.apache.pulsar.io.core.KeyValue<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text> convert(org.apache.pulsar.io.core.KeyValue<java.lang.String,java.lang.String> kv)

Specified by:

convert in class HdfsAbstractSequenceFileSink<java.lang.String,java.lang.String,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>