org.apache.pulsar.io.hdfs3.sink.seq

Class HdfsTextSink

java.lang.Object

org.apache.pulsar.io.hdfs3.AbstractHdfsConnector

org.apache.pulsar.io.hdfs3.sink.HdfsAbstractSink<K,V>

org.apache.pulsar.io.hdfs3.sink.seq.HdfsAbstractSequenceFileSink<String,String,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>

org.apache.pulsar.io.hdfs3.sink.seq.HdfsTextSink

All Implemented Interfaces:

AutoCloseable, Sink<String>

public class HdfsTextSink
extends HdfsAbstractSequenceFileSink<String,String,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>

A Simple Sink class for Hdfs Sequence File.

Field Summary

Fields inherited from class org.apache.pulsar.io.hdfs3.sink.seq.HdfsAbstractSequenceFileSink

counter, hdfsStream, writer

Fields inherited from class org.apache.pulsar.io.hdfs3.sink.HdfsAbstractSink

hdfsSinkConfig, syncThread, unackedRecords

Fields inherited from class org.apache.pulsar.io.hdfs3.AbstractHdfsConnector

compressionCodecFactory, connectorConfig, hdfsResources

Constructor Summary

Constructors

Constructor and Description
HdfsTextSink()

Method Summary

Modifier and Type	Method and Description
KeyValue<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>	convert(KeyValue<String,String> kv)
KeyValue<String,String>	extractKeyValue(org.apache.pulsar.functions.api.Record<String> record)
protected List<org.apache.hadoop.io.SequenceFile.Writer.Option>	getOptions()

Methods inherited from class org.apache.pulsar.io.hdfs3.sink.seq.HdfsAbstractSequenceFileSink

close, createWriter, getWriter, write

Methods inherited from class org.apache.pulsar.io.hdfs3.sink.HdfsAbstractSink

connectToHdfs, getHdfsStream, getOutputStreamBuilder, getPath, launchSyncThread, open

Methods inherited from class org.apache.pulsar.io.hdfs3.AbstractHdfsConnector

checkHdfsUriForTimeout, getCompressionCodec, getCompressionCodecFactory, getConfiguration, getEncoding, getFileSystem, getFileSystem, getFileSystemAsUser, getUserGroupInformation, resetHDFSResources

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.pulsar.io.core.Sink

open

Constructor Detail

HdfsTextSink

public HdfsTextSink()

Method Detail

getOptions

protected List<org.apache.hadoop.io.SequenceFile.Writer.Option> getOptions()
                                                                    throws IllegalArgumentException,
                                                                           IOException

Overrides:

getOptions in class HdfsAbstractSequenceFileSink<String,String,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>

Throws:

IllegalArgumentException

IOException

extractKeyValue

public KeyValue<String,String> extractKeyValue(org.apache.pulsar.functions.api.Record<String> record)

Specified by:

extractKeyValue in class HdfsAbstractSink<String,String>

convert

public KeyValue<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text> convert(KeyValue<String,String> kv)

Specified by:

convert in class HdfsAbstractSequenceFileSink<String,String,org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>