A C E G H I K L O R S U V W

A

AbstractHdfsConfig - Class in org.apache.pulsar.io.hdfs2

Configuration object for all HDFS components.

AbstractHdfsConfig() - Constructor for class org.apache.pulsar.io.hdfs2.AbstractHdfsConfig

AbstractHdfsConnector - Class in org.apache.pulsar.io.hdfs2

A Simple abstract class for HDFS connectors.

AbstractHdfsConnector() - Constructor for class org.apache.pulsar.io.hdfs2.AbstractHdfsConnector

C

checkHdfsUriForTimeout(Configuration) - Method in class org.apache.pulsar.io.hdfs2.AbstractHdfsConnector

close() - Method in class org.apache.pulsar.io.hdfs2.sink.HdfsAbstractSink

close() - Method in class org.apache.pulsar.io.hdfs2.sink.seq.HdfsAbstractSequenceFileSink

close() - Method in class org.apache.pulsar.io.hdfs2.sink.text.HdfsAbstractTextFileSink

Compression - Enum in org.apache.pulsar.io.hdfs2

An enumeration of compression codecs available for HDFS.

compressionCodecFactory - Variable in class org.apache.pulsar.io.hdfs2.AbstractHdfsConnector

connectorConfig - Variable in class org.apache.pulsar.io.hdfs2.AbstractHdfsConnector

connectToHdfs() - Method in class org.apache.pulsar.io.hdfs2.sink.HdfsAbstractSink

convert(KeyValue<K, V>) - Method in class org.apache.pulsar.io.hdfs2.sink.seq.HdfsAbstractSequenceFileSink

convert(KeyValue<Long, String>) - Method in class org.apache.pulsar.io.hdfs2.sink.seq.HdfsSequentialTextSink

convert(KeyValue<String, String>) - Method in class org.apache.pulsar.io.hdfs2.sink.seq.HdfsTextSink

counter - Variable in class org.apache.pulsar.io.hdfs2.sink.seq.HdfsAbstractSequenceFileSink

createWriter() - Method in class org.apache.pulsar.io.hdfs2.sink.HdfsAbstractSink

createWriter() - Method in class org.apache.pulsar.io.hdfs2.sink.seq.HdfsAbstractSequenceFileSink

createWriter() - Method in class org.apache.pulsar.io.hdfs2.sink.text.HdfsAbstractTextFileSink

E

extractKeyValue(Record<V>) - Method in class org.apache.pulsar.io.hdfs2.sink.HdfsAbstractSink

extractKeyValue(Record<String>) - Method in class org.apache.pulsar.io.hdfs2.sink.seq.HdfsSequentialTextSink

extractKeyValue(Record<String>) - Method in class org.apache.pulsar.io.hdfs2.sink.seq.HdfsTextSink

extractKeyValue(Record<String>) - Method in class org.apache.pulsar.io.hdfs2.sink.text.HdfsStringSink

G

getCompressionCodec() - Method in class org.apache.pulsar.io.hdfs2.AbstractHdfsConnector

getCompressionCodecFactory() - Method in class org.apache.pulsar.io.hdfs2.AbstractHdfsConnector

getConfiguration() - Method in class org.apache.pulsar.io.hdfs2.AbstractHdfsConnector

getConfiguration() - Method in class org.apache.pulsar.io.hdfs2.HdfsResources

getEncoding() - Method in class org.apache.pulsar.io.hdfs2.AbstractHdfsConnector

getFileSystem(Configuration) - Method in class org.apache.pulsar.io.hdfs2.AbstractHdfsConnector

This exists in order to allow unit tests to override it so that they don't take several minutes waiting for UDP packets to be received.

getFileSystem() - Method in class org.apache.pulsar.io.hdfs2.AbstractHdfsConnector

getFileSystem() - Method in class org.apache.pulsar.io.hdfs2.HdfsResources

getFileSystemAsUser(Configuration, UserGroupInformation) - Method in class org.apache.pulsar.io.hdfs2.AbstractHdfsConnector

getHdfsStream() - Method in class org.apache.pulsar.io.hdfs2.sink.HdfsAbstractSink

getOptions() - Method in class org.apache.pulsar.io.hdfs2.sink.seq.HdfsAbstractSequenceFileSink

getOptions() - Method in class org.apache.pulsar.io.hdfs2.sink.seq.HdfsSequentialTextSink

getOptions() - Method in class org.apache.pulsar.io.hdfs2.sink.seq.HdfsTextSink

getPath() - Method in class org.apache.pulsar.io.hdfs2.sink.HdfsAbstractSink

getUserGroupInformation() - Method in class org.apache.pulsar.io.hdfs2.AbstractHdfsConnector

getUserGroupInformation() - Method in class org.apache.pulsar.io.hdfs2.HdfsResources

getWriter() - Method in class org.apache.pulsar.io.hdfs2.sink.seq.HdfsAbstractSequenceFileSink

getWriter() - Method in class org.apache.pulsar.io.hdfs2.sink.seq.HdfsSequentialTextSink

H

HADOOP_SECURITY_AUTHENTICATION - Static variable in class org.apache.pulsar.io.hdfs2.SecurityUtil

halt() - Method in class org.apache.pulsar.io.hdfs2.sink.HdfsSyncThread

HdfsAbstractSequenceFileSink<K,V,HdfsK,HdfsV> - Class in org.apache.pulsar.io.hdfs2.sink.seq

HDFS Sink that writes it contents to HDFS as Sequence Files.

HdfsAbstractSequenceFileSink() - Constructor for class org.apache.pulsar.io.hdfs2.sink.seq.HdfsAbstractSequenceFileSink

HdfsAbstractSink<K,V> - Class in org.apache.pulsar.io.hdfs2.sink

A Simple abstract class for HDFS sink.

HdfsAbstractSink() - Constructor for class org.apache.pulsar.io.hdfs2.sink.HdfsAbstractSink

HdfsAbstractTextFileSink<K,V> - Class in org.apache.pulsar.io.hdfs2.sink.text

Base class for HDFS Sinks that writes there contents to HDFS as Text Files.

HdfsAbstractTextFileSink() - Constructor for class org.apache.pulsar.io.hdfs2.sink.text.HdfsAbstractTextFileSink

hdfsResources - Variable in class org.apache.pulsar.io.hdfs2.AbstractHdfsConnector

HdfsResources - Class in org.apache.pulsar.io.hdfs2

A wrapper class for HDFS resources.

HdfsResources(Configuration, FileSystem, UserGroupInformation) - Constructor for class org.apache.pulsar.io.hdfs2.HdfsResources

HdfsSequentialTextSink - Class in org.apache.pulsar.io.hdfs2.sink.seq

This Sink should be used when the records are originating from a sequential source, and we want to retain the record sequence.This class uses the record's sequence id as the sequence id in the HDFS Sequence File if it is available, if not a sequence id is auto-generated for each new record.

HdfsSequentialTextSink() - Constructor for class org.apache.pulsar.io.hdfs2.sink.seq.HdfsSequentialTextSink

hdfsSinkConfig - Variable in class org.apache.pulsar.io.hdfs2.sink.HdfsAbstractSink

HdfsSinkConfig - Class in org.apache.pulsar.io.hdfs2.sink

Configuration object for all HDFS Sink components.

HdfsSinkConfig() - Constructor for class org.apache.pulsar.io.hdfs2.sink.HdfsSinkConfig

hdfsStream - Variable in class org.apache.pulsar.io.hdfs2.sink.seq.HdfsAbstractSequenceFileSink

HdfsStringSink - Class in org.apache.pulsar.io.hdfs2.sink.text

A Simple Sink class for Hdfs Text File.

HdfsStringSink() - Constructor for class org.apache.pulsar.io.hdfs2.sink.text.HdfsStringSink

HdfsSyncThread<V> - Class in org.apache.pulsar.io.hdfs2.sink

A thread that runs in the background and acknowledges Records after they have been written to disk.

HdfsSyncThread(Syncable, BlockingQueue<Record<V>>, long) - Constructor for class org.apache.pulsar.io.hdfs2.sink.HdfsSyncThread

HdfsTextSink - Class in org.apache.pulsar.io.hdfs2.sink.seq

A Simple Sink class for Hdfs Sequence File.

HdfsTextSink() - Constructor for class org.apache.pulsar.io.hdfs2.sink.seq.HdfsTextSink

I

isSecurityEnabled(Configuration) - Static method in class org.apache.pulsar.io.hdfs2.SecurityUtil

Initializes UserGroupInformation with the given Configuration and returns UserGroupInformation.isSecurityEnabled().

K

KERBEROS - Static variable in class org.apache.pulsar.io.hdfs2.SecurityUtil

L

launchSyncThread() - Method in class org.apache.pulsar.io.hdfs2.sink.HdfsAbstractSink

load(String) - Static method in class org.apache.pulsar.io.hdfs2.sink.HdfsSinkConfig

load(Map<String, Object>) - Static method in class org.apache.pulsar.io.hdfs2.sink.HdfsSinkConfig

loginKerberos(Configuration, String, String) - Static method in class org.apache.pulsar.io.hdfs2.SecurityUtil

Initializes UserGroupInformation with the given Configuration and performs the login for the given principal and keytab.

loginSimple(Configuration) - Static method in class org.apache.pulsar.io.hdfs2.SecurityUtil

Initializes UserGroupInformation with the given Configuration and returns UserGroupInformation.getLoginUser().

O

open(Map<String, Object>, SinkContext) - Method in class org.apache.pulsar.io.hdfs2.sink.HdfsAbstractSink

org.apache.pulsar.io.hdfs2 - package org.apache.pulsar.io.hdfs2

Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements.

org.apache.pulsar.io.hdfs2.sink - package org.apache.pulsar.io.hdfs2.sink

Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements.

org.apache.pulsar.io.hdfs2.sink.seq - package org.apache.pulsar.io.hdfs2.sink.seq

Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements.

org.apache.pulsar.io.hdfs2.sink.text - package org.apache.pulsar.io.hdfs2.sink.text

Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements.

R

resetHDFSResources(HdfsSinkConfig) - Method in class org.apache.pulsar.io.hdfs2.AbstractHdfsConnector

run() - Method in class org.apache.pulsar.io.hdfs2.sink.HdfsSyncThread

S

SecurityUtil - Class in org.apache.pulsar.io.hdfs2

Provides synchronized access to UserGroupInformation to avoid multiple processors/services from interfering with each other.

SecurityUtil() - Constructor for class org.apache.pulsar.io.hdfs2.SecurityUtil

syncThread - Variable in class org.apache.pulsar.io.hdfs2.sink.HdfsAbstractSink

U

unackedRecords - Variable in class org.apache.pulsar.io.hdfs2.sink.HdfsAbstractSink

V

validate() - Method in class org.apache.pulsar.io.hdfs2.AbstractHdfsConfig

validate() - Method in class org.apache.pulsar.io.hdfs2.sink.HdfsSinkConfig

valueOf(String) - Static method in enum org.apache.pulsar.io.hdfs2.Compression

Returns the enum constant of this type with the specified name.

values() - Static method in enum org.apache.pulsar.io.hdfs2.Compression

Returns an array containing the constants of this enum type, in the order they are declared.

W

write(Record<V>) - Method in class org.apache.pulsar.io.hdfs2.sink.seq.HdfsAbstractSequenceFileSink

write(Record<V>) - Method in class org.apache.pulsar.io.hdfs2.sink.text.HdfsAbstractTextFileSink

writer - Variable in class org.apache.pulsar.io.hdfs2.sink.seq.HdfsAbstractSequenceFileSink

writer - Variable in class org.apache.pulsar.io.hdfs2.sink.text.HdfsAbstractTextFileSink

A C E G H I K L O R S U V W