Serialized Form

  • Package org.apache.pulsar.io.hdfs3

    • Class org.apache.pulsar.io.hdfs3.AbstractHdfsConfig

      class AbstractHdfsConfig extends Object implements Serializable
      serialVersionUID:
      1L

      • Serialized Fields

        • compression
          Compression compression
          The compression codec used to compress/de-compress the files on HDFS.
        • directory
          String directory
          The HDFS directory from which files should be read from or written to.
        • encoding
          String encoding
          The character encoding for the files, e.g. UTF-8, ASCII, etc.
        • hdfsConfigResources
          String hdfsConfigResources
          A file or comma separated list of files which contains the Hadoop file system configuration, e.g. 'core-site.xml', 'hdfs-site.xml'.
        • kerberosUserPrincipal
          String kerberosUserPrincipal
          The Kerberos user principal account to use for authentication.
        • keytab
          String keytab
          The full pathname to the Kerberos keytab file to use for authentication.

  • Package org.apache.pulsar.io.hdfs3.sink

    • Class org.apache.pulsar.io.hdfs3.sink.HdfsSinkConfig

      class HdfsSinkConfig extends AbstractHdfsConfig implements Serializable
      serialVersionUID:
      1L

      • Serialized Fields

        • fileExtension
          String fileExtension
          The extension to add to the files written to HDFS, e.g. '.txt', '.seq', etc.
        • filenamePrefix
          String filenamePrefix
          The prefix of the files to create inside the HDFS directory, i.e. a value of "topicA" will result in files named topicA-, topicA-, etc being produced
        • maxPendingRecords
          int maxPendingRecords
          The maximum number of records that we hold in memory before acking. Default is Integer.MAX_VALUE. Setting this value to one, results in every record being sent to disk before the record is acked, while setting it to a higher values allows us to buffer records before flushing them all to disk.
        • separator
          char separator
          The character to use to separate records in a text file. If no value is provided then the content from all of the records will be concatenated together in one continuous byte array.
        • syncInterval
          long syncInterval
          The interval (in milliseconds) between calls to flush data to HDFS disk.