Package org.openstreetmap.atlas.generator.tools.streaming.resource

Class HDFSFile

  • All Implemented Interfaces:
    java.lang.AutoCloseable, org.openstreetmap.atlas.streaming.resource.Resource, org.openstreetmap.atlas.streaming.resource.WritableResource
    public class HDFSFile
extends org.openstreetmap.atlas.streaming.resource.AbstractWritableResource
implements java.lang.AutoCloseable
    Readable and Writable resource for HDFS files

    • Field Summary

      • Fields inherited from interface org.openstreetmap.atlas.streaming.resource.Resource

        BYTE_MASK

    • Constructor Summary

      Constructors 
      Constructor Description
      HDFSFile​(java.lang.String path)  
      HDFSFile​(java.lang.String server, int port, java.lang.String path)  
      HDFSFile​(java.lang.String server, int port, java.lang.String path, org.apache.hadoop.conf.Configuration configuration)  
      HDFSFile​(java.lang.String path, org.apache.hadoop.conf.Configuration configuration)  
      HDFSFile​(org.apache.hadoop.fs.Path path)  
      HDFSFile​(org.apache.hadoop.fs.Path path, org.apache.hadoop.conf.Configuration configuration)  

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void close()  
      boolean exists()  
      org.apache.hadoop.fs.Path getPath()  
      int getRetries()  
      org.openstreetmap.atlas.utilities.scalars.Duration getRetryWait()  
      boolean isAppendToFile()  
      boolean isDirectory()  
      long length()  
      void mkdirs​(boolean clean)  
      protected java.io.InputStream onRead()  
      protected java.io.OutputStream onWrite()  
      void remove​(boolean recursive)
      CAUTION: Please use this wisely in case of removing directories recursively
      void setAppendToFile​(boolean appendToFile)  
      void setRetries​(int retries)  
      void setRetryWait​(org.openstreetmap.atlas.utilities.scalars.Duration retryWait)  

      • Methods inherited from class org.openstreetmap.atlas.streaming.resource.AbstractWritableResource

        getCompressor, setCompressor, write

      • Methods inherited from class org.openstreetmap.atlas.streaming.resource.AbstractResource

        getDecompressor, getName, read, setDecompressor, setName, toString

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

      • Methods inherited from interface org.openstreetmap.atlas.streaming.resource.Resource

        all, copyTo, copyTo, firstLine, getName, isGzipped, lines, linesList, read, readAndClose, readBytesAndClose, reader

      • Methods inherited from interface org.openstreetmap.atlas.streaming.resource.WritableResource

        copyFrom, copyFrom, writeAndClose, writeAndClose, writer

    • Constructor Detail

      • HDFSFile

        public HDFSFile​(org.apache.hadoop.fs.Path path)
         throws java.io.IOException
        Throws:
        java.io.IOException

      • HDFSFile

        public HDFSFile​(org.apache.hadoop.fs.Path path,
                org.apache.hadoop.conf.Configuration configuration)
         throws java.io.IOException
        Throws:
        java.io.IOException

      • HDFSFile

        public HDFSFile​(java.lang.String path)
         throws java.io.IOException
        Throws:
        java.io.IOException

      • HDFSFile

        public HDFSFile​(java.lang.String path,
                org.apache.hadoop.conf.Configuration configuration)
         throws java.io.IOException
        Throws:
        java.io.IOException

      • HDFSFile

        public HDFSFile​(java.lang.String server,
                int port,
                java.lang.String path)
         throws java.net.URISyntaxException,
                java.io.IOException
        Throws:
        java.net.URISyntaxException
        java.io.IOException

      • HDFSFile

        public HDFSFile​(java.lang.String server,
                int port,
                java.lang.String path,
                org.apache.hadoop.conf.Configuration configuration)
         throws java.net.URISyntaxException,
                java.io.IOException
        Throws:
        java.net.URISyntaxException
        java.io.IOException

    • Method Detail

      • close

        public void close()
           throws java.io.IOException
        Specified by:
        close in interface java.lang.AutoCloseable
        Throws:
        java.io.IOException

      • exists

        public boolean exists()

      • getPath

        public org.apache.hadoop.fs.Path getPath()

      • getRetries

        public int getRetries()

      • getRetryWait

        public org.openstreetmap.atlas.utilities.scalars.Duration getRetryWait()

      • isAppendToFile

        public boolean isAppendToFile()

      • isDirectory

        public boolean isDirectory()

      • length

        public long length()
        Specified by:
        length in interface org.openstreetmap.atlas.streaming.resource.Resource
        Overrides:
        length in class org.openstreetmap.atlas.streaming.resource.AbstractResource

      • mkdirs

        public void mkdirs​(boolean clean)

      • remove

        public void remove​(boolean recursive)
        CAUTION: Please use this wisely in case of removing directories recursively
        Parameters:
        recursive - true to recursively delete a directory

      • setAppendToFile

        public void setAppendToFile​(boolean appendToFile)

      • setRetries

        public void setRetries​(int retries)

      • setRetryWait

        public void setRetryWait​(org.openstreetmap.atlas.utilities.scalars.Duration retryWait)

      • onRead

        protected java.io.InputStream onRead()
        Specified by:
        onRead in class org.openstreetmap.atlas.streaming.resource.AbstractResource

      • onWrite

        protected java.io.OutputStream onWrite()
        Specified by:
        onWrite in class org.openstreetmap.atlas.streaming.resource.AbstractWritableResource