Package org.openstreetmap.atlas.generator.tools.streaming.resource

Class HDFSFile

  • All Implemented Interfaces:
    org.openstreetmap.atlas.streaming.resource.Resource, org.openstreetmap.atlas.streaming.resource.WritableResource
    public class HDFSFile
extends org.openstreetmap.atlas.streaming.resource.AbstractWritableResource
    Readable and Writable resource for HDFS files

    • Field Summary

      • Fields inherited from interface org.openstreetmap.atlas.streaming.resource.Resource

        BYTE_MASK

    • Constructor Summary

      Constructors 
      Constructor Description
      HDFSFile​(java.lang.String path)  
      HDFSFile​(java.lang.String server, int port, java.lang.String path)  
      HDFSFile​(java.lang.String server, int port, java.lang.String path, org.apache.hadoop.conf.Configuration configuration)  
      HDFSFile​(java.lang.String path, org.apache.hadoop.conf.Configuration configuration)  
      HDFSFile​(org.apache.hadoop.fs.Path path)  
      HDFSFile​(org.apache.hadoop.fs.Path path, org.apache.hadoop.conf.Configuration configuration)  

    • Method Summary

      All Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      boolean exists()  
      org.apache.hadoop.fs.Path getPath()  
      int getRetries()  
      org.openstreetmap.atlas.utilities.scalars.Duration getRetryWait()  
      boolean isAppendToFile()  
      boolean isDirectory()  
      long length()  
      void mkdirs​(boolean clean)  
      protected java.io.InputStream onRead()  
      protected java.io.OutputStream onWrite()  
      void remove​(boolean recursive)
      CAUTION: Please use this wisely in case of removing directories recursively
      void setAppendToFile​(boolean appendToFile)  
      void setRetries​(int retries)  
      void setRetryWait​(org.openstreetmap.atlas.utilities.scalars.Duration retryWait)  

      • Methods inherited from class org.openstreetmap.atlas.streaming.resource.AbstractWritableResource

        getCompressor, setCompressor, write

      • Methods inherited from class org.openstreetmap.atlas.streaming.resource.AbstractResource

        getDecompressor, getName, read, setDecompressor, setName, toString

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

      • Methods inherited from interface org.openstreetmap.atlas.streaming.resource.Resource

        all, copyTo, copyTo, firstLine, getName, isGzipped, lines, linesList, read, readAndClose, readBytesAndClose, reader

      • Methods inherited from interface org.openstreetmap.atlas.streaming.resource.WritableResource

        copyFrom, copyFrom, writeAndClose, writeAndClose, writer

    • Constructor Detail

      • HDFSFile

        public HDFSFile​(org.apache.hadoop.fs.Path path)
         throws java.io.IOException
        Throws:
        java.io.IOException

      • HDFSFile

        public HDFSFile​(org.apache.hadoop.fs.Path path,
                org.apache.hadoop.conf.Configuration configuration)
         throws java.io.IOException
        Throws:
        java.io.IOException

      • HDFSFile

        public HDFSFile​(java.lang.String path)
         throws java.io.IOException
        Throws:
        java.io.IOException

      • HDFSFile

        public HDFSFile​(java.lang.String path,
                org.apache.hadoop.conf.Configuration configuration)
         throws java.io.IOException
        Throws:
        java.io.IOException

      • HDFSFile

        public HDFSFile​(java.lang.String server,
                int port,
                java.lang.String path)
         throws java.net.URISyntaxException,
                java.io.IOException
        Throws:
        java.net.URISyntaxException
        java.io.IOException

      • HDFSFile

        public HDFSFile​(java.lang.String server,
                int port,
                java.lang.String path,
                org.apache.hadoop.conf.Configuration configuration)
         throws java.net.URISyntaxException,
                java.io.IOException
        Throws:
        java.net.URISyntaxException
        java.io.IOException

    • Method Detail

      • exists

        public boolean exists()

      • getPath

        public org.apache.hadoop.fs.Path getPath()

      • getRetries

        public int getRetries()

      • getRetryWait

        public org.openstreetmap.atlas.utilities.scalars.Duration getRetryWait()

      • isAppendToFile

        public boolean isAppendToFile()

      • isDirectory

        public boolean isDirectory()

      • length

        public long length()
        Specified by:
        length in interface org.openstreetmap.atlas.streaming.resource.Resource
        Overrides:
        length in class org.openstreetmap.atlas.streaming.resource.AbstractResource

      • mkdirs

        public void mkdirs​(boolean clean)

      • remove

        public void remove​(boolean recursive)
        CAUTION: Please use this wisely in case of removing directories recursively
        Parameters:
        recursive - true to recursively delete a directory

      • setAppendToFile

        public void setAppendToFile​(boolean appendToFile)

      • setRetries

        public void setRetries​(int retries)

      • setRetryWait

        public void setRetryWait​(org.openstreetmap.atlas.utilities.scalars.Duration retryWait)

      • onRead

        protected java.io.InputStream onRead()
        Specified by:
        onRead in class org.openstreetmap.atlas.streaming.resource.AbstractResource

      • onWrite

        protected java.io.OutputStream onWrite()
        Specified by:
        onWrite in class org.openstreetmap.atlas.streaming.resource.AbstractWritableResource