Package org.openstreetmap.atlas.generator.tools.streaming

Class ResourceFileSystem

  • All Implemented Interfaces:
    java.io.Closeable, java.lang.AutoCloseable, org.apache.hadoop.conf.Configurable
    public class ResourceFileSystem
extends org.apache.hadoop.fs.FileSystem
    FileSystem that is based on a set of resources.

    • Nested Class Summary

      • Nested classes/interfaces inherited from class org.apache.hadoop.fs.FileSystem

        org.apache.hadoop.fs.FileSystem.Statistics

    • Field Summary

      Fields 
      Modifier and Type Field Description
      static java.lang.String RESOURCE_FILE_SYSTEM_CONFIGURATION  
      static java.lang.String SCHEME  

      • Fields inherited from class org.apache.hadoop.fs.FileSystem

        DEFAULT_FS, FS_DEFAULT_NAME_KEY, LOG, SHUTDOWN_HOOK_PRIORITY, statistics

    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      static void addResource​(java.lang.String path, java.lang.String name)  
      static void addResource​(java.lang.String path, java.lang.String name, boolean gzipIt)  
      static void addResource​(java.lang.String path, java.lang.String name, boolean gzipIt, java.lang.Class<?> clazz)  
      static void addResource​(java.lang.String name, org.openstreetmap.atlas.streaming.resource.Resource resource)  
      static void addResourceContents​(java.lang.String path, java.lang.String contents)  
      org.apache.hadoop.fs.FSDataOutputStream append​(org.apache.hadoop.fs.Path hadoopPath, int bufferSize, org.apache.hadoop.util.Progressable progress)  
      static void clear()  
      static org.apache.spark.SparkConf configuredConf()  
      static org.apache.hadoop.conf.Configuration configuredConfiguration()  
      org.apache.hadoop.fs.FSDataOutputStream create​(org.apache.hadoop.fs.Path hadoopPath, org.apache.hadoop.fs.permission.FsPermission permission, boolean overwrite, int bufferSize, short replication, long blockSize, org.apache.hadoop.util.Progressable progress)  
      boolean delete​(org.apache.hadoop.fs.Path hadoopPath, boolean recursive)  
      static void dumpToDisk​(org.openstreetmap.atlas.streaming.resource.File folder)  
      static java.util.Set<java.lang.String> files()  
      org.apache.hadoop.fs.FileStatus getFileStatus​(org.apache.hadoop.fs.Path hadoopPath)  
      java.net.URI getUri()  
      org.apache.hadoop.fs.Path getWorkingDirectory()  
      void initialize​(java.net.URI uri, org.apache.hadoop.conf.Configuration conf)  
      org.apache.hadoop.fs.FileStatus[] listStatus​(org.apache.hadoop.fs.Path hadoopPath)  
      boolean mkdirs​(org.apache.hadoop.fs.Path hadoopPath, org.apache.hadoop.fs.permission.FsPermission permission)  
      org.apache.hadoop.fs.FSDataInputStream open​(org.apache.hadoop.fs.Path hadoopPath, int bufferSize)  
      static void printContents()  
      static void registerResourceExtractionClass​(java.lang.Class<?> clazz)  
      boolean rename​(org.apache.hadoop.fs.Path source, org.apache.hadoop.fs.Path destination)  
      void setWorkingDirectory​(org.apache.hadoop.fs.Path newDirectory)  
      static java.util.Map<java.lang.String,​java.lang.String> simpleconfiguration()  

      • Methods inherited from class org.apache.hadoop.fs.FileSystem

        access, addDelegationTokens, append, append, areSymlinksEnabled, cancelDeleteOnExit, canonicalizeUri, checkPath, clearStatistics, close, closeAll, closeAllForUGI, completeLocalOutput, concat, copyFromLocalFile, copyFromLocalFile, copyFromLocalFile, copyFromLocalFile, copyToLocalFile, copyToLocalFile, copyToLocalFile, create, create, create, create, create, create, create, create, create, create, create, create, createNewFile, createNonRecursive, createNonRecursive, createNonRecursive, createSnapshot, createSnapshot, createSymlink, delete, deleteOnExit, deleteSnapshot, enableSymlinks, exists, fixRelativePart, get, get, get, getAclStatus, getAllStatistics, getBlockSize, getCanonicalServiceName, getCanonicalUri, getChildFileSystems, getContentSummary, getDefaultBlockSize, getDefaultBlockSize, getDefaultPort, getDefaultReplication, getDefaultReplication, getDefaultUri, getDelegationToken, getFileBlockLocations, getFileBlockLocations, getFileChecksum, getFileChecksum, getFileLinkStatus, getFileSystemClass, getFSofPath, getHomeDirectory, getInitialWorkingDirectory, getLength, getLinkTarget, getLocal, getName, getNamed, getReplication, getScheme, getServerDefaults, getServerDefaults, getStatistics, getStatistics, getStatus, getStatus, getUsed, getXAttr, getXAttrs, getXAttrs, globStatus, globStatus, isDirectory, isFile, listCorruptFileBlocks, listFiles, listLocatedStatus, listLocatedStatus, listStatus, listStatus, listStatus, listXAttrs, makeQualified, mkdirs, mkdirs, modifyAclEntries, moveFromLocalFile, moveFromLocalFile, moveToLocalFile, newInstance, newInstance, newInstance, newInstanceLocal, open, primitiveCreate, primitiveMkdir, primitiveMkdir, printStatistics, processDeleteOnExit, removeAcl, removeAclEntries, removeDefaultAcl, removeXAttr, rename, renameSnapshot, resolveLink, resolvePath, setAcl, setDefaultUri, setDefaultUri, setOwner, setPermission, setReplication, setTimes, setVerifyChecksum, setWriteChecksum, setXAttr, setXAttr, startLocalOutput, supportsSymlinks

      • Methods inherited from class org.apache.hadoop.conf.Configured

        getConf, setConf

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Field Detail

      • RESOURCE_FILE_SYSTEM_CONFIGURATION

        public static final java.lang.String RESOURCE_FILE_SYSTEM_CONFIGURATION
        See Also:
        Constant Field Values

    • Constructor Detail

      • ResourceFileSystem

        public ResourceFileSystem()

    • Method Detail

      • addResource

        public static void addResource​(java.lang.String path,
                               java.lang.String name,
                               boolean gzipIt,
                               java.lang.Class<?> clazz)

      • addResource

        public static void addResource​(java.lang.String path,
                               java.lang.String name,
                               boolean gzipIt)

      • addResource

        public static void addResource​(java.lang.String name,
                               org.openstreetmap.atlas.streaming.resource.Resource resource)

      • addResource

        public static void addResource​(java.lang.String path,
                               java.lang.String name)

      • addResourceContents

        public static void addResourceContents​(java.lang.String path,
                                       java.lang.String contents)

      • clear

        public static void clear()

      • configuredConf

        public static org.apache.spark.SparkConf configuredConf()

      • configuredConfiguration

        public static org.apache.hadoop.conf.Configuration configuredConfiguration()

      • dumpToDisk

        public static void dumpToDisk​(org.openstreetmap.atlas.streaming.resource.File folder)

      • files

        public static java.util.Set<java.lang.String> files()

      • printContents

        public static void printContents()

      • registerResourceExtractionClass

        public static void registerResourceExtractionClass​(java.lang.Class<?> clazz)

      • simpleconfiguration

        public static java.util.Map<java.lang.String,​java.lang.String> simpleconfiguration()

      • append

        public org.apache.hadoop.fs.FSDataOutputStream append​(org.apache.hadoop.fs.Path hadoopPath,
                                                      int bufferSize,
                                                      org.apache.hadoop.util.Progressable progress)
                                               throws java.io.IOException
        Specified by:
        append in class org.apache.hadoop.fs.FileSystem
        Throws:
        java.io.IOException

      • create

        public org.apache.hadoop.fs.FSDataOutputStream create​(org.apache.hadoop.fs.Path hadoopPath,
                                                      org.apache.hadoop.fs.permission.FsPermission permission,
                                                      boolean overwrite,
                                                      int bufferSize,
                                                      short replication,
                                                      long blockSize,
                                                      org.apache.hadoop.util.Progressable progress)
                                               throws java.io.IOException
        Specified by:
        create in class org.apache.hadoop.fs.FileSystem
        Throws:
        java.io.IOException

      • delete

        public boolean delete​(org.apache.hadoop.fs.Path hadoopPath,
                      boolean recursive)
               throws java.io.IOException
        Specified by:
        delete in class org.apache.hadoop.fs.FileSystem
        Throws:
        java.io.IOException

      • getFileStatus

        public org.apache.hadoop.fs.FileStatus getFileStatus​(org.apache.hadoop.fs.Path hadoopPath)
                                              throws java.io.IOException
        Specified by:
        getFileStatus in class org.apache.hadoop.fs.FileSystem
        Throws:
        java.io.IOException

      • getUri

        public java.net.URI getUri()
        Specified by:
        getUri in class org.apache.hadoop.fs.FileSystem

      • getWorkingDirectory

        public org.apache.hadoop.fs.Path getWorkingDirectory()
        Specified by:
        getWorkingDirectory in class org.apache.hadoop.fs.FileSystem

      • initialize

        public void initialize​(java.net.URI uri,
                       org.apache.hadoop.conf.Configuration conf)
                throws java.io.IOException
        Overrides:
        initialize in class org.apache.hadoop.fs.FileSystem
        Throws:
        java.io.IOException

      • listStatus

        public org.apache.hadoop.fs.FileStatus[] listStatus​(org.apache.hadoop.fs.Path hadoopPath)
                                             throws java.io.FileNotFoundException,
                                                    java.io.IOException
        Specified by:
        listStatus in class org.apache.hadoop.fs.FileSystem
        Throws:
        java.io.FileNotFoundException
        java.io.IOException

      • mkdirs

        public boolean mkdirs​(org.apache.hadoop.fs.Path hadoopPath,
                      org.apache.hadoop.fs.permission.FsPermission permission)
               throws java.io.IOException
        Specified by:
        mkdirs in class org.apache.hadoop.fs.FileSystem
        Throws:
        java.io.IOException

      • open

        public org.apache.hadoop.fs.FSDataInputStream open​(org.apache.hadoop.fs.Path hadoopPath,
                                                   int bufferSize)
                                            throws java.io.IOException
        Specified by:
        open in class org.apache.hadoop.fs.FileSystem
        Throws:
        java.io.IOException

      • rename

        public boolean rename​(org.apache.hadoop.fs.Path source,
                      org.apache.hadoop.fs.Path destination)
               throws java.io.IOException
        Specified by:
        rename in class org.apache.hadoop.fs.FileSystem
        Throws:
        java.io.IOException

      • setWorkingDirectory

        public void setWorkingDirectory​(org.apache.hadoop.fs.Path newDirectory)
        Specified by:
        setWorkingDirectory in class org.apache.hadoop.fs.FileSystem