Class FileSystemPerformanceHelper


  • public final class FileSystemPerformanceHelper
    extends java.lang.Object
    When working with a FileSystem that is slow at renaming, this provides a hack to speed up the rename of output files. The rename calls made by Spark are sent to many threads.
    • Method Detail

      • executeRename

        public static boolean executeRename​(java.util.concurrent.Callable<java.lang.Boolean> renameOperation)
      • openRenamePool

        public static void openRenamePool()
      • waitForAndCloseRenamePool

        public static void waitForAndCloseRenamePool()