alluxio.hadoop

Class HadoopUtils

java.lang.Object

alluxio.hadoop.HadoopUtils

@ThreadSafe
public final class HadoopUtils
extends java.lang.Object

Utility class for using Alluxio with Hadoop.

Method Summary

Modifier and Type	Method and Description
static void	addSwiftCredentials(org.apache.hadoop.conf.Configuration configuration) Adds Swift keys to the given Hadoop Configuration object if the user has specified them using System properties, and they're not already set.
static java.lang.String	getAlluxioFileName(java.lang.String path) Given a String path, returns an equivalent Alluxio path.
static java.lang.String	getPathWithoutScheme(org.apache.hadoop.fs.Path path) Given a Path path, it returns the path component of its URI, which has the form scheme://authority/path.
static AlluxioConfiguration	toAlluxioConf(org.apache.hadoop.conf.Configuration conf)
static org.apache.hadoop.fs.Path	toPath(AlluxioURI alluxioURI)
static java.lang.String	toStringHadoopFileSplit(org.apache.hadoop.mapred.FileSplit fs) Returns a string representation of a Hadoop FileSplit.
static java.lang.String	toStringHadoopFileStatus(org.apache.hadoop.fs.FileStatus fs) Returns a string representation of a Hadoop FileStatus.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

getPathWithoutScheme

public static java.lang.String getPathWithoutScheme(org.apache.hadoop.fs.Path path)

Given a Path path, it returns the path component of its URI, which has the form scheme://authority/path.

Parameters:

path - an HDFS Path

Returns:

the path component of the Path URI

getAlluxioFileName

public static java.lang.String getAlluxioFileName(java.lang.String path)

Given a String path, returns an equivalent Alluxio path.

Parameters:

path - the path to parse

Returns:

a valid Alluxio path

toStringHadoopFileSplit

public static java.lang.String toStringHadoopFileSplit(org.apache.hadoop.mapred.FileSplit fs)

Returns a string representation of a Hadoop FileSplit.

Parameters:

fs - Hadoop FileSplit

Returns:

its string representation

toStringHadoopFileStatus

public static java.lang.String toStringHadoopFileStatus(org.apache.hadoop.fs.FileStatus fs)

Returns a string representation of a Hadoop FileStatus.

Parameters:

fs - Hadoop FileStatus

Returns:

its string representation

addSwiftCredentials

public static void addSwiftCredentials(org.apache.hadoop.conf.Configuration configuration)

Adds Swift keys to the given Hadoop Configuration object if the user has specified them using System properties, and they're not already set. This function is duplicated from alluxio.underfs.hdfs.HdfsUnderFileSystemUtils, to prevent the module alluxio-core-client-hdfs from depending on the module alluxio-underfs.

Parameters:

configuration - Hadoop configuration

toPath

public static org.apache.hadoop.fs.Path toPath(AlluxioURI alluxioURI)

Parameters:

alluxioURI - Alluxio uri

Returns:

corresponding Hadoop Path instance

toAlluxioConf

public static AlluxioConfiguration toAlluxioConf(org.apache.hadoop.conf.Configuration conf)

Parameters:

conf - Hadoop conf

Returns:

Alluxio configuration merged from Hadoop conf