io.cdap.wrangler.service.s3

Class S3Handler

java.lang.Object

io.cdap.cdap.internal.api.AbstractProgramDatasetConfigurable<T>

io.cdap.cdap.internal.api.AbstractPluginConfigurable<V>

io.cdap.cdap.api.service.http.AbstractHttpServiceHandler<io.cdap.cdap.api.service.http.SystemHttpServiceContext,io.cdap.cdap.api.service.http.SystemHttpServiceConfigurer>

io.cdap.cdap.api.service.http.AbstractSystemHttpServiceHandler

io.cdap.wrangler.service.common.AbstractWranglerHandler

io.cdap.wrangler.service.s3.S3Handler

All Implemented Interfaces:

io.cdap.cdap.api.ProgramLifecycle<io.cdap.cdap.api.service.http.SystemHttpServiceContext>, io.cdap.cdap.api.service.http.HttpServiceHandler<io.cdap.cdap.api.service.http.SystemHttpServiceContext,io.cdap.cdap.api.service.http.SystemHttpServiceConfigurer>

public class S3Handler
extends AbstractWranglerHandler

Service to explore S3 filesystem.

Nested Class Summary

Nested classes/interfaces inherited from class io.cdap.wrangler.service.common.AbstractWranglerHandler

AbstractWranglerHandler.NamespacedResponder<T>

Constructor Summary

Constructors

Constructor and Description
S3Handler()

Method Summary

Modifier and Type	Method and Description
static S3ObjectInfo	fromDir(String dir)
static S3ObjectInfo	fromObject(com.amazonaws.services.s3.model.S3ObjectSummary summary, FileTypeDetector detector)
void	getS3BucketInfo(io.cdap.cdap.api.service.http.HttpServiceRequest request, io.cdap.cdap.api.service.http.HttpServiceResponder responder, String namespace, String connectionId, String path, int bucketLimit) Lists S3 bucket's contents for the given prefix path.
void	loadObject(io.cdap.cdap.api.service.http.HttpServiceRequest request, io.cdap.cdap.api.service.http.HttpServiceResponder responder, String namespace, String connectionId, String bucketName, String key, int lines, String sampler, double fraction, String scope) Reads s3 object into workspace
void	specification(io.cdap.cdap.api.service.http.HttpServiceRequest request, io.cdap.cdap.api.service.http.HttpServiceResponder responder, String namespace, String connectionId, String bucketName, String key, String workspaceId) Specification for the source.
void	testS3Connection(io.cdap.cdap.api.service.http.HttpServiceRequest request, io.cdap.cdap.api.service.http.HttpServiceResponder responder, String namespace)

Methods inherited from class io.cdap.wrangler.service.common.AbstractWranglerHandler

getConnection, getValidatedConnection, getValidatedConnection, getWorkspace, respond, respond, shouldCopyHeader

Methods inherited from class io.cdap.cdap.api.service.http.AbstractSystemHttpServiceHandler

createTable

Methods inherited from class io.cdap.cdap.api.service.http.AbstractHttpServiceHandler

configure, configure, destroy, getConfigurer, getContext, initialize, setProperties

Methods inherited from class io.cdap.cdap.internal.api.AbstractPluginConfigurable

usePlugin, usePlugin, usePluginClass, usePluginClass

Methods inherited from class io.cdap.cdap.internal.api.AbstractProgramDatasetConfigurable

addDatasetModule, addDatasetType, createDataset, createDataset, createDataset, createDataset

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

S3Handler

public S3Handler()

Method Detail

testS3Connection

@POST
 @Path(value="/contexts/{context}/connections/s3/test")
public void testS3Connection(io.cdap.cdap.api.service.http.HttpServiceRequest request,
                                                                                           io.cdap.cdap.api.service.http.HttpServiceResponder responder,
                                                                                           @PathParam(value="context")
                                                                                           String namespace)

getS3BucketInfo

@GET
 @Path(value="contexts/{context}/connections/{connection-id}/s3/explore")
public void getS3BucketInfo(io.cdap.cdap.api.service.http.HttpServiceRequest request,
                                                                                                           io.cdap.cdap.api.service.http.HttpServiceResponder responder,
                                                                                                           @PathParam(value="context")
                                                                                                           String namespace,
                                                                                                           @PathParam(value="connection-id")
                                                                                                           String connectionId,
                                                                                                           @QueryParam(value="path")
                                                                                                           String path,
                                                                                                           @QueryParam(value="limit") @DefaultValue(value="1000")
                                                                                                           int bucketLimit)

Lists S3 bucket's contents for the given prefix path.

Parameters:

request - HTTP Request handler.

responder - HTTP Response handler.

loadObject

@POST
 @Path(value="contexts/{context}/connections/{connection-id}/s3/buckets/{bucket-name}/read")
public void loadObject(io.cdap.cdap.api.service.http.HttpServiceRequest request,
                                                                                                                          io.cdap.cdap.api.service.http.HttpServiceResponder responder,
                                                                                                                          @PathParam(value="context")
                                                                                                                          String namespace,
                                                                                                                          @PathParam(value="connection-id")
                                                                                                                          String connectionId,
                                                                                                                          @PathParam(value="bucket-name")
                                                                                                                          String bucketName,
                                                                                                                          @QueryParam(value="key")
                                                                                                                          String key,
                                                                                                                          @QueryParam(value="lines")
                                                                                                                          int lines,
                                                                                                                          @QueryParam(value="sampler")
                                                                                                                          String sampler,
                                                                                                                          @QueryParam(value="fraction")
                                                                                                                          double fraction,
                                                                                                                          @QueryParam(value="scope") @DefaultValue(value="default")
                                                                                                                          String scope)

Reads s3 object into workspace

Parameters:

request - HTTP Request handler.

responder - HTTP Response handler.

specification

@GET
 @Path(value="contexts/{context}/connections/{connection-id}/s3/buckets/{bucket-name}/specification")
public void specification(io.cdap.cdap.api.service.http.HttpServiceRequest request,
                                                                                                                                     io.cdap.cdap.api.service.http.HttpServiceResponder responder,
                                                                                                                                     @PathParam(value="context")
                                                                                                                                     String namespace,
                                                                                                                                     @PathParam(value="connection-id")
                                                                                                                                     String connectionId,
                                                                                                                                     @PathParam(value="bucket-name")
                                                                                                                                     String bucketName,
                                                                                                                                     @QueryParam(value="key")
                                                                                                                                     String key,
                                                                                                                                     @QueryParam(value="wid")
                                                                                                                                     String workspaceId)

Specification for the source.

Parameters:

request - HTTP request handler.

responder - HTTP response handler.

bucketName - S3 object's bucket name

key - S3 object's key

fromDir

public static S3ObjectInfo fromDir(String dir)

fromObject

public static S3ObjectInfo fromObject(com.amazonaws.services.s3.model.S3ObjectSummary summary,
                                      FileTypeDetector detector)