io.cdap.wrangler

Class Wrangler

java.lang.Object

io.cdap.cdap.etl.api.Transform<io.cdap.cdap.api.data.format.StructuredRecord,io.cdap.cdap.api.data.format.StructuredRecord>

io.cdap.wrangler.Wrangler

All Implemented Interfaces:

io.cdap.cdap.etl.api.Destroyable, io.cdap.cdap.etl.api.PipelineConfigurable, io.cdap.cdap.etl.api.StageLifecycle<io.cdap.cdap.etl.api.TransformContext>, io.cdap.cdap.etl.api.SubmitterLifecycle<io.cdap.cdap.etl.api.StageSubmitterContext>, io.cdap.cdap.etl.api.Transformation<io.cdap.cdap.api.data.format.StructuredRecord,io.cdap.cdap.api.data.format.StructuredRecord>

public class Wrangler
extends io.cdap.cdap.etl.api.Transform<io.cdap.cdap.api.data.format.StructuredRecord,io.cdap.cdap.api.data.format.StructuredRecord>

Wrangler - A interactive tool for data cleansing and transformation. This plugin is an implementation of the transformation that are performed in the backend for operationalizing all the interactive wrangling that is being performed by the user.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Wrangler.Config Config for the plugin.

Field Summary

Fields inherited from class io.cdap.cdap.etl.api.Transform

PLUGIN_TYPE

Constructor Summary

Constructors

Constructor and Description
Wrangler(Wrangler.Config config)

Method Summary

Modifier and Type	Method and Description
void	configurePipeline(io.cdap.cdap.etl.api.PipelineConfigurer configurer) Configures the plugin during deployment of the pipeline that uses the plugin.
void	destroy()
void	initialize(io.cdap.cdap.etl.api.TransformContext context) Initialize the wrangler by parsing the directives and creating the runtime context.
void	prepareRun(io.cdap.cdap.etl.api.StageSubmitterContext context) prepareRun is invoked by the client once before the job is submitted, but after the resolution of macros if there are any defined.
void	transform(io.cdap.cdap.api.data.format.StructuredRecord input, io.cdap.cdap.etl.api.Emitter<io.cdap.cdap.api.data.format.StructuredRecord> emitter) Transforms the input record by applying directives on the record being passed.

Methods inherited from class io.cdap.cdap.etl.api.Transform

getContext, onRunFinish

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

Wrangler

public Wrangler(Wrangler.Config config)

Method Detail

configurePipeline

public void configurePipeline(io.cdap.cdap.etl.api.PipelineConfigurer configurer)

Configures the plugin during deployment of the pipeline that uses the plugin.

• Parses the directives configured. If there are any issues they will highlighted during deployment
• Input schema is validated.
• Compiles pre-condition expression.

Specified by:

configurePipeline in interface io.cdap.cdap.etl.api.PipelineConfigurable

Overrides:

configurePipeline in class io.cdap.cdap.etl.api.Transform<io.cdap.cdap.api.data.format.StructuredRecord,io.cdap.cdap.api.data.format.StructuredRecord>

prepareRun

public void prepareRun(io.cdap.cdap.etl.api.StageSubmitterContext context)
                throws Exception

prepareRun is invoked by the client once before the job is submitted, but after the resolution of macros if there are any defined.

Specified by:

prepareRun in interface io.cdap.cdap.etl.api.SubmitterLifecycle<io.cdap.cdap.etl.api.StageSubmitterContext>

Overrides:

prepareRun in class io.cdap.cdap.etl.api.Transform<io.cdap.cdap.api.data.format.StructuredRecord,io.cdap.cdap.api.data.format.StructuredRecord>

Parameters:

context - a instance StageSubmitterContext

Throws:

Exception - thrown if there any issue with prepareRun.

initialize

public void initialize(io.cdap.cdap.etl.api.TransformContext context)
                throws Exception

Initialize the wrangler by parsing the directives and creating the runtime context.

Specified by:

initialize in interface io.cdap.cdap.etl.api.StageLifecycle<io.cdap.cdap.etl.api.TransformContext>

Overrides:

initialize in class io.cdap.cdap.etl.api.Transform<io.cdap.cdap.api.data.format.StructuredRecord,io.cdap.cdap.api.data.format.StructuredRecord>

Parameters:

context - framework context being passed.

Throws:

Exception

destroy

public void destroy()

Specified by:

destroy in interface io.cdap.cdap.etl.api.Destroyable

Overrides:

destroy in class io.cdap.cdap.etl.api.Transform<io.cdap.cdap.api.data.format.StructuredRecord,io.cdap.cdap.api.data.format.StructuredRecord>

transform

public void transform(io.cdap.cdap.api.data.format.StructuredRecord input,
                      io.cdap.cdap.etl.api.Emitter<io.cdap.cdap.api.data.format.StructuredRecord> emitter)
               throws Exception

Transforms the input record by applying directives on the record being passed.

Parameters:

input - record to be transformed.

emitter - to collect all the output of the transformation.

Throws:

Exception - thrown if there are any issue with the transformation.