io.cdap.wrangler

Class Wrangler

java.lang.Object

io.cdap.cdap.etl.api.Transform<io.cdap.cdap.api.data.format.StructuredRecord,io.cdap.cdap.api.data.format.StructuredRecord>

io.cdap.wrangler.Wrangler

All Implemented Interfaces:

Destroyable, PipelineConfigurable, LinearRelationalTransform, RelationalTransform, StageLifecycle<TransformContext>, SubmitterLifecycle<StageSubmitterContext>, Transformation<io.cdap.cdap.api.data.format.StructuredRecord,io.cdap.cdap.api.data.format.StructuredRecord>

public class Wrangler
extends Transform<io.cdap.cdap.api.data.format.StructuredRecord,io.cdap.cdap.api.data.format.StructuredRecord>
implements LinearRelationalTransform

Wrangler - A interactive tool for data cleansing and transformation. This plugin is an implementation of the transformation that are performed in the backend for operationalizing all the interactive wrangling that is being performed by the user.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Wrangler.Config Config for the plugin.

Field Summary

Fields

Modifier and Type	Field and Description
static String	DIRECTIVE_ENTITY_TYPE
static int	DIRECTIVE_METRIC_COUNT
static String	DIRECTIVE_METRIC_NAME

Fields inherited from class io.cdap.cdap.etl.api.Transform

PLUGIN_TYPE

Constructor Summary

Constructors

Constructor and Description
Wrangler(Wrangler.Config config)

Method Summary

Modifier and Type	Method and Description
void	configurePipeline(PipelineConfigurer configurer) Configures the plugin during deployment of the pipeline that uses the plugin.
void	destroy()
void	initialize(TransformContext context) Initialize the wrangler by parsing the directives and creating the runtime context.
void	prepareRun(StageSubmitterContext context) prepareRun is invoked by the client once before the job is submitted, but after the resolution of macros if there are any defined.
Relation	transform(RelationalTranformContext relationalTranformContext, Relation relation)
void	transform(io.cdap.cdap.api.data.format.StructuredRecord input, Emitter<io.cdap.cdap.api.data.format.StructuredRecord> emitter) Transforms the input record by applying directives on the record being passed.

Methods inherited from class io.cdap.cdap.etl.api.Transform

getContext, onRunFinish

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface io.cdap.cdap.etl.api.relational.LinearRelationalTransform

requireUnionInputs, transform

Methods inherited from interface io.cdap.cdap.etl.api.relational.RelationalTransform

canUseEngine

Field Detail

DIRECTIVE_METRIC_NAME

public static final String DIRECTIVE_METRIC_NAME

See Also:

Constant Field Values

DIRECTIVE_METRIC_COUNT

public static final int DIRECTIVE_METRIC_COUNT

See Also:

Constant Field Values

DIRECTIVE_ENTITY_TYPE

public static final String DIRECTIVE_ENTITY_TYPE

See Also:

Constant Field Values

Constructor Detail

Wrangler

public Wrangler(Wrangler.Config config)

Method Detail

configurePipeline

public void configurePipeline(PipelineConfigurer configurer)

Configures the plugin during deployment of the pipeline that uses the plugin.

• Parses the directives configured. If there are any issues they will highlighted during deployment
• Input schema is validated.
• Compiles pre-condition expression.

Specified by:

configurePipeline in interface PipelineConfigurable

Overrides:

configurePipeline in class Transform<io.cdap.cdap.api.data.format.StructuredRecord,io.cdap.cdap.api.data.format.StructuredRecord>

prepareRun

public void prepareRun(StageSubmitterContext context)
                throws Exception

prepareRun is invoked by the client once before the job is submitted, but after the resolution of macros if there are any defined.

Specified by:

prepareRun in interface SubmitterLifecycle<StageSubmitterContext>

Overrides:

prepareRun in class Transform<io.cdap.cdap.api.data.format.StructuredRecord,io.cdap.cdap.api.data.format.StructuredRecord>

Parameters:

context - a instance StageSubmitterContext

Throws:

Exception - thrown if there any issue with prepareRun.

initialize

public void initialize(TransformContext context)
                throws Exception

Initialize the wrangler by parsing the directives and creating the runtime context.

Specified by:

initialize in interface StageLifecycle<TransformContext>

Overrides:

initialize in class Transform<io.cdap.cdap.api.data.format.StructuredRecord,io.cdap.cdap.api.data.format.StructuredRecord>

Parameters:

context - framework context being passed.

Throws:

Exception

destroy

public void destroy()

Specified by:

destroy in interface Destroyable

Overrides:

destroy in class Transform<io.cdap.cdap.api.data.format.StructuredRecord,io.cdap.cdap.api.data.format.StructuredRecord>

transform

public void transform(io.cdap.cdap.api.data.format.StructuredRecord input,
                      Emitter<io.cdap.cdap.api.data.format.StructuredRecord> emitter)
               throws Exception

Transforms the input record by applying directives on the record being passed.

Specified by:

transform in interface Transformation<io.cdap.cdap.api.data.format.StructuredRecord,io.cdap.cdap.api.data.format.StructuredRecord>

Parameters:

input - record to be transformed.

emitter - to collect all the output of the transformation.

Throws:

Exception - thrown if there are any issue with the transformation.

transform

public Relation transform(RelationalTranformContext relationalTranformContext,
                          Relation relation)

Specified by:

transform in interface LinearRelationalTransform