Package net.sansa_stack.spark.rdd.function

Interface JavaRddFunction<I,O>

Type Parameters:

I -

O -

All Superinterfaces:

Function<org.apache.spark.api.java.JavaRDD<I>,org.apache.spark.api.java.JavaRDD<O>>, Serializable, org.aksw.commons.lambda.serializable.SerializableFunction<org.apache.spark.api.java.JavaRDD<I>,org.apache.spark.api.java.JavaRDD<O>>

Functional Interface:

This is a functional interface and can therefore be used as the assignment target for a lambda expression or method reference.

@FunctionalInterface
public interface JavaRddFunction<I,O>
extends org.aksw.commons.lambda.serializable.SerializableFunction<org.apache.spark.api.java.JavaRDD<I>,org.apache.spark.api.java.JavaRDD<O>>

Interface for building chains of transformations over JavaRDDs and JavaPairRDDs. Example usage:

 {code
 JavaRddFunction<Resource, Resource> compositeTransform =
     JavaRddFunction.identity()
         .toPairRdd(JavaRddOfResourcesOps::mapToNamedModels)
         .andThen(rdd -> JavaRddOfNamedModelsOps.groupNamedModels(rdd, true, true, 0))
         .toRdd(JavaRddOfNamedModelsOps::mapToResources);

 JavaRDD newRdd = compositeTransform.apply(rdd);
 }
 

Author:

Claus Stadler 2021-10-08

Method Summary

Modifier and Type	Method	Description
default <X> JavaRddFunction<I,X>	andThen(JavaRddFunction<O,X> next)
static <I> JavaRddFunction<I,I>	identity()
default <K, V> ToJavaPairRddFunction<I,K,V>	toPairRdd(ToJavaPairRddFunction<O,K,V> next)

Methods inherited from interface java.util.function.Function

andThen, apply, compose

Method Details

andThen

default <X> JavaRddFunction<I,X> andThen(JavaRddFunction<O,X> next)

toPairRdd

default <K,
V> ToJavaPairRddFunction<I,K,V> toPairRdd(ToJavaPairRddFunction<O,K,V> next)

identity

static <I> JavaRddFunction<I,I> identity()