io.ray.streaming.python

Class PythonPartition

java.lang.Object

io.ray.streaming.python.PythonPartition

All Implemented Interfaces:

Function, Partition<Object>, Serializable

public class PythonPartition
extends Object
implements Partition<Object>

Represents a python partition function.

Python worker can create a partition object using information in this PythonPartition.

If this object is constructed from serialized python partition, python worker can deserialize it to create python partition directly. If this object is constructed from moduleName and className/functionName, python worker will use `importlib` to load python partition function.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
static PythonPartition	BroadcastPartition
static String	FORWARD_PARTITION_CLASS
static PythonPartition	ForwardPartition
static PythonPartition	KeyPartition
static PythonPartition	RoundRobinPartition

Constructor Summary

Constructors

Constructor and Description
PythonPartition(byte[] partition)
PythonPartition(String moduleName, String functionName) Create a python partition from a moduleName and partition function name

Method Summary

Modifier and Type	Method and Description
String	getFunctionName()
String	getModuleName()
byte[]	getPartition()
boolean	isConstructedFromBinary()
int[]	partition(Object record, int numPartition) Given a record and downstream partitions, determine which partition(s) should receive the record.
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface io.ray.streaming.api.function.Function

loadCheckpoint, saveCheckpoint

Field Detail

BroadcastPartition

public static final PythonPartition BroadcastPartition

KeyPartition

public static final PythonPartition KeyPartition

RoundRobinPartition

public static final PythonPartition RoundRobinPartition

FORWARD_PARTITION_CLASS

public static final String FORWARD_PARTITION_CLASS

See Also:

Constant Field Values

ForwardPartition

public static final PythonPartition ForwardPartition

Constructor Detail

PythonPartition

public PythonPartition(byte[] partition)

PythonPartition

public PythonPartition(String moduleName,
                       String functionName)

Create a python partition from a moduleName and partition function name

Parameters:

moduleName - module name of python partition

functionName - function/class name of the partition function.

Method Detail

partition

public int[] partition(Object record,
                       int numPartition)

Description copied from interface: Partition

Given a record and downstream partitions, determine which partition(s) should receive the record.

Specified by:

partition in interface Partition<Object>

Parameters:

record - The record.

numPartition - num of partitions

Returns:

IDs of the downstream partitions that should receive the record.

getPartition

public byte[] getPartition()

getModuleName

public String getModuleName()

getFunctionName

public String getFunctionName()

isConstructedFromBinary

public boolean isConstructedFromBinary()

toString

public String toString()

Overrides:

toString in class Object