io.ray.streaming.runtime.worker

Class JobWorker

java.lang.Object

io.ray.streaming.runtime.worker.JobWorker

All Implemented Interfaces:

Serializable

public class JobWorker
extends Object
implements Serializable

The streaming worker implementation class, it is ray actor. JobWorker is created by JobMaster through ray api, and JobMaster communicates with JobWorker through Ray.call().

The JobWorker is responsible for creating tasks and defines the methods of communication between workers.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
ContextBackend	contextBackend
Object	initialStateChangeLock
AtomicBoolean	isRecreate isRecreate=true means this worker is initialized more than once after actor created.

Constructor Summary

Constructors

Constructor and Description
JobWorker(ExecutionVertex executionVertex)

Method Summary

Modifier and Type	Method and Description
Boolean	checkIfNeedRollback(Long startCallTs)
Boolean	clearExpiredCheckpoint(Long expiredStateCpId, Long expiredQueueCpId)
ExecutionVertex	getExecutionVertex()
StreamTask	getTask()
StreamingWorkerConfig	getWorkerConfig()
JobWorkerContext	getWorkerContext()
Boolean	init(JobWorkerContext workerContext) Initialize JobWorker and data communication pipeline.
Boolean	notifyCheckpointTimeout(Long checkpointId)
void	onReaderMessage(byte[] buffer) Used by upstream streaming queue to send data to this actor
byte[]	onReaderMessageSync(byte[] buffer) Used by upstream streaming queue to send data to this actor and receive result from this actor
void	onWriterMessage(byte[] buffer) Used by downstream streaming queue to send data to this actor
byte[]	onWriterMessageSync(byte[] buffer) Used by downstream streaming queue to send data to this actor and receive result from this actor
void	requestRollback(String exceptionMsg)
CallResult<ChannelRecoverInfo>	rollback(Long checkpointId, Long startRollbackTs) Start worker's stream tasks with specific checkpoint ID.
void	saveContext()
Boolean	triggerCheckpoint(Long barrierId) Trigger source job worker checkpoint

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

initialStateChangeLock

public final Object initialStateChangeLock

isRecreate

public AtomicBoolean isRecreate

isRecreate=true means this worker is initialized more than once after actor created.

contextBackend

public ContextBackend contextBackend

Constructor Detail

JobWorker

public JobWorker(ExecutionVertex executionVertex)

Method Detail

saveContext

public void saveContext()

init

public Boolean init(JobWorkerContext workerContext)

Initialize JobWorker and data communication pipeline.

rollback

public CallResult<ChannelRecoverInfo> rollback(Long checkpointId,
                                               Long startRollbackTs)

Start worker's stream tasks with specific checkpoint ID.

Returns:

a CallResult with ChannelRecoverInfo, contains ChannelRecoverInfo.ChannelCreationStatus of each input queue.

triggerCheckpoint

public Boolean triggerCheckpoint(Long barrierId)

Trigger source job worker checkpoint

notifyCheckpointTimeout

public Boolean notifyCheckpointTimeout(Long checkpointId)

clearExpiredCheckpoint

public Boolean clearExpiredCheckpoint(Long expiredStateCpId,
                                      Long expiredQueueCpId)

requestRollback

public void requestRollback(String exceptionMsg)

checkIfNeedRollback

public Boolean checkIfNeedRollback(Long startCallTs)

getWorkerConfig

public StreamingWorkerConfig getWorkerConfig()

getWorkerContext

public JobWorkerContext getWorkerContext()

getExecutionVertex

public ExecutionVertex getExecutionVertex()

getTask

public StreamTask getTask()

onReaderMessage

public void onReaderMessage(byte[] buffer)

Used by upstream streaming queue to send data to this actor

onReaderMessageSync

public byte[] onReaderMessageSync(byte[] buffer)

Used by upstream streaming queue to send data to this actor and receive result from this actor

onWriterMessage

public void onWriterMessage(byte[] buffer)

Used by downstream streaming queue to send data to this actor

onWriterMessageSync

public byte[] onWriterMessageSync(byte[] buffer)

Used by downstream streaming queue to send data to this actor and receive result from this actor