Package org.opensearch.index.reindex

Class AbstractAsyncBulkByScrollAction<Request extends AbstractBulkByScrollRequest<Request>,Action extends TransportAction<Request,?>>

java.lang.Object

org.opensearch.index.reindex.AbstractAsyncBulkByScrollAction<Request,Action>

Direct Known Subclasses:

AsyncDeleteByQueryAction

public abstract class AbstractAsyncBulkByScrollAction<Request extends AbstractBulkByScrollRequest<Request>,Action extends TransportAction<Request,?>>
extends java.lang.Object

Abstract base for scrolling across a search and executing bulk actions on all results. All package private methods are package private so their tests can use them. Most methods run in the listener thread pool because the are meant to be fast and don't expect to block.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	AbstractAsyncBulkByScrollAction.DeleteRequestWrapper	AbstractAsyncBulkByScrollAction.RequestWrapper for DeleteRequest
static class	AbstractAsyncBulkByScrollAction.IndexRequestWrapper	AbstractAsyncBulkByScrollAction.RequestWrapper for IndexRequest
static class	AbstractAsyncBulkByScrollAction.OpType
static interface	AbstractAsyncBulkByScrollAction.RequestWrapper<Self extends DocWriteRequest<Self>>	Wrapper for the DocWriteRequest that are used in this action class.
static class	AbstractAsyncBulkByScrollAction.ScriptApplier	Apply a Script to a AbstractAsyncBulkByScrollAction.RequestWrapper

Field Summary

Fields

Modifier and Type	Field	Description
protected java.util.Optional<org.apache.http.HttpRequestInterceptor>	interceptor
protected org.apache.logging.log4j.Logger	logger
protected Request	mainRequest	The request for this action.
protected ScriptService	scriptService
protected org.opensearch.index.reindex.ReindexSslConfig	sslConfig
protected BulkByScrollTask	task
protected ThreadPool	threadPool
protected WorkerBulkByScrollTaskState	worker

Method Summary

Modifier and Type	Method	Description
protected boolean	accept(ScrollableHitSource.Hit doc)	Used to accept or ignore a search hit.
protected abstract AbstractAsyncBulkByScrollAction.RequestWrapper<?>	buildRequest(ScrollableHitSource.Hit doc)	Build the AbstractAsyncBulkByScrollAction.RequestWrapper for a single search hit.
protected BulkByScrollResponse	buildResponse(TimeValue took, java.util.List<BulkItemResponse.Failure> indexingFailures, java.util.List<ScrollableHitSource.SearchFailure> searchFailures, boolean timedOut)	Build the response for reindex actions.
java.util.function.BiFunction<AbstractAsyncBulkByScrollAction.RequestWrapper<?>,ScrollableHitSource.Hit,AbstractAsyncBulkByScrollAction.RequestWrapper<?>>	buildScriptApplier()	Build the BiFunction to apply to all AbstractAsyncBulkByScrollAction.RequestWrapper.
protected ScrollableHitSource	buildScrollableResultSource(BackoffPolicy backoffPolicy)
protected AbstractAsyncBulkByScrollAction.RequestWrapper<?>	copyMetadata(AbstractAsyncBulkByScrollAction.RequestWrapper<?> request, ScrollableHitSource.Hit doc)	Copies the metadata from a hit to the request.
protected void	copyRouting(AbstractAsyncBulkByScrollAction.RequestWrapper<?> request, java.lang.String routing)	Copy the routing from a search hit to the request.
protected void	finishHim(java.lang.Exception failure)	Finish the request.
protected void	finishHim(java.lang.Exception failure, java.util.List<BulkItemResponse.Failure> indexingFailures, java.util.List<ScrollableHitSource.SearchFailure> searchFailures, boolean timedOut)	Finish the request.
void	start()	Start the action by firing the initial search request.
static AbstractAsyncBulkByScrollAction.RequestWrapper<DeleteRequest>	wrap(DeleteRequest request)	Wraps a DeleteRequest in a AbstractAsyncBulkByScrollAction.RequestWrapper
static AbstractAsyncBulkByScrollAction.RequestWrapper<IndexRequest>	wrap(IndexRequest request)	Wraps a IndexRequest in a AbstractAsyncBulkByScrollAction.RequestWrapper

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

logger

protected final org.apache.logging.log4j.Logger logger

task

protected final BulkByScrollTask task

worker

protected final WorkerBulkByScrollTaskState worker

threadPool

protected final ThreadPool threadPool

scriptService

protected final ScriptService scriptService

sslConfig

protected final org.opensearch.index.reindex.ReindexSslConfig sslConfig

interceptor

protected final java.util.Optional<org.apache.http.HttpRequestInterceptor> interceptor

mainRequest

protected final Request extends AbstractBulkByScrollRequest<Request> mainRequest

The request for this action. Named mainRequest because we create lots of request variables all representing child requests of this mainRequest.

Method Detail

buildScriptApplier

public java.util.function.BiFunction<AbstractAsyncBulkByScrollAction.RequestWrapper<?>,ScrollableHitSource.Hit,AbstractAsyncBulkByScrollAction.RequestWrapper<?>> buildScriptApplier()

Build the BiFunction to apply to all AbstractAsyncBulkByScrollAction.RequestWrapper. Public for testings....

buildRequest

protected abstract AbstractAsyncBulkByScrollAction.RequestWrapper<?> buildRequest(ScrollableHitSource.Hit doc)

Build the AbstractAsyncBulkByScrollAction.RequestWrapper for a single search hit. This shouldn't handle metadata or scripting. That will be handled by copyMetadata and apply functions that can be overridden.

copyMetadata

protected AbstractAsyncBulkByScrollAction.RequestWrapper<?> copyMetadata(AbstractAsyncBulkByScrollAction.RequestWrapper<?> request,
                                                                         ScrollableHitSource.Hit doc)

Copies the metadata from a hit to the request.

copyRouting

protected void copyRouting(AbstractAsyncBulkByScrollAction.RequestWrapper<?> request,
                           java.lang.String routing)

Copy the routing from a search hit to the request.

accept

protected boolean accept(ScrollableHitSource.Hit doc)

Used to accept or ignore a search hit. Ignored search hits will be excluded from the bulk request. It is also where we fail on invalid search hits, like when the document has no source but it's required.

buildScrollableResultSource

protected ScrollableHitSource buildScrollableResultSource(BackoffPolicy backoffPolicy)

buildResponse

protected BulkByScrollResponse buildResponse(TimeValue took,
                                             java.util.List<BulkItemResponse.Failure> indexingFailures,
                                             java.util.List<ScrollableHitSource.SearchFailure> searchFailures,
                                             boolean timedOut)

Build the response for reindex actions.

start

public void start()

Start the action by firing the initial search request.

finishHim

protected void finishHim(java.lang.Exception failure)

Finish the request.

Parameters:

failure - if non null then the request failed catastrophically with this exception

finishHim

protected void finishHim(java.lang.Exception failure,
                         java.util.List<BulkItemResponse.Failure> indexingFailures,
                         java.util.List<ScrollableHitSource.SearchFailure> searchFailures,
                         boolean timedOut)

Finish the request.

Parameters:

failure - if non null then the request failed catastrophically with this exception

indexingFailures - any indexing failures accumulated during the request

searchFailures - any search failures accumulated during the request

timedOut - have any of the sub-requests timed out?

wrap

public static AbstractAsyncBulkByScrollAction.RequestWrapper<IndexRequest> wrap(IndexRequest request)

Wraps a IndexRequest in a AbstractAsyncBulkByScrollAction.RequestWrapper

wrap

public static AbstractAsyncBulkByScrollAction.RequestWrapper<DeleteRequest> wrap(DeleteRequest request)

Wraps a DeleteRequest in a AbstractAsyncBulkByScrollAction.RequestWrapper