Package org.opensearch.index.rankeval

Interface EvaluationMetric

All Superinterfaces:

NamedWriteable, ToXContent, ToXContentObject, Writeable

All Known Implementing Classes:

DiscountedCumulativeGain, ExpectedReciprocalRank, MeanReciprocalRank, PrecisionAtK, RecallAtK

public interface EvaluationMetric
extends ToXContentObject, NamedWriteable

Implementations of EvaluationMetric need to provide a way to compute the quality metric for a result list returned by some search (@link SearchHits) and a list of rated documents.

Nested Class Summary

Nested classes/interfaces inherited from interface org.opensearch.common.xcontent.ToXContent

ToXContent.DelegatingMapParams, ToXContent.MapParams, ToXContent.Params

Nested classes/interfaces inherited from interface org.opensearch.common.io.stream.Writeable

Writeable.Reader<V extends java.lang.Object>, Writeable.Writer<V extends java.lang.Object>

Field Summary

Fields inherited from interface org.opensearch.common.xcontent.ToXContent

EMPTY_PARAMS

Method Summary

Modifier and Type	Method	Description
default double	combine(java.util.Collection<EvalQueryQuality> partialResults)	Combine several EvalQueryQuality results into the overall evaluation score.
EvalQueryQuality	evaluate(java.lang.String taskId, SearchHit[] hits, java.util.List<RatedDocument> ratedDocs)	Evaluates a single ranking evaluation case.
static java.util.List<org.opensearch.index.rankeval.RatedDocument.DocumentKey>	filterUnratedDocuments(java.util.List<RatedSearchHit> ratedHits)	Filter RatedSearchHits that do not have a rating.
default java.util.OptionalInt	forcedSearchSize()	Metrics can define a size of the search hits windows they want to retrieve by overwriting this method.
static java.util.List<RatedSearchHit>	joinHitsWithRatings(SearchHit[] hits, java.util.List<RatedDocument> ratedDocs)	Joins hits with rated documents using the joint _index/_id document key.

Methods inherited from interface org.opensearch.common.io.stream.NamedWriteable

getWriteableName

Methods inherited from interface org.opensearch.common.xcontent.ToXContent

toXContent

Methods inherited from interface org.opensearch.common.xcontent.ToXContentObject

isFragment

Methods inherited from interface org.opensearch.common.io.stream.Writeable

writeTo

Method Details

evaluate

EvalQueryQuality evaluate(java.lang.String taskId, SearchHit[] hits, java.util.List<RatedDocument> ratedDocs)

Evaluates a single ranking evaluation case.

Parameters:

taskId - an identifier of the query for which the search ranking is evaluated

hits - the search result hits

ratedDocs - the documents that contain the document rating for this query case

Returns:

an EvalQueryQuality instance that contains the metric score with respect to the provided search hits and ratings

joinHitsWithRatings

static java.util.List<RatedSearchHit> joinHitsWithRatings(SearchHit[] hits, java.util.List<RatedDocument> ratedDocs)

Joins hits with rated documents using the joint _index/_id document key.

filterUnratedDocuments

static java.util.List<org.opensearch.index.rankeval.RatedDocument.DocumentKey> filterUnratedDocuments(java.util.List<RatedSearchHit> ratedHits)

Filter RatedSearchHits that do not have a rating.

combine

default double combine(java.util.Collection<EvalQueryQuality> partialResults)

Combine several EvalQueryQuality results into the overall evaluation score. This defaults to averaging over the partial results, but can be overwritten to obtain a different behavior.

forcedSearchSize

default java.util.OptionalInt forcedSearchSize()

Metrics can define a size of the search hits windows they want to retrieve by overwriting this method. The default implementation returns an empty optional.

Returns:

the number of search hits this metrics requests