Package org.opensearch.analysis.common

Class KeywordMarkerTokenFilterFactory

java.lang.Object

org.opensearch.index.AbstractIndexComponent

org.opensearch.index.analysis.AbstractTokenFilterFactory

org.opensearch.analysis.common.KeywordMarkerTokenFilterFactory

All Implemented Interfaces:

TokenFilterFactory, IndexComponent

public class KeywordMarkerTokenFilterFactory
extends AbstractTokenFilterFactory

A factory for creating keyword marker token filters that prevent tokens from being modified by stemmers. Two types of keyword marker filters are available: the SetKeywordMarkerFilter and the PatternKeywordMarkerFilter. The SetKeywordMarkerFilter uses a set of keywords to denote which tokens should be excluded from stemming. This filter is created if the settings include keywords, which contains the list of keywords, or `keywords_path`, which contains a path to a file in the config directory with the keywords. The PatternKeywordMarkerFilter uses a regular expression pattern to match against tokens that should be excluded from stemming. This filter is created if the settings include keywords_pattern, which contains the regular expression to match against.

Field Summary

Fields inherited from class org.opensearch.index.AbstractIndexComponent

deprecationLogger, indexSettings, logger

Fields inherited from interface org.opensearch.index.analysis.TokenFilterFactory

IDENTITY_FILTER

Method Summary

Modifier and Type	Method	Description
org.apache.lucene.analysis.TokenStream	create(org.apache.lucene.analysis.TokenStream tokenStream)

Methods inherited from class org.opensearch.index.analysis.AbstractTokenFilterFactory

name

Methods inherited from class org.opensearch.index.AbstractIndexComponent

getIndexSettings, index

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.opensearch.index.analysis.TokenFilterFactory

breaksFastVectorHighlighter, getAnalysisMode, getChainAwareTokenFilterFactory, getSynonymFilter, normalize

Method Detail

create

public org.apache.lucene.analysis.TokenStream create(org.apache.lucene.analysis.TokenStream tokenStream)