Package org.opensearch.analysis.common

Class CJKBigramFilterFactory

java.lang.Object

org.opensearch.index.AbstractIndexComponent

org.opensearch.index.analysis.AbstractTokenFilterFactory

org.opensearch.analysis.common.CJKBigramFilterFactory

All Implemented Interfaces:

TokenFilterFactory, IndexComponent

public final class CJKBigramFilterFactory
extends AbstractTokenFilterFactory

Factory that creates a CJKBigramFilter to form bigrams of CJK terms that are generated from StandardTokenizer or ICUTokenizer.

CJK types are set by these tokenizers, but you can also use flags to explicitly control which of the CJK scripts are turned into bigrams.

By default, when a CJK character has no adjacent characters to form a bigram, it is output in unigram form. If you want to always output both unigrams and bigrams, set the outputUnigrams flag. This can be used for a combined unigram+bigram approach.

In all cases, all non-CJK input is passed thru unmodified.

Field Summary

Fields inherited from class org.opensearch.index.analysis.AbstractTokenFilterFactory

version

Fields inherited from class org.opensearch.index.AbstractIndexComponent

deprecationLogger, indexSettings, logger

Fields inherited from interface org.opensearch.index.analysis.TokenFilterFactory

IDENTITY_FILTER

Method Summary

Modifier and Type	Method	Description
org.apache.lucene.analysis.TokenStream	create(org.apache.lucene.analysis.TokenStream tokenStream)
TokenFilterFactory	getSynonymFilter()

Methods inherited from class org.opensearch.index.analysis.AbstractTokenFilterFactory

name, version

Methods inherited from class org.opensearch.index.AbstractIndexComponent

getIndexSettings, index

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.opensearch.index.analysis.TokenFilterFactory

breaksFastVectorHighlighter, getAnalysisMode, getChainAwareTokenFilterFactory, normalize

Method Details

create

public org.apache.lucene.analysis.TokenStream create(org.apache.lucene.analysis.TokenStream tokenStream)

getSynonymFilter

public TokenFilterFactory getSynonymFilter()