ConcatenateGraphTokenFilterFactory (analysis-common 1.3.1 API)

java.lang.Object
- org.opensearch.index.AbstractIndexComponent
- - org.opensearch.index.analysis.AbstractTokenFilterFactory
  - - org.opensearch.analysis.common.ConcatenateGraphTokenFilterFactory

All Implemented Interfaces:

TokenFilterFactory, IndexComponent
```
public class ConcatenateGraphTokenFilterFactory
extends AbstractTokenFilterFactory
```
Factory for ConcatenateGraphFilter. Adopted from ConcatenateGraphFilterFactory, with some changes to default values: token_separator is a "space", preserve_position_increments is false to avoid duplicated separators, max_graph_expansions is 100 as the default value of 10_000 seems to be unnecessarily large and preserve_separator is false.
- preserve_separator: For LegacyESVersion lesser than LegacyESVersion.V_7_6_0 i.e. lucene versions lesser than Version.LUCENE_8_4_0 Whether ConcatenateGraphFilter.SEP_LABEL should separate the input tokens in the concatenated token.
- token_separator: Separator to use for concatenation. Must be a String with a single character or empty. If not present, DEFAULT_TOKEN_SEPARATOR will be used. If empty i.e. "", tokens will be concatenated without any separators.
- preserve_position_increments: Whether to add an empty token for missing positions. If not present, DEFAULT_PRESERVE_POSITION_INCREMENTS will be used.
- max_graph_expansions: If the tokenStream graph has more than this many possible paths through, then we'll throw TooComplexToDeterminizeException to preserve the stability and memory of the machine. If not present, DEFAULT_MAX_GRAPH_EXPANSIONS will be used.
See Also:

ConcatenateGraphFilter

Field Summary

Fields
Modifier and Type	Field	Description
`static int`	`DEFAULT_MAX_GRAPH_EXPANSIONS`
`static boolean`	`DEFAULT_PRESERVE_POSITION_INCREMENTS`
`static java.lang.String`	`DEFAULT_TOKEN_SEPARATOR`

Fields inherited from class org.opensearch.index.AbstractIndexComponent
deprecationLogger, indexSettings, logger

Fields inherited from interface org.opensearch.index.analysis.TokenFilterFactory
IDENTITY_FILTER

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type Method Description

org.apache.lucene.analysis.TokenStream create(org.apache.lucene.analysis.TokenStream tokenStream)
- Methods inherited from class org.opensearch.index.analysis.AbstractTokenFilterFactory
  name
- Methods inherited from class org.opensearch.index.AbstractIndexComponent
  getIndexSettings, index
- Methods inherited from class java.lang.Object
  clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
- Methods inherited from interface org.opensearch.index.analysis.TokenFilterFactory
  breaksFastVectorHighlighter, getAnalysisMode, getChainAwareTokenFilterFactory, getSynonymFilter, normalize

Field Detail
- DEFAULT_TOKEN_SEPARATOR
```
public static final java.lang.String DEFAULT_TOKEN_SEPARATOR
```
  See Also:
  
  Constant Field Values
- DEFAULT_MAX_GRAPH_EXPANSIONS
```
public static final int DEFAULT_MAX_GRAPH_EXPANSIONS
```
  See Also:
  
  Constant Field Values
- DEFAULT_PRESERVE_POSITION_INCREMENTS
```
public static final boolean DEFAULT_PRESERVE_POSITION_INCREMENTS
```
  See Also:
  
  Constant Field Values

Method Detail

create

public org.apache.lucene.analysis.TokenStream create(org.apache.lucene.analysis.TokenStream tokenStream)