org.apache.lucene.analysis.gosen

Class GosenAnalyzer

java.lang.Object

org.apache.lucene.analysis.Analyzer

org.apache.lucene.analysis.util.StopwordAnalyzerBase

org.apache.lucene.analysis.gosen.GosenAnalyzer

All Implemented Interfaces:

Closeable, AutoCloseable

public class GosenAnalyzer
extends org.apache.lucene.analysis.util.StopwordAnalyzerBase

Analyzer for Japanese which uses "Sen" morphological analyzer.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.lucene.analysis.Analyzer

org.apache.lucene.analysis.Analyzer.ReuseStrategy, org.apache.lucene.analysis.Analyzer.TokenStreamComponents

Field Summary

Fields inherited from class org.apache.lucene.analysis.util.StopwordAnalyzerBase

stopwords

Fields inherited from class org.apache.lucene.analysis.Analyzer

GLOBAL_REUSE_STRATEGY, PER_FIELD_REUSE_STRATEGY

Constructor Summary

Constructors

Constructor and Description
GosenAnalyzer() Create a GosenAnalyzer with the default stopwords and stoptags and no stemExclusionSet
GosenAnalyzer(org.apache.lucene.analysis.util.CharArraySet stopwords, Set<String> stoptags, org.apache.lucene.analysis.util.CharArraySet stemExclusionSet, String dictionaryDir) Create a GosenAnalyzer with the specified stopwords, stoptags, and stemExclusionSet
GosenAnalyzer(String dictionaryDir) Create a GosenAnalyzer with the default stopwords and stoptags and no stemExclusionSet and argument of dictionaryDir.

Method Summary

Methods

Modifier and Type	Method and Description
protected org.apache.lucene.analysis.Analyzer.TokenStreamComponents	createComponents(String field) Creates Analyzer.TokenStreamComponents used to tokenize all the text in the provided Reader.
static Set<?>	getDefaultStopSet()
static Set<String>	getDefaultStopTags()

Methods inherited from class org.apache.lucene.analysis.util.StopwordAnalyzerBase

getStopwordSet, loadStopwordSet, loadStopwordSet, loadStopwordSet

Methods inherited from class org.apache.lucene.analysis.Analyzer

close, getOffsetGap, getPositionIncrementGap, getReuseStrategy, getVersion, initReader, setVersion, tokenStream, tokenStream

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

GosenAnalyzer

public GosenAnalyzer()

Create a GosenAnalyzer with the default stopwords and stoptags and no stemExclusionSet

GosenAnalyzer

public GosenAnalyzer(String dictionaryDir)

Create a GosenAnalyzer with the default stopwords and stoptags and no stemExclusionSet
and argument of dictionaryDir.

GosenAnalyzer

public GosenAnalyzer(org.apache.lucene.analysis.util.CharArraySet stopwords,
             Set<String> stoptags,
             org.apache.lucene.analysis.util.CharArraySet stemExclusionSet,
             String dictionaryDir)

Create a GosenAnalyzer with the specified stopwords, stoptags, and stemExclusionSet

Parameters:

stopwords - a stopword set: words matching these (Surf

stoptags - a stoptags set: words containing these parts of speech will be removed from the stream.

stemExclusionSet - a stemming exclusion set: these words are ignored by GosenBasicFormFilter and GosenKatakanaStemFilter

dictionaryDir - a directory of dictionary

Method Detail

getDefaultStopSet

public static Set<?> getDefaultStopSet()

getDefaultStopTags

public static Set<String> getDefaultStopTags()

createComponents

protected org.apache.lucene.analysis.Analyzer.TokenStreamComponents createComponents(String field)

Creates Analyzer.TokenStreamComponents used to tokenize all the text in the provided Reader.

Specified by:

createComponents in class org.apache.lucene.analysis.Analyzer

Returns:

Analyzer.TokenStreamComponents built from a GosenTokenizer filtered with GosenWidthFilter, GosenPunctuationFilter, GosenPartOfSpeechStopFilter, StopFilter, SetKeywordMarkerFilter if a stem exclusion set is provided, GosenBasicFormFilter, GosenKatakanaStemFilter, and LowerCaseFilter