net.sourceforge.openutils.mgnlcriteria.utils
Class BaseAnalyzer

java.lang.Object
  org.apache.lucene.analysis.Analyzer
      net.sourceforge.openutils.mgnlcriteria.utils.BaseAnalyzer

Direct Known Subclasses:

ASCIIFoldingAnalyzer, Latin1Analyzer

public abstract class BaseAnalyzer
extends org.apache.lucene.analysis.Analyzer

Base analyzer class. Subclasses should simply implement the tokenFiltersChain() mathod in order to add TokenFilters.

Version:

$Id: BaseAnalyzer.java 3285 2011-01-24 18:10:19Z fgiust $

Author:

fgiust

Constructor Summary

BaseAnalyzer()

Method Summary

org.apache.lucene.analysis.TokenStream	reusableTokenStream(String fieldName, Reader reader)
protected abstract org.apache.lucene.analysis.TokenStream	tokenFiltersChain(org.apache.lucene.analysis.TokenStream tokenStream) Apply a set of TokenFilters to the TokenStream
protected org.apache.lucene.analysis.standard.StandardTokenizer	tokenize(Reader reader) Tokenize using a StandardTokenizer.
org.apache.lucene.analysis.TokenStream	tokenStream(String fieldName, Reader reader)

Methods inherited from class org.apache.lucene.analysis.Analyzer

close, getPositionIncrementGap, getPreviousTokenStream, setPreviousTokenStream

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

BaseAnalyzer

public BaseAnalyzer()

Method Detail

tokenStream

public org.apache.lucene.analysis.TokenStream tokenStream(String fieldName,
                                                          Reader reader)

Specified by:

tokenStream in class org.apache.lucene.analysis.Analyzer

reusableTokenStream

public org.apache.lucene.analysis.TokenStream reusableTokenStream(String fieldName,
                                                                  Reader reader)
                                                           throws IOException

Overrides:

reusableTokenStream in class org.apache.lucene.analysis.Analyzer

Throws:

IOException

tokenize

protected org.apache.lucene.analysis.standard.StandardTokenizer tokenize(Reader reader)

Tokenize using a StandardTokenizer. Subclasses may override this mehod.

Parameters:

reader - base reader

Returns:

tokenizer

tokenFiltersChain

protected abstract org.apache.lucene.analysis.TokenStream tokenFiltersChain(org.apache.lucene.analysis.TokenStream tokenStream)

Apply a set of TokenFilters to the TokenStream

Parameters:

tokenStream - original tokenStream

Returns:

filtered tokenStream