eu.dicodeproject.analysis.lucene
Class CleansingAnalyzer

java.lang.Object
  org.apache.lucene.analysis.Analyzer
      eu.dicodeproject.analysis.lucene.CleansingAnalyzer

All Implemented Interfaces:

Closeable

public final class CleansingAnalyzer
extends org.apache.lucene.analysis.Analyzer

In contrast to the Lucene standard analyser this one adds filtering tokens of less then minimum length (default two characters) and tokens that contain only digits.

Constructor Summary

CleansingAnalyzer()
Default init of lower bound to be equal to 2.

CleansingAnalyzer(int lowerBound, boolean enablePositionIncrement)

Method Summary

org.apache.lucene.analysis.TokenStream

tokenStream(String fieldName, Reader reader) Delegate most of the analysis to the Lucene standard analyzer, add filtering tokens of less than minimum length and filtering tokens that are digit only.

Methods inherited from class org.apache.lucene.analysis.Analyzer

close, getOffsetGap, getPositionIncrementGap, getPreviousTokenStream, reusableTokenStream, setPreviousTokenStream

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CleansingAnalyzer

public CleansingAnalyzer()

Default init of lower bound to be equal to 2.

CleansingAnalyzer

public CleansingAnalyzer(int lowerBound,
                         boolean enablePositionIncrement)

Method Detail

tokenStream

public org.apache.lucene.analysis.TokenStream tokenStream(String fieldName,
                                                          Reader reader)

Delegate most of the analysis to the Lucene standard analyzer, add filtering tokens of less than minimum length and filtering tokens that are digit only.

Specified by:

tokenStream in class org.apache.lucene.analysis.Analyzer