NaiveTextTokenizer

All Implemented Interfaces:

org.readium.r2.shared.util.tokenizer.Tokenizer
```
public final class NaiveTextTokenizer
 implements Tokenizer<String, IntRange>
                    
```
A naive Tokenizer relying on java.text.BreakIterator to split the content.
Use IcuTextTokenizer for better results.

Constructor Summary

Constructors
Constructor	Description
`NaiveTextTokenizer(TextUnit unit)`

Method Summary

Modifier and Type	Method	Description
`List<IntRange>`	`tokenize(String data)`

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail
- NaiveTextTokenizer
```
NaiveTextTokenizer(TextUnit unit)
```

Method Detail
- tokenize
```
 List<IntRange> tokenize(String data)
```