IcuTextTokenizer

All Implemented Interfaces:

org.readium.r2.shared.util.tokenizer.Tokenizer
```
@RequiresApi(value = 24) 
public final class IcuTextTokenizer
 implements Tokenizer<String, IntRange>
                    
```
Implementation of a TextTokenizer using ICU components to perform the actual tokenization while taking into account languages specificities.

Constructor Summary

Constructors
Constructor	Description
`IcuTextTokenizer(Language language, TextUnit unit)`

Method Summary

Modifier and Type	Method	Description
`List<IntRange>`	`tokenize(String data)`

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

IcuTextTokenizer

IcuTextTokenizer(Language language, TextUnit unit)

Method Detail
- tokenize
```
 List<IntRange> tokenize(String data)
```