Package 

Interface Tokenizer

  • 
public interface Tokenizer<D extends Object, T extends Object>

    A tokenizer splits a piece of data D into a list of T tokens.

    • Method Summary

      Modifier and Type Method Description
      abstract List<T> tokenize(D data)

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait