Package 

Interface Tokenizer

  • All Implemented Interfaces:
    
public interface Tokenizer<D extends Object, T extends Object>

    A tokenizer splits a piece of data D into a list of T tokens.

    • Nested Class Summary

      Nested Classes 
      Modifier and Type Class Description

    • Field Summary

      Fields 
      Modifier and Type Field Description

    • Constructor Summary

      Constructors 
      Constructor Description

    • Enum Constant Summary

      Enum Constants 
      Enum Constant Description

    • Method Summary

      Modifier and Type Method Description
      abstract List<T> tokenize(D data)

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait