Task.Support.CoreSupport
Class TheStringTokenizer.Tokenizer

java.lang.Object
  Task.Support.CoreSupport.TheStringTokenizer.Tokenizer

Enclosing class:

TheStringTokenizer

public class TheStringTokenizer.Tokenizer
extends java.lang.Object

This class simply takes an input String or StringBuffer and tokenizes the input based on a delimiter String.

Usage:

   String input = "20,LOR;ispq.com;490030,LOR;ispq.com;490040,LOR;ispq.com;490";
   String delim = "";
 

   Tokenizer t = new Tokenizer();
   t.set( input , delim );
 

   //this is for input with tokens
   System.out.println( "***valid input with tokens***" );
   System.out.println( "input:"+input );
   Tokenizer t = new Tokenizer( input , delim );
   System.out.println( "hasTokens():"+t.hasTokens() );
 

   String[] strRay = t.getTokens();
   for( int i=0; i
 

Output:

 ***valid input with tokens input:20,LOR;ispq.com;490030,LOR;ispq.com;490040,LOR;ispq.com;490
 hasTokens():true token:20,LOR;ispq.com;4900 <--token1 token:30,LOR;ispq.com;4900
 <--token2 40,LOR;ispq.com;490 <--remainder 

Note: Definition of a token

This class is different from the StringTokenizer class. Unlike the StringTokenizer class (which just looks for chars in the String to break up the String into tokens), the Tokenizer class looks for delimiters to break up words( and it there MUST be trailing delimiters to get the last token, or else it will be the remainder).

For example: t.set( "LOR;ispq.com;4900" , ";" ); will yield {"LOR","ispq.com"} when getTokens() is called. Whereas StringTokenizer would return {"LOR", "ispq.com","4900"}

Note: dependencies

TheStringTokenizer class depends on this class, so be sure to check this class if Tokenzier is modified.

Since:

2/18/2000,12:08pm

Version:

1.0

Author:

Nazmul Idris

Field Summary

protected java.lang.String	delim
protected java.lang.String	input
protected java.lang.String	remainder

Constructor Summary

TheStringTokenizer.Tokenizer()

Method Summary

java.lang.StringBuffer	getRemainderOfInput() This method returns an empty StringBuffer if there is no remainder in this Tokenizer object.
java.lang.String[]	getTokens() Returns all the tokens in the current message buffer, that are delimited by the String.
boolean	hasRemainder() Determines whether there is any data left over in the message buffer after all the tokens have been extracted from it.
boolean	hasTokens() Returns true if there tokens exist in the message buffer.
TheStringTokenizer.Tokenizer	set(java.lang.StringBuffer sb, java.lang.String delim) same as set( String , String ), only it converts the StringBuffer to a String before calling the other set()
TheStringTokenizer.Tokenizer	set(java.lang.String input, java.lang.String delim) This method sets up the Tokenizer to process the given message buffer and delim (delimiter) string.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

input

protected java.lang.String input

delim

protected java.lang.String delim

remainder

protected java.lang.String remainder

Constructor Detail

TheStringTokenizer.Tokenizer

public TheStringTokenizer.Tokenizer()

Method Detail

set

public TheStringTokenizer.Tokenizer set(java.lang.String input,
                                        java.lang.String delim)

This method sets up the Tokenizer to process the given message buffer and delim (delimiter) string. The internal state of the Tokenizer object is reset and setup to work with all the accessor methods available to this class.

Parameters:

input - message buffer that contains the untokenized data received from the socket

delim - delimiter string used to delimit the tokens in the message buffer.

Returns:

returns a reference to this object (just as a convenience) to invoke other methods on it.

set

public TheStringTokenizer.Tokenizer set(java.lang.StringBuffer sb,
                                        java.lang.String delim)

same as set( String , String ), only it converts the StringBuffer to a String before calling the other set()

hasTokens

public boolean hasTokens()

Returns true if there tokens exist in the message buffer.

Returns:

true if there are any tokens (delimted by delim), else returns false

getTokens

public java.lang.String[] getTokens()

Returns all the tokens in the current message buffer, that are delimited by the String. The tokens are returned as an array of Strings.

If the input parameter(s) are invalid, null is returned.

If there are no tokens, null is returned.

Returns:

String array of tokens (if they exist), otherwise return null.

hasRemainder

public boolean hasRemainder()

Determines whether there is any data left over in the message buffer after all the tokens have been extracted from it.

Returns:

true if there is data left over, after tokens are removed, false otherwise.

getRemainderOfInput

public java.lang.StringBuffer getRemainderOfInput()

This method returns an empty StringBuffer if there is no remainder in this Tokenizer object. Otherwise, it simply returns a StringBuffer with the data in it.

This behavior is different as it does not return null if there is no remainder.

The remainder is defined as data left in the buffer trailing the last delim string.

Returns:

StringBuffer containing remainder text (is never null)