org.lastbamboo.common.sdp
Class LexerCore

java.lang.Object
  org.lastbamboo.common.sdp.StringTokenizer
      org.lastbamboo.common.sdp.LexerCore

Direct Known Subclasses:

Lexer

public class LexerCore
extends StringTokenizer

A lexical analyzer that is used by all parsers in our implementation.

Author:

M. Ranganathan

Field Summary

static int	ALPHA
static int	AND
static int	AT
static int	BACK_QUOTE
static int	BACKSLASH
static int	BAR
static int	COLON
protected Hashtable	currentLexer
protected String	currentLexerName
protected Token	currentMatch
static int	DIGIT
static int	DOLLAR
static int	DOT
static int	DOUBLEQUOTE
static int	END
static int	EQUALS
static int	EXCLAMATION
protected static Hashtable	globalSymbolTable
static int	GREATER_THAN
static int	HAT
static int	HT
static int	ID
static int	L_CURLY
static int	L_SQUARE_BRACKET
static int	LESS_THAN
protected static Hashtable	lexerTables
static int	LPAREN
static int	MINUS
static int	NULL
static int	PERCENT
static int	PLUS
static int	POUND
static int	QUESTION
static int	QUOTE
static int	R_CURLY
static int	R_SQUARE_BRACKET
static int	RPAREN
static int	SAFE
static int	SEMICOLON
static int	SLASH
static int	SP
static int	STAR
static int	START
static int	TILDE
static int	UNDERSCORE
static int	WHITESPACE

Fields inherited from class org.lastbamboo.common.sdp.StringTokenizer

buffer, ptr, savedPtr

Constructor Summary

protected	LexerCore()
	LexerCore(String lexerName)
	LexerCore(String lexerName, String buffer) Initialize the lexer with a buffer.

Method Summary

protected void	addKeyword(String name, int value)
protected Hashtable	addLexer(String lexerName)
String	byteStringNoComma() Return a substring containing no commas
String	byteStringNoSemicolon() Return a substring containing no semicolons.
static String	charAsString(char ch)
String	charAsString(int nchars) Lookahead in the inputBuffer for n chars and return as a string.
String	comment() Parse a comment string cursor is at a "(".
ParseException	createParseException() Create a parse exception.
String	getBuffer() Get the buffer.
String	getNextId() Get the next id.
Token	getNextToken()
int	getPtr() Get the read pointer.
String	getRest() Get the rest of the String
String	getString(char c) Get the sub-String until the character is encountered
String	lookupToken(int value)
int	markInputPosition() Mark the position for backtracking.
Token	match(int tok) Match the given token or throw an exception if no such token can be matched.
String	number() Get and consume the next number.
String	peekNextId() Peek the next id but dont move the buffer pointer forward.
Token	peekNextToken() Look ahead for one token.
Vector	peekNextToken(int ntokens)
String	quotedString() Parse a comment string cursor is at a ".
void	rewindInputPosition(int position) Rewind the input ptr to the marked position.
void	selectLexer(String lexerName)
void	SPorHT()
boolean	startsId()
boolean	startsSafeToken()
String	ttoken()
String	ttokenAllowSpace()
String	ttokenSafe()

Methods inherited from class org.lastbamboo.common.sdp.StringTokenizer

consume, consume, getLine, getLines, getNextChar, getNextToken, getSDPFieldName, hasMoreChars, isAlpha, isDigit, isHexDigit, lookAhead, lookAhead, nextToken, peekLine

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

START

public static final int START

See Also:

Constant Field Values

END

public static final int END

See Also:

Constant Field Values

ID

public static final int ID

See Also:

Constant Field Values

SAFE

public static final int SAFE

See Also:

Constant Field Values

WHITESPACE

public static final int WHITESPACE

See Also:

Constant Field Values

DIGIT

public static final int DIGIT

See Also:

Constant Field Values

ALPHA

public static final int ALPHA

See Also:

Constant Field Values

BACKSLASH

public static final int BACKSLASH

See Also:

Constant Field Values

QUOTE

public static final int QUOTE

See Also:

Constant Field Values

AT

public static final int AT

See Also:

Constant Field Values

SP

public static final int SP

See Also:

Constant Field Values

HT

public static final int HT

See Also:

Constant Field Values

COLON

public static final int COLON

See Also:

Constant Field Values

STAR

public static final int STAR

See Also:

Constant Field Values

DOLLAR

public static final int DOLLAR

See Also:

Constant Field Values

PLUS

public static final int PLUS

See Also:

Constant Field Values

POUND

public static final int POUND

See Also:

Constant Field Values

MINUS

public static final int MINUS

See Also:

Constant Field Values

DOUBLEQUOTE

public static final int DOUBLEQUOTE

See Also:

Constant Field Values

TILDE

public static final int TILDE

See Also:

Constant Field Values

BACK_QUOTE

public static final int BACK_QUOTE

See Also:

Constant Field Values

NULL

public static final int NULL

See Also:

Constant Field Values

EQUALS

public static final int EQUALS

See Also:

Constant Field Values

SEMICOLON

public static final int SEMICOLON

See Also:

Constant Field Values

SLASH

public static final int SLASH

See Also:

Constant Field Values

L_SQUARE_BRACKET

public static final int L_SQUARE_BRACKET

See Also:

Constant Field Values

R_SQUARE_BRACKET

public static final int R_SQUARE_BRACKET

See Also:

Constant Field Values

R_CURLY

public static final int R_CURLY

See Also:

Constant Field Values

L_CURLY

public static final int L_CURLY

See Also:

Constant Field Values

HAT

public static final int HAT

See Also:

Constant Field Values

BAR

public static final int BAR

See Also:

Constant Field Values

DOT

public static final int DOT

See Also:

Constant Field Values

EXCLAMATION

public static final int EXCLAMATION

See Also:

Constant Field Values

LPAREN

public static final int LPAREN

See Also:

Constant Field Values

RPAREN

public static final int RPAREN

See Also:

Constant Field Values

GREATER_THAN

public static final int GREATER_THAN

See Also:

Constant Field Values

LESS_THAN

public static final int LESS_THAN

See Also:

Constant Field Values

PERCENT

public static final int PERCENT

See Also:

Constant Field Values

QUESTION

public static final int QUESTION

See Also:

Constant Field Values

AND

public static final int AND

See Also:

Constant Field Values

UNDERSCORE

public static final int UNDERSCORE

See Also:

Constant Field Values

globalSymbolTable

protected static Hashtable globalSymbolTable

lexerTables

protected static Hashtable lexerTables

currentLexer

protected Hashtable currentLexer

currentLexerName

protected String currentLexerName

currentMatch

protected Token currentMatch

Constructor Detail

LexerCore

protected LexerCore()

LexerCore

public LexerCore(String lexerName)

LexerCore

public LexerCore(String lexerName,
                 String buffer)

Initialize the lexer with a buffer.

Method Detail

addKeyword

protected void addKeyword(String name,
                          int value)

lookupToken

public String lookupToken(int value)

addLexer

protected Hashtable addLexer(String lexerName)

selectLexer

public void selectLexer(String lexerName)

peekNextId

public String peekNextId()

Peek the next id but dont move the buffer pointer forward.

getNextId

public String getNextId()

Get the next id.

getNextToken

public Token getNextToken()

peekNextToken

public Token peekNextToken()
                    throws ParseException

Look ahead for one token.

Throws:

ParseException

peekNextToken

public Vector peekNextToken(int ntokens)
                     throws ParseException

Throws:

ParseException

match

public Token match(int tok)
            throws ParseException

Match the given token or throw an exception if no such token can be matched.

Throws:

ParseException

SPorHT

public void SPorHT()

startsId

public boolean startsId()

startsSafeToken

public boolean startsSafeToken()

ttoken

public String ttoken()

ttokenAllowSpace

public String ttokenAllowSpace()

ttokenSafe

public String ttokenSafe()

quotedString

public String quotedString()
                    throws ParseException

Parse a comment string cursor is at a ". Leave cursor at closing "

Returns:

the substring containing the quoted string excluding the closing quote.

Throws:

ParseException

comment

public String comment()
               throws ParseException

Parse a comment string cursor is at a "(". Leave cursor at )

Returns:

the substring containing the comment excluding the closing brace.

Throws:

ParseException

byteStringNoSemicolon

public String byteStringNoSemicolon()

Return a substring containing no semicolons.

Returns:

a substring containing no semicolons.

byteStringNoComma

public String byteStringNoComma()

Return a substring containing no commas

Returns:

a substring containing no commas.

charAsString

public static String charAsString(char ch)

charAsString

public String charAsString(int nchars)

Lookahead in the inputBuffer for n chars and return as a string. Do not consume the input.

number

public String number()
              throws ParseException

Get and consume the next number.

Returns:

a substring corresponding to a number (i.e. sequence of digits).

Throws:

ParseException

markInputPosition

public int markInputPosition()

Mark the position for backtracking.

Returns:

the current location of the pointer.

rewindInputPosition

public void rewindInputPosition(int position)

Rewind the input ptr to the marked position.

Parameters:

position - - the position to rewind the parser to.

getRest

public String getRest()

Get the rest of the String

Returns:

rest of the buffer.

getString

public String getString(char c)
                 throws ParseException

Get the sub-String until the character is encountered

Parameters:

c - the character to match

Returns:

the substring that matches.

Throws:

ParseException

getPtr

public int getPtr()

Get the read pointer.

getBuffer

public String getBuffer()

Get the buffer.

createParseException

public ParseException createParseException()

Create a parse exception.