sjm.parse.tokens
Class Token

java.lang.Object
  |
  +--sjm.parse.tokens.Token

Direct Known Subclasses:

Token

public class Token

extends java.lang.Object

A token represents a logical chunk of a string. For example, a typical tokenizer would break the string "1.23 <= 12.3" into three tokens: the number 1.23, a less-than-or-equal symbol, and the number 12.3. A token is a receptacle, and relies on a tokenizer to decide precisely how to divide a string into tokens.

Version:

1.0

Author:

Steven J. Metsker

Field Summary

static Token	EOF A constant indicating that there are no more tokens
protected double	nval
protected java.lang.String	sval
static TokenType	TT_EOF A constant indicating that the end of the stream has been read.
static TokenType	TT_NUMBER A constant indicating that a token is a number, like 3.14
static TokenType	TT_QUOTED A constant indicating that a token is a quoted string, like "Launch Mi".
static TokenType	TT_SYMBOL A constant indicating that a token is a symbol like "<=".
static TokenType	TT_WORD A constant indicating a token is a word, like "cat"
protected TokenType	ttype

Constructor Summary

Token(char c)
Constructs a token from the given char.

Token(double nval)
Constructs a token from the given number.

Token(java.lang.String sval)
Constructs a token from the given string.

Token(TokenType ttype, java.lang.String sval, double nval)
Constructs a token of the indicated type and associated string or numeric values.

Method Summary

boolean	equals(java.lang.Object o) Returns true if the supplied object is an equivalent token.
boolean	equalsIgnoreCase(java.lang.Object o) Returns true if the supplied object is an equivalent token, given mellowness about case in strings and characters.
boolean	isNumber() Returns true if this token is a number.
boolean	isQuotedString() Returns true if this token is a quoted string.
boolean	isSymbol() Returns true if this token is a symbol.
boolean	isWord() Returns true if this token is a word.
double	nval() Returns the numeric value of this token.
java.lang.String	sval() Returns the string value of this token.
java.lang.String	toString() Return a textual description of this object.
TokenType	ttype() Returns the type of this token.
java.lang.Object	value() Returns an object that represents the value of this token.

Methods inherited from class java.lang.Object

, clone, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

ttype

protected TokenType ttype

sval

protected java.lang.String sval

nval

protected double nval

TT_EOF

public static final TokenType TT_EOF

A constant indicating that the end of the stream has been read.

EOF

public static final Token EOF

A constant indicating that there are no more tokens

TT_NUMBER

public static final TokenType TT_NUMBER

A constant indicating that a token is a number, like 3.14

TT_WORD

public static final TokenType TT_WORD

A constant indicating a token is a word, like "cat"

TT_SYMBOL

public static final TokenType TT_SYMBOL

A constant indicating that a token is a symbol like "<=".

TT_QUOTED

public static final TokenType TT_QUOTED

A constant indicating that a token is a quoted string, like "Launch Mi".

Constructor Detail

Token

public Token(char c)

Constructs a token from the given char.

Parameters:

char - the char

Token

public Token(double nval)

Constructs a token from the given number.

Parameters:

double - the number

Token

public Token(java.lang.String sval)

Constructs a token from the given string.

Parameters:

String - the string

Token

public Token(TokenType ttype,
             java.lang.String sval,
             double nval)

Constructs a token of the indicated type and associated string or numeric values.

Parameters:

TokenType - the type of the token, typically one of the constants this class defines

string - the string value of the token, typically null except for WORD and QUOTED tokens

double - the numeric value of the token, typically 0 except for NUMBER tokens

Method Detail

equals

public boolean equals(java.lang.Object o)

Returns true if the supplied object is an equivalent token.

Overrides:

equals in class java.lang.Object

Parameters:

object - the object to compare

Returns:

true, if the supplied object is of the same type and value

equalsIgnoreCase

public boolean equalsIgnoreCase(java.lang.Object o)

Returns true if the supplied object is an equivalent token, given mellowness about case in strings and characters.

Parameters:

object - the object to compare

Returns:

true, if the supplied object is of the same type and value. This method disregards case when comparing the string value of tokens.

isNumber

public boolean isNumber()

Returns true if this token is a number.

Returns:

true, if this token is a number

isQuotedString

public boolean isQuotedString()

Returns true if this token is a quoted string.

Returns:

true, if this token is a quoted string

isSymbol

public boolean isSymbol()

Returns true if this token is a symbol.

Returns:

true, if this token is a symbol

isWord

public boolean isWord()

Returns true if this token is a word.

Returns:

true, if this token is a word.

nval

public double nval()

Returns the numeric value of this token.

Returns:

the numeric value of this token

sval

public java.lang.String sval()

Returns the string value of this token.

Returns:

the string value of this token

toString

public java.lang.String toString()

Return a textual description of this object.

Overrides:

toString in class java.lang.Object

Returns:

a textual description of this object

ttype

public TokenType ttype()

Returns the type of this token.

Returns:

the type of this token, typically one of the constants this class defines

value

public java.lang.Object value()

Returns an object that represents the value of this token.

Returns:

an object that represents the value of this token