|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: INNER | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |
java.lang.Object | +--sjm.parse.tokens.Tokenizer | +--org.jsesoft.jpp.Tokenizer
Customizable tokenizer fitting Java requirements.
Field Summary | |
(package private) CommentSlashState |
commentSlashState
The commentSlash state. |
protected static int |
DEFAULT_SYMBOL_MAX
|
(package private) DefaultState |
defaultState
The default state. |
(package private) IdentifierState |
identifierState
The identifier state. |
(package private) KeywordIdentifierState |
keywordIdentifierState
The keyword state. |
(package private) NumberState |
numberState
The number state. |
(package private) OperatorState |
operatorState
The operator state. |
(package private) QuotedState |
quotedState
The quoted state. |
(package private) SeparatorState |
separatorState
The separator state. |
(package private) WhitespaceState |
whitespaceState
The whitespace state. |
Fields inherited from class sjm.parse.tokens.Tokenizer |
characterState, quoteState, reader, slashState, symbolState, wordState |
Constructor Summary | |
Tokenizer()
|
|
Tokenizer(java.lang.String string)
Constructs a tokenizer to read from the supplied string. |
Method Summary | |
CommentSlashState |
getCommentSlashState()
Returns the commentSlash state of the tokenizer. |
DefaultState |
getDefaultState()
Returns the default state of the tokenizer. |
IdentifierState |
getIdentifierState()
Returns the identifier state of the tokenizer. |
KeywordIdentifierState |
getKeywordIdentifierState()
Returns the keyword state of the tokenizer. |
NumberState |
getNumberState()
Returns the number state of the tokenizer. |
OperatorState |
getOperatorState()
Returns the operator state of the tokenizer. |
QuotedState |
getQuotedState()
Returns the quoted state of the tokenizer. |
SeparatorState |
getSeparatorState()
Returns the separator state of the tokenizer. |
WhitespaceState |
getWhitespaceState()
Returns the whitespace state of the tokenizer. |
boolean |
isIdentifierStart(char character)
Determines if the character may be at the start of an identifier. |
boolean |
isNumberStart(char character)
Determines if the character may be at the start of a number. |
boolean |
isWhitespace(char character)
Determines if the character is a whitespace. |
static void |
main(java.lang.String[] args)
|
Token |
nextToken()
Scans the next token from the reader. |
void |
setCommentSlashState(CommentSlashState value)
Specifies the commentSlash state of the tokenizer. |
void |
setDefaultState(DefaultState value)
Specifies the default state of the tokenizer. |
void |
setIdentifierState(IdentifierState value)
Specifies the identifier state of the tokenizer. |
void |
setKeywordIdentifierState(KeywordIdentifierState value)
Specifies the keyword state of the tokenizer. |
void |
setNumberState(NumberState value)
Specifies the number state of the tokenizer. |
void |
setOperatorState(OperatorState value)
Specifies the operator state of the tokenizer. |
void |
setQuotedtState(QuotedState value)
Specifies the quoted state of the tokenizer. |
void |
setSeparatorState(SeparatorState value)
Specifies the separator state of the tokenizer. |
void |
setString(java.lang.String string)
Set the string to read from. |
void |
setWhitespaceState(WhitespaceState value)
Specifies the whitespace state of the tokenizer. |
Methods inherited from class sjm.parse.tokens.Tokenizer |
getReader, numberState, quoteState, setCharacterState, setReader, setString, slashState, symbolState, whitespaceState, wordState |
Methods inherited from class java.lang.Object |
|
Field Detail |
protected static final int DEFAULT_SYMBOL_MAX
IdentifierState identifierState
OperatorState operatorState
SeparatorState separatorState
WhitespaceState whitespaceState
NumberState numberState
QuotedState quotedState
KeywordIdentifierState keywordIdentifierState
CommentSlashState commentSlashState
DefaultState defaultState
Constructor Detail |
public Tokenizer()
public Tokenizer(java.lang.String string)
string
- the string to read fromMethod Detail |
public Token nextToken() throws java.io.IOException
nextToken
in class Tokenizer
java.io.IOException
- if there is any problem readingpublic IdentifierState getIdentifierState()
public void setIdentifierState(IdentifierState value)
value
- the identifier statepublic OperatorState getOperatorState()
public void setOperatorState(OperatorState value)
value
- the operator statepublic SeparatorState getSeparatorState()
public void setSeparatorState(SeparatorState value)
value
- the separator statepublic WhitespaceState getWhitespaceState()
public void setWhitespaceState(WhitespaceState value)
value
- the whitespace statepublic NumberState getNumberState()
public void setNumberState(NumberState value)
value
- the number statepublic QuotedState getQuotedState()
public void setQuotedtState(QuotedState value)
value
- the quoted statepublic KeywordIdentifierState getKeywordIdentifierState()
public void setKeywordIdentifierState(KeywordIdentifierState value)
value
- the keyword statepublic CommentSlashState getCommentSlashState()
public void setCommentSlashState(CommentSlashState value)
value
- the commentSlash statepublic DefaultState getDefaultState()
public void setDefaultState(DefaultState value)
value
- the default statepublic boolean isIdentifierStart(char character)
character
- the character to checkCharacter.isJavaIdentifierStart(char)
public boolean isWhitespace(char character)
character
- the character to checkCharacter.isWhitespace(char)
public boolean isNumberStart(char character)
character
- the character to checkCharacter.isDigit(char)
public void setString(java.lang.String string)
setString
in class Tokenizer
string
- the string to read frompublic static void main(java.lang.String[] args)
|
||||||||||
PREV CLASS NEXT CLASS | FRAMES NO FRAMES | |||||||||
SUMMARY: INNER | FIELD | CONSTR | METHOD | DETAIL: FIELD | CONSTR | METHOD |