sjm.parse.tokens
Class WordState
java.lang.Object
|
+--sjm.parse.tokens.TokenizerState
|
+--sjm.parse.tokens.WordState
- public class WordState
- extends TokenizerState
A wordState returns a word from a reader. Like other
states, a tokenizer transfers the job of reading to this
state, depending on an initial character. Thus, the
tokenizer decides which characters may begin a word, and
this state determines which characters may appear as a
second or later character in a word. These are typically
different sets of characters; in particular, it is typical
for digits to appear as parts of a word, but not as the
initial character of a word.
By default, the following characters may appear in a word.
The method setWordChars()
allows customizing
this.
From To
'a', 'z'
'A', 'Z'
'0', '9'
as well as: minus sign, underscore, and apostrophe.
- Version:
- 1.0
- Author:
- Steven J. Metsker
Constructor Summary |
WordState()
Constructs a word state with a default idea of what
characters are admissible inside a word (as described in
the class comment). |
Method Summary |
protected void |
checkBufLength(int i)
|
Token |
nextToken(java.io.PushbackReader r,
int c,
Tokenizer t)
Return a word token from a reader. |
void |
setWordChars(int from,
int to,
boolean b)
Establish characters in the given range as valid
characters for part of a word after the first character.
|
protected boolean |
wordChar(int c)
|
Methods inherited from class java.lang.Object |
, clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait |
charbuf
protected char[] charbuf
wordChar
protected boolean[] wordChar
WordState
public WordState()
- Constructs a word state with a default idea of what
characters are admissible inside a word (as described in
the class comment).
checkBufLength
protected void checkBufLength(int i)
nextToken
public Token nextToken(java.io.PushbackReader r,
int c,
Tokenizer t)
throws java.io.IOException
- Return a word token from a reader.
- Overrides:
nextToken
in class TokenizerState
- Returns:
- a word token from a reader
setWordChars
public void setWordChars(int from,
int to,
boolean b)
- Establish characters in the given range as valid
characters for part of a word after the first character.
Note that the tokenizer must determine which characters
are valid as the beginning character of a word.
- Parameters:
from
- charto
- charboolean
- true, if this state should allow
characters in the given range as part
of a word
wordChar
protected boolean wordChar(int c)