sjm.parse.tokens
Class WordState

java.lang.Object
  |
  +--sjm.parse.tokens.TokenizerState
        |
        +--sjm.parse.tokens.WordState

public class WordState
extends TokenizerState

A wordState returns a word from a reader. Like other states, a tokenizer transfers the job of reading to this state, depending on an initial character. Thus, the tokenizer decides which characters may begin a word, and this state determines which characters may appear as a second or later character in a word. These are typically different sets of characters; in particular, it is typical for digits to appear as parts of a word, but not as the initial character of a word.

By default, the following characters may appear in a word. The method setWordChars() allows customizing this.

     From    To
      'a', 'z'
      'A', 'Z'
      '0', '9'

     as well as: minus sign, underscore, and apostrophe.
 
 

Version:
1.0
Author:
Steven J. Metsker

Field Summary
protected  char[] charbuf
           
protected  boolean[] wordChar
           
 
Constructor Summary
WordState()
          Constructs a word state with a default idea of what characters are admissible inside a word (as described in the class comment).
 
Method Summary
protected  void checkBufLength(int i)
           
 Token nextToken(java.io.PushbackReader r, int c, Tokenizer t)
          Return a word token from a reader.
 void setWordChars(int from, int to, boolean b)
          Establish characters in the given range as valid characters for part of a word after the first character.
protected  boolean wordChar(int c)
           
 
Methods inherited from class java.lang.Object
, clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

charbuf

protected char[] charbuf

wordChar

protected boolean[] wordChar
Constructor Detail

WordState

public WordState()
Constructs a word state with a default idea of what characters are admissible inside a word (as described in the class comment).
Method Detail

checkBufLength

protected void checkBufLength(int i)

nextToken

public Token nextToken(java.io.PushbackReader r,
                       int c,
                       Tokenizer t)
                throws java.io.IOException
Return a word token from a reader.
Overrides:
nextToken in class TokenizerState
Returns:
a word token from a reader

setWordChars

public void setWordChars(int from,
                         int to,
                         boolean b)
Establish characters in the given range as valid characters for part of a word after the first character. Note that the tokenizer must determine which characters are valid as the beginning character of a word.
Parameters:
from - char
to - char
boolean - true, if this state should allow characters in the given range as part of a word

wordChar

protected boolean wordChar(int c)