Uses of Package
sjm.parse.tokens

Packages that use sjm.parse.tokens

org.jsesoft.jpp	org.jsesoft.jpp: A Java macro preprocessor Title: A Java macro preprocessor.
sjm.parse.tokens

Classes in sjm.parse.tokens used by org.jsesoft.jpp

Token
A token represents a logical chunk of a string.

Tokenizer
A tokenizer divides a string into tokens.

TokenType
Objects of this class represent a type of token, such as "number" or "word".

Classes in sjm.parse.tokens used by sjm.parse.tokens

Literal
A Literal matches a specific String from an assembly.

NumberState
A NumberState object returns a number from a reader.

QuoteState
A quoteState returns a quoted string token from a reader.

SlashSlashState
A slashSlash state ignores everything up to an end-of-line and returns the tokenizer's next token.

SlashStarState
A slashStar state ignores everything up to a closing star and slash, and then returns the tokenizer's next token.

SlashState
This state will either delegate to a comment-handling state, or return a token with just a slash in it.

SymbolNode
A SymbolNode object is a member of a tree that contains all possible prefixes of allowable symbols.

SymbolRootNode
This class is a special case of a SymbolNode.

SymbolState
The idea of a symbol is a character that stands on its own, such as an ampersand or a parenthesis.

Token
A token represents a logical chunk of a string.

Tokenizer
A tokenizer divides a string into tokens.

TokenizerState
A tokenizerState returns a token, given a reader, an initial character read from the reader, and a tokenizer that is conducting an overall tokenization of the reader.

TokenString
A TokenString is like a String, but it is a series of Tokens rather than a series of chars.

WhitespaceState
A whitespace state ignores whitespace (such as blanks and tabs), and returns the tokenizer's next token.

WordState
A wordState returns a word from a reader.