Name | Description |
---|---|
Token | Represents a lexical token of the language. This is an abstract class that particular language implementations must extend to define language-specific token types |
TokenList | A list of tokens |
Tokenized | Provides read-only view over TokenList |