| Name | Description |
|---|---|
| CharBuffer | Buffer for characters. This approximates StringBuilder but is designed to be faster for specific operations. This is about 30% faster for the operations I'm interested in (Append, Clear, Length, ToString). This trades off memory for speed. |
| CharToken | Token type for characters, meaning non-word characters. |
| CommentToken | Token type for comments, including line and block comments. |
| EofToken | Represents end of file/stream. |
| EolToken | Represents end-of-lines (line separator characters). |
| FloatToken | Token type for floating point numbers, stored internally as a Double. |
| HighResClock | This provides access to the Kernel32.dll high resolution clock API. This is motivated by the need to have higher resolution than .Net's DateTime.Now, which is apparently about 10ms. The effective resolution of HighResClock on a P4 1.4GHz is about 10us. System.TimeSpan System.DateTime |
| IntToken | Token type for integer tokens. This handles both Int32 and Int64. |
| Logger | This is a simple implementation of a Logger class. The purpose is to threshold output based on a verbosity setting, format messages similarly, and collect all message writes such that they can be redirected easily. You (and I) should probably use the Apache Software Foundation's log4net instead of this class. |
| QuoteToken | Token type for Quotes such as "this is a quote". |
| StreamTokenizer | A StreamTokenizer similar to Java's. This breaks an input stream (coming from a TextReader) into Tokens based on various settings. The settings are stored in the TokenizerSettings property, which is a StreamTokenizerSettings instance. |
| StreamTokenizerSettings | |
| StreamTokenizerUntermCommentException | Exception class for unterminated block comments. |
| StreamTokenizerUntermException | Exception class for unterminated tokens. |
| StreamTokenizerUntermQuoteException | Exception class for unterminated quotes. |
| StringToken | Abstract base class for string tokens. |
| Token | Token class used by StreamTokenizer. This represents a single token in the input stream. This is subclassed to provide specific token types, such as CharToken, FloatToken, etc. |
| WhitespaceToken | Token type for whitespace such as spaces and tabs. |
| WordToken | Token type for words, meaning sequences of word characters. |
