• Home
• Schemin
• Tokenize
• Tokenizer

C# Class Schemin.Tokenize.Tokenizer

显示文件 Open project: imphasing/schemin Class Usage Examples

Public Methods

Method	Description
Tokenize ( string input ) : List

Private Methods

Method	Description
GetColNumber ( char input, int position ) : int
GetLineNumber ( char input, int position ) : int
GetNextToken ( char input, int position ) : int>.KeyValuePair
MakeTokenPair ( TokenType type, String tokenName, char input, int beginPosition, int endPosition ) : int>.KeyValuePair
MapEscaped ( char escaped ) : char
Newline ( char candidate ) : bool
NumberLiteral ( char input, int position ) : int>.KeyValuePair
Numerical ( char candidate ) : bool
NumericalPart ( char candidate ) : bool
QuoteSugar ( char input, int position ) : int>.KeyValuePair
StringLiteral ( char input, int position ) : int>.KeyValuePair
Symbol ( char input, int position ) : int>.KeyValuePair
SymbolPart ( char candidate ) : bool
VectorOrBooleanOrChar ( char input, int position ) : int>.KeyValuePair
Whitespace ( char candidate ) : bool

Method Details