C# Class ComponentFactory.Quicksilver.Binding.Tokeniser

Converts a string into a list of lexical tokens.

Show file Open project: ComponentFactory/Quicksilver Class Usage Examples

Public Methods

Method	Description
Next ( ) : Token	Gets the next token and removes it.
Next ( TokenType type ) : bool	If the next token matchs the requested type then just eat it.
Tokeniser ( string input, System.Language language ) : System	Initialize a new instance of the Tokeniser class.

Private Methods

Method	Description
NextCharOrStringLiteral ( char chars, int &index, bool escaped ) : Token
NextHexLiteral ( char chars, int start, int &index ) : Token
NextIdentifier ( char chars, int &index ) : Token
NextIntegerLiteral ( char chars, int start, int &index, int numberBase ) : Token
NextIntegerLiteralCS ( char chars, int start, int &index, int numberBase ) : Token
NextIntegerLiteralVB ( char chars, int start, int &index, int numberBase ) : Token
NextNumberCodedLiteralVB ( char chars, int &index, int numberBase ) : Token
NextNumberLiteral ( char chars, int &index ) : Token
NextRealLiteral ( char chars, int start, int &index ) : Token
NextToken ( char chars, int &index ) : Token
Tokeniser ( ) : System

Method Details

Next() public method

Gets the next token and removes it.

public Next ( ) : Token
return	Token

Next() public method

If the next token matchs the requested type then just eat it.

public Next ( TokenType type ) : bool
type	TokenType	Type to eat if found.
return	bool

Tokeniser() public method

Initialize a new instance of the Tokeniser class.

public Tokeniser ( string input, System.Language language ) : System
input	string	String to convert into a series of tokens.
language	System.Language	Language used for parsing.
return	System