C# Class Fanx.Serial.Tokenizer

Tokenizer inputs a stream of Unicode characters and outputs tokens for the Fantom serialization grammar.
Afficher le fichier Open project: xored/f4 Class Usage Examples

Méthodes publiques

Свойство Type Description
m_in Fan.Sys.InStream
m_line int
m_type int
m_val object

Méthodes publiques

Méthode Description
Tokenizer ( InStream @in ) : System.Text

Construct for specified input stream.
err ( string msg ) : System.Exception
next ( ) : int

Read the next token from the stream. The token is available via the 'type' and 'val' fields. The line of the current token is available in 'line' field. Return the 'type' field or -1 if at end of stream.
reset ( int type, object val, int line ) : int

Reset the current token state.
undo ( int type, object val, int line ) : void

Pushback a token which will be the next read.

Private Methods

Méthode Description
Tokenizer ( ) : System.Text
ch ( ) : int

Parse a char literal token (as Long literal).
consume ( ) : void
doNext ( ) : int

Read the next token, set the 'val' field but return type without worrying setting the 'type' field.
escape ( ) : char

Parse an escapse sequence which starts with a \
hex ( ) : int

Process hex int/long literal starting with 0x
hex ( int c ) : int
id ( ) : int

Parse an identifier: alpha (alpha|number)*
number ( bool neg ) : int

Parse a number literal token.
skipCommentML ( ) : Token

Skip a multi line /* comment. Note unlike C/Java, slash/star comments can be nested.
skipCommentSL ( ) : Token

Skip a single line // comment
str ( ) : int

Parse a string literal token.
uri ( ) : int

Parse a uri literal token.

Method Details

Tokenizer() public méthode

Construct for specified input stream.
public Tokenizer ( InStream @in ) : System.Text
@in Fan.Sys.InStream
Résultat System.Text

err() public méthode

public err ( string msg ) : System.Exception
msg string
Résultat System.Exception

next() public méthode

Read the next token from the stream. The token is available via the 'type' and 'val' fields. The line of the current token is available in 'line' field. Return the 'type' field or -1 if at end of stream.
public next ( ) : int
Résultat int

reset() public méthode

Reset the current token state.
public reset ( int type, object val, int line ) : int
type int
val object
line int
Résultat int

undo() public méthode

Pushback a token which will be the next read.
public undo ( int type, object val, int line ) : void
type int
val object
line int
Résultat void

Property Details

m_in public_oe property

public InStream,Fan.Sys m_in
Résultat Fan.Sys.InStream

m_line public_oe property

public int m_line
Résultat int

m_type public_oe property

public int m_type
Résultat int

m_val public_oe property

public object m_val
Résultat object