C# Class Antlr.Runtime.Lexer

* A lexer is recognizer that draws input symbols from a character stream. * lexer grammars result in a subclass of this object. A Lexer object * uses simplified match() and error recovery mechanisms in the interest * of speed. *

Inheritance: BaseRecognizer, ITokenSource

Show file Open project: antlr/antlrcs

Protected Properties

Property	Type	Description
input	ICharStream

Public Methods

Method	Description
Emit ( ) : IToken	* The standard method called to automatically emit a token at the * outermost lexical rule. The token object should point into the * char buffer start..stop. If there is a text override in 'text', * use that to set the token's text. Override this method to emit * custom Token objects. * * If you are building trees, then you should also override * Parser or TreeParser.getMissingSymbol(). *
Emit ( IToken token ) : void	* Currently does not support multiple emits per nextToken invocation * for efficiency reasons. Subclass and override this method and * nextToken (to push tokens into a list and pull from that list rather * than a single variable as this implementation does). *
GetCharErrorDisplay ( int c ) : string
GetEndOfFileToken ( ) : IToken
GetErrorMessage ( RecognitionException e, string tokenNames ) : string
Lexer ( )
Lexer ( ICharStream input )
Lexer ( ICharStream input, Antlr.Runtime.RecognizerSharedState state )
Match ( int c ) : void
Match ( string s ) : void
MatchAny ( ) : void
MatchRange ( int a, int b ) : void
NextToken ( ) : IToken	Return a token from this source; i.e., match a token on the char stream.
Recover ( RecognitionException re ) : void	* Lexers can normally match any char in it's vocabulary after matching * a token, so do the easy thing and just kill a character and hope * it all works out. You can instead use the rule invocation stack * to do sophisticated error recovery if you are in a fragment rule. *
ReportError ( RecognitionException e ) : void
Reset ( ) : void
Skip ( ) : void	* Instruct the lexer to skip creating a token for current lexer rule * and look for another token. nextToken() knows to keep looking when * a lexer rule finishes with token set to SKIP_TOKEN. Recall that * if token==null at end of any token rule, it creates one for you * and emits it. *
mTokens ( ) : void	This is the lexer entry point that sets instance var 'token'

Protected Methods

Method	Description
ParseNextToken ( ) : void

Private Methods

Method	Description
TraceIn ( string ruleName, int ruleIndex ) : void
TraceOut ( string ruleName, int ruleIndex ) : void

Method Details

Emit() public method

* The standard method called to automatically emit a token at the * outermost lexical rule. The token object should point into the * char buffer start..stop. If there is a text override in 'text', * use that to set the token's text. Override this method to emit * custom Token objects. *

* If you are building trees, then you should also override * Parser or TreeParser.getMissingSymbol(). *

public Emit ( ) : IToken
return	IToken

Emit() public method

* Currently does not support multiple emits per nextToken invocation * for efficiency reasons. Subclass and override this method and * nextToken (to push tokens into a list and pull from that list rather * than a single variable as this implementation does). *

public Emit ( IToken token ) : void
token	IToken
return	void

GetCharErrorDisplay() public method

public GetCharErrorDisplay ( int c ) : string
c	int
return	string

GetEndOfFileToken() public method

public GetEndOfFileToken ( ) : IToken
return	IToken

GetErrorMessage() public method

public GetErrorMessage ( RecognitionException e, string tokenNames ) : string
e	RecognitionException
tokenNames	string
return	string

Lexer() public method

public Lexer ( )

Lexer() public method

public Lexer ( ICharStream input )
input	ICharStream

Lexer() public method

public Lexer ( ICharStream input, Antlr.Runtime.RecognizerSharedState state )
input	ICharStream
state	Antlr.Runtime.RecognizerSharedState

Match() public method

public Match ( int c ) : void
c	int
return	void

Match() public method

public Match ( string s ) : void
s	string
return	void

MatchAny() public method

public MatchAny ( ) : void
return	void

MatchRange() public method

public MatchRange ( int a, int b ) : void
a	int
b	int
return	void

NextToken() public method

Return a token from this source; i.e., match a token on the char stream.

public NextToken ( ) : IToken
return	IToken

ParseNextToken() protected method

protected ParseNextToken ( ) : void
return	void

Recover() public method

* Lexers can normally match any char in it's vocabulary after matching * a token, so do the easy thing and just kill a character and hope * it all works out. You can instead use the rule invocation stack * to do sophisticated error recovery if you are in a fragment rule. *

public Recover ( RecognitionException re ) : void
re	RecognitionException
return	void

ReportError() public method

public ReportError ( RecognitionException e ) : void
e	RecognitionException
return	void

Reset() public method

public Reset ( ) : void
return	void

Skip() public method

* Instruct the lexer to skip creating a token for current lexer rule * and look for another token. nextToken() knows to keep looking when * a lexer rule finishes with token set to SKIP_TOKEN. Recall that * if token==null at end of any token rule, it creates one for you * and emits it. *

public Skip ( ) : void
return	void

mTokens() public abstract method

This is the lexer entry point that sets instance var 'token'

public abstract mTokens ( ) : void
return	void

Property Details

input protected property

Where is the lexer drawing characters from?

protected ICharStream input
return	ICharStream