C# Class Antlr.Runtime.Lexer

* A lexer is recognizer that draws input symbols from a character stream. * lexer grammars result in a subclass of this object. A Lexer object * uses simplified match() and error recovery mechanisms in the interest * of speed. *
Inheritance: BaseRecognizer, ITokenSource
Show file Open project: antlr/antlrcs

Protected Properties

Property Type Description
input ICharStream

Public Methods

Method Description
Emit ( ) : IToken

* The standard method called to automatically emit a token at the * outermost lexical rule. The token object should point into the * char buffer start..stop. If there is a text override in 'text', * use that to set the token's text. Override this method to emit * custom Token objects. *

* If you are building trees, then you should also override * Parser or TreeParser.getMissingSymbol(). *

Emit ( IToken token ) : void

* Currently does not support multiple emits per nextToken invocation * for efficiency reasons. Subclass and override this method and * nextToken (to push tokens into a list and pull from that list rather * than a single variable as this implementation does). *

GetCharErrorDisplay ( int c ) : string
GetEndOfFileToken ( ) : IToken
GetErrorMessage ( RecognitionException e, string tokenNames ) : string
Lexer ( )
Lexer ( ICharStream input )
Lexer ( ICharStream input, Antlr.Runtime.RecognizerSharedState state )
Match ( int c ) : void
Match ( string s ) : void
MatchAny ( ) : void
MatchRange ( int a, int b ) : void
NextToken ( ) : IToken

Return a token from this source; i.e., match a token on the char stream.

Recover ( RecognitionException re ) : void

* Lexers can normally match any char in it's vocabulary after matching * a token, so do the easy thing and just kill a character and hope * it all works out. You can instead use the rule invocation stack * to do sophisticated error recovery if you are in a fragment rule. *

ReportError ( RecognitionException e ) : void
Reset ( ) : void
Skip ( ) : void

* Instruct the lexer to skip creating a token for current lexer rule * and look for another token. nextToken() knows to keep looking when * a lexer rule finishes with token set to SKIP_TOKEN. Recall that * if token==null at end of any token rule, it creates one for you * and emits it. *

mTokens ( ) : void

This is the lexer entry point that sets instance var 'token'

Protected Methods

Method Description
ParseNextToken ( ) : void

Private Methods

Method Description
TraceIn ( string ruleName, int ruleIndex ) : void
TraceOut ( string ruleName, int ruleIndex ) : void

Method Details

Emit() public method

* The standard method called to automatically emit a token at the * outermost lexical rule. The token object should point into the * char buffer start..stop. If there is a text override in 'text', * use that to set the token's text. Override this method to emit * custom Token objects. *
* If you are building trees, then you should also override * Parser or TreeParser.getMissingSymbol(). *
public Emit ( ) : IToken
return IToken

Emit() public method

* Currently does not support multiple emits per nextToken invocation * for efficiency reasons. Subclass and override this method and * nextToken (to push tokens into a list and pull from that list rather * than a single variable as this implementation does). *
public Emit ( IToken token ) : void
token IToken
return void

GetCharErrorDisplay() public method

public GetCharErrorDisplay ( int c ) : string
c int
return string

GetEndOfFileToken() public method

public GetEndOfFileToken ( ) : IToken
return IToken

GetErrorMessage() public method

public GetErrorMessage ( RecognitionException e, string tokenNames ) : string
e RecognitionException
tokenNames string
return string

Lexer() public method

public Lexer ( )

Lexer() public method

public Lexer ( ICharStream input )
input ICharStream

Lexer() public method

public Lexer ( ICharStream input, Antlr.Runtime.RecognizerSharedState state )
input ICharStream
state Antlr.Runtime.RecognizerSharedState

Match() public method

public Match ( int c ) : void
c int
return void

Match() public method

public Match ( string s ) : void
s string
return void

MatchAny() public method

public MatchAny ( ) : void
return void

MatchRange() public method

public MatchRange ( int a, int b ) : void
a int
b int
return void

NextToken() public method

Return a token from this source; i.e., match a token on the char stream.
public NextToken ( ) : IToken
return IToken

ParseNextToken() protected method

protected ParseNextToken ( ) : void
return void

Recover() public method

* Lexers can normally match any char in it's vocabulary after matching * a token, so do the easy thing and just kill a character and hope * it all works out. You can instead use the rule invocation stack * to do sophisticated error recovery if you are in a fragment rule. *
public Recover ( RecognitionException re ) : void
re RecognitionException
return void

ReportError() public method

public ReportError ( RecognitionException e ) : void
e RecognitionException
return void

Reset() public method

public Reset ( ) : void
return void

Skip() public method

* Instruct the lexer to skip creating a token for current lexer rule * and look for another token. nextToken() knows to keep looking when * a lexer rule finishes with token set to SKIP_TOKEN. Recall that * if token==null at end of any token rule, it creates one for you * and emits it. *
public Skip ( ) : void
return void

mTokens() public abstract method

This is the lexer entry point that sets instance var 'token'
public abstract mTokens ( ) : void
return void

Property Details

input protected property

Where is the lexer drawing characters from?
protected ICharStream input
return ICharStream