C# Class Antlr4.Runtime.CommonTokenStream

This class extends BufferedTokenStream with functionality to filter token streams to tokens on a particular channel (tokens where IToken.Channel() returns a particular value).

This token stream provides access to all tokens by index or when calling methods like BufferedTokenStream.GetText() . The channel filtering is only used for code accessing tokens via the lookahead methods BufferedTokenStream.La(int) , Lt(int) , and Lb(int) .

By default, tokens are placed on the default channel ( TokenConstants.DefaultChannel ), but may be reassigned by using the ->channel(HIDDEN) lexer command, or by using an embedded action to call Lexer.Channel .

Note: lexer rules which use the ->skip lexer command or call Lexer.Skip() do not produce tokens at all, so input text matched by such a rule will not be available as part of the token stream, regardless of channel.

Inheritance: BufferedTokenStream

Show file Open project: sharwell/antlr4cs Class Usage Examples

Protected Properties

Property	Type	Description
channel	int

Public Methods

Method	Description
CommonTokenStream ( ITokenSource tokenSource ) : Antlr4.Runtime.Sharpen	Constructs a new CommonTokenStream using the specified token source and the default token channel ( TokenConstants.DefaultChannel ).
CommonTokenStream ( ITokenSource tokenSource, int channel ) : Antlr4.Runtime.Sharpen	Constructs a new CommonTokenStream using the specified token source and filtering tokens to the specified channel. Only tokens whose IToken.Channel() matches channel or have the IToken.Type() equal to TokenConstants.Eof will be returned by the token stream lookahead methods.
GetNumberOfOnChannelTokens ( ) : int	Count EOF just once.
Lt ( int k ) : IToken

Protected Methods

Method	Description
AdjustSeekIndex ( int i ) : int
Lb ( int k ) : IToken

Method Details

AdjustSeekIndex() protected method

protected AdjustSeekIndex ( int i ) : int
i	int
return	int

CommonTokenStream() public method

Constructs a new CommonTokenStream using the specified token source and the default token channel ( TokenConstants.DefaultChannel ).

public CommonTokenStream ( ITokenSource tokenSource ) : Antlr4.Runtime.Sharpen
tokenSource	ITokenSource	The token source.
return	Antlr4.Runtime.Sharpen

CommonTokenStream() public method

Constructs a new CommonTokenStream using the specified token source and filtering tokens to the specified channel. Only tokens whose IToken.Channel() matches channel or have the IToken.Type() equal to TokenConstants.Eof will be returned by the token stream lookahead methods.

public CommonTokenStream ( ITokenSource tokenSource, int channel ) : Antlr4.Runtime.Sharpen
tokenSource	ITokenSource	The token source.
channel	int	The channel to use for filtering tokens.
return	Antlr4.Runtime.Sharpen

GetNumberOfOnChannelTokens() public method

Count EOF just once.

public GetNumberOfOnChannelTokens ( ) : int
return	int

Lb() protected method

protected Lb ( int k ) : IToken
k	int
return	IToken

Lt() public method

public Lt ( int k ) : IToken
k	int
return	IToken

Property Details

channel protected property

Specifies the channel to use for filtering tokens.

The default value is TokenConstants.DefaultChannel , which matches the default channel assigned to tokens created by the lexer.

protected int channel
return	int