C# Класс Antlr4.Runtime.BufferedTokenStream

This implementation of ITokenStream loads tokens from a ITokenSource on-demand, and places the tokens in a buffer to provide access to any previous token by index.

This token stream ignores the value of IToken.Channel() . If your parser requires the token stream filter tokens to only those on a particular channel, such as TokenConstants.DefaultChannel or TokenConstants.HiddenChannel , use a filtering token stream such a CommonTokenStream .

Наследование: ITokenStream

Показать файл Открыть проект Примеры использования класса

Защищенные свойства (Protected)

Свойство	Тип	Описание
fetchedEOF	bool
p	int
tokens	IList

Открытые методы

Метод	Описание
BufferedTokenStream ( ITokenSource tokenSource ) : System
Consume ( ) : void
Fill ( ) : void	Get all tokens from lexer until EOF. Get all tokens from lexer until EOF.
Get ( int start, int stop ) : IList	Get all tokens from start..stop inclusively. Get all tokens from start..stop inclusively.
Get ( int i ) : IToken
GetHiddenTokensToLeft ( int tokenIndex ) : IList	Collect all hidden tokens (any off-default channel) to the left of the current token up until we see a token on Lexer.DefaultTokenChannel .
GetHiddenTokensToLeft ( int tokenIndex, int channel ) : IList	Collect all tokens on specified channel to the left of the current token up until we see a token on Lexer.DefaultTokenChannel . If channel is `-1` , find any non default channel token.
GetHiddenTokensToRight ( int tokenIndex ) : IList	Collect all hidden tokens (any off-default channel) to the right of the current token up until we see a token on Lexer.DefaultTokenChannel or EOF.
GetHiddenTokensToRight ( int tokenIndex, int channel ) : IList	Collect all tokens on specified channel to the right of the current token up until we see a token on Lexer.DefaultTokenChannel or EOF. If channel is `-1` , find any non default channel token.
GetTokens ( ) : IList
GetTokens ( int start, int stop ) : IList
GetTokens ( int start, int stop, BitSet types ) : IList	Given a start and stop index, return a `List` of all tokens in the token type `BitSet` . Return if no tokens were found. This method looks at both on and off channel tokens.
GetTokens ( int start, int stop, int ttype ) : IList
LA ( int i ) : int
Mark ( ) : int
Release ( int marker ) : void
Reset ( ) : void
Seek ( int index ) : void
SetTokenSource ( ITokenSource tokenSource ) : void	Reset this token stream by setting its token source. Reset this token stream by setting its token source.

Защищенные методы

Метод	Описание
AdjustSeekIndex ( int i ) : int	Allowed derived classes to modify the behavior of operations which change the current stream position by adjusting the target token index of a seek operation. Allowed derived classes to modify the behavior of operations which change the current stream position by adjusting the target token index of a seek operation. The default implementation simply returns i . If an exception is thrown in this method, the current stream index should not be changed. For example, CommonTokenStream overrides this method to ensure that the seek target is always an on-channel token.
Fetch ( int n ) : int	Add n elements to buffer.
FilterForChannel ( int from, int to, int channel ) : IList
LazyInit ( ) : void
Lb ( int k ) : IToken
NextTokenOnChannel ( int i, int channel ) : int	Given a starting index, return the index of the next token on channel. Given a starting index, return the index of the next token on channel. Return i if `tokens[i]` is on channel. Return the index of the EOF token if there are no tokens on channel between i and EOF.
PreviousTokenOnChannel ( int i, int channel ) : int	Given a starting index, return the index of the previous token on channel. Given a starting index, return the index of the previous token on channel. Return i if `tokens[i]` is on channel. Return -1 if there are no tokens on channel between i and 0. If i specifies an index at or after the EOF token, the EOF token index is returned. This is due to the fact that the EOF token is treated as though it were on every channel.
Setup ( ) : void
Sync ( int i ) : bool	Make sure index i in tokens has a token.

Приватные методы

Метод	Описание
GetText ( ) : string
GetText ( IToken start, IToken stop ) : string
GetText ( Interval interval ) : string
GetText ( RuleContext ctx ) : string
LT ( int k ) : IToken

Описание методов

AdjustSeekIndex() защищенный Метод

Allowed derived classes to modify the behavior of operations which change the current stream position by adjusting the target token index of a seek operation.

Allowed derived classes to modify the behavior of operations which change the current stream position by adjusting the target token index of a seek operation. The default implementation simply returns i . If an exception is thrown in this method, the current stream index should not be changed.

For example, CommonTokenStream overrides this method to ensure that the seek target is always an on-channel token.

protected AdjustSeekIndex ( int i ) : int
i	int	The target token index.
Результат	int

BufferedTokenStream() публичный Метод

public BufferedTokenStream ( ITokenSource tokenSource ) : System
tokenSource	ITokenSource
Результат	System

Consume() публичный Метод

public Consume ( ) : void
Результат	void

Fetch() защищенный Метод

Add n elements to buffer.

protected Fetch ( int n ) : int
n	int
Результат	int

Fill() публичный Метод

Get all tokens from lexer until EOF.

public Fill ( ) : void
Результат	void

FilterForChannel() защищенный Метод

protected FilterForChannel ( int from, int to, int channel ) : IList
from	int
to	int
channel	int
Результат	IList

Get() публичный Метод

Get all tokens from start..stop inclusively.

public Get ( int start, int stop ) : IList
start	int
stop	int
Результат	IList

Get() публичный Метод

public Get ( int i ) : IToken
i	int
Результат	IToken

GetHiddenTokensToLeft() публичный Метод

Collect all hidden tokens (any off-default channel) to the left of the current token up until we see a token on Lexer.DefaultTokenChannel .

public GetHiddenTokensToLeft ( int tokenIndex ) : IList
tokenIndex	int
Результат	IList

GetHiddenTokensToLeft() публичный Метод

Collect all tokens on specified channel to the left of the current token up until we see a token on Lexer.DefaultTokenChannel . If channel is -1 , find any non default channel token.

public GetHiddenTokensToLeft ( int tokenIndex, int channel ) : IList
tokenIndex	int
channel	int
Результат	IList

GetHiddenTokensToRight() публичный Метод

Collect all hidden tokens (any off-default channel) to the right of the current token up until we see a token on Lexer.DefaultTokenChannel or EOF.

public GetHiddenTokensToRight ( int tokenIndex ) : IList
tokenIndex	int
Результат	IList

GetHiddenTokensToRight() публичный Метод

Collect all tokens on specified channel to the right of the current token up until we see a token on Lexer.DefaultTokenChannel or EOF. If channel is -1 , find any non default channel token.

public GetHiddenTokensToRight ( int tokenIndex, int channel ) : IList
tokenIndex	int
channel	int
Результат	IList

GetTokens() публичный Метод

public GetTokens ( ) : IList
Результат	IList

GetTokens() публичный Метод

public GetTokens ( int start, int stop ) : IList
start	int
stop	int
Результат	IList

GetTokens() публичный Метод

Given a start and stop index, return a List of all tokens in the token type BitSet . Return if no tokens were found. This method looks at both on and off channel tokens.

public GetTokens ( int start, int stop, BitSet types ) : IList
start	int
stop	int
types	Antlr4.Runtime.Sharpen.BitSet
Результат	IList

GetTokens() публичный Метод

public GetTokens ( int start, int stop, int ttype ) : IList
start	int
stop	int
ttype	int
Результат	IList

LA() публичный Метод

public LA ( int i ) : int
i	int
Результат	int

LazyInit() защищенный Метод

protected LazyInit ( ) : void
Результат	void

Lb() защищенный Метод

protected Lb ( int k ) : IToken
k	int
Результат	IToken

Mark() публичный Метод

public Mark ( ) : int
Результат	int

NextTokenOnChannel() защищенный Метод

Given a starting index, return the index of the next token on channel.

Given a starting index, return the index of the next token on channel. Return i if tokens[i] is on channel. Return the index of the EOF token if there are no tokens on channel between i and EOF.

protected NextTokenOnChannel ( int i, int channel ) : int
i	int
channel	int
Результат	int

PreviousTokenOnChannel() защищенный Метод

Given a starting index, return the index of the previous token on channel.

Given a starting index, return the index of the previous token on channel. Return i if tokens[i] is on channel. Return -1 if there are no tokens on channel between i and 0.

If i specifies an index at or after the EOF token, the EOF token index is returned. This is due to the fact that the EOF token is treated as though it were on every channel.

protected PreviousTokenOnChannel ( int i, int channel ) : int
i	int
channel	int
Результат	int

Release() публичный Метод

public Release ( int marker ) : void
marker	int
Результат	void

Reset() публичный Метод

public Reset ( ) : void
Результат	void

Seek() публичный Метод

public Seek ( int index ) : void
index	int
Результат	void

SetTokenSource() публичный Метод

Reset this token stream by setting its token source.

public SetTokenSource ( ITokenSource tokenSource ) : void
tokenSource	ITokenSource
Результат	void

Setup() защищенный Метод

protected Setup ( ) : void
Результат	void

Sync() защищенный Метод

Make sure index i in tokens has a token.

protected Sync ( int i ) : bool
i	int
Результат	bool

Описание свойств

fetchedEOF защищенное свойство

Indicates whether the TokenConstants.EOF token has been fetched from _tokenSource and added to tokens . This field improves performance for the following cases:

Consume() : The lookahead check in Consume() to prevent consuming the EOF symbol is optimized by checking the values of fetchedEOF and p instead of calling LA(int) .
Fetch(int) : The check to prevent adding multiple EOF symbols into tokens is trivial with this field.

protected bool fetchedEOF
Результат	bool

p защищенное свойство

The index into tokens of the current token (next token to Consume() ). tokens [ p ] should be LT(1) .

This field is set to -1 when the stream is first constructed or when SetTokenSource(ITokenSource) is called, indicating that the first token has not yet been fetched from the token source. For additional information, see the documentation of IIntStream for a description of Initializing Methods.

protected int p
Результат	int

tokens защищенное свойство

A collection of all tokens fetched from the token source.

A collection of all tokens fetched from the token source. The list is considered a complete view of the input once fetchedEOF is set to .

protected IList tokens
Результат	IList