C# Class Antlr4.Runtime.UnbufferedTokenStream

Inheritance: ITokenStream

Protected Properties

Property	Type	Description
currentTokenIndex	int
lastToken	IToken
lastTokenBufferStart	IToken
n	int
numMarkers	int
p	int
tokens	IToken[]

Public Methods

Method	Description
Consume ( ) : void
Get ( int i ) : IToken
LA ( int i ) : int
LT ( int i ) : IToken
Mark ( ) : int	Return a marker that we can release later. Return a marker that we can release later. The specific marker value used for this class allows for some level of protection against misuse where `seek()` is called on a mark or `release()` is called in the wrong order.
Release ( int marker ) : void
Seek ( int index ) : void
UnbufferedTokenStream ( ITokenSource tokenSource ) : System
UnbufferedTokenStream ( ITokenSource tokenSource, int bufferSize ) : System

Protected Methods

Method	Description
Add ( IToken t ) : void
Fill ( int n ) : int	Add n elements to the buffer. Returns the number of tokens actually added to the buffer. If the return value is less than n , then EOF was reached before n tokens could be added.
GetBufferStartIndex ( ) : int
Sync ( int want ) : void	Make sure we have 'need' elements from current position p . Last valid `p` index is `tokens.length-1` . `p+need-1` is the tokens index 'need' elements ahead. If we need 1 element, `(p+1-1)==p` must be less than `tokens.length` .

Private Methods

Method	Description
GetText ( ) : string
GetText ( IToken start, IToken stop ) : string
GetText ( Interval interval ) : string
GetText ( RuleContext ctx ) : string

Method Details

Add() protected method

protected Add ( IToken t ) : void
t	IToken
return	void

Consume() public method

public Consume ( ) : void
return	void

Fill() protected method

Add n elements to the buffer. Returns the number of tokens actually added to the buffer. If the return value is less than n , then EOF was reached before n tokens could be added.

protected Fill ( int n ) : int
n	int
return	int

Get() public method

public Get ( int i ) : IToken
i	int
return	IToken

GetBufferStartIndex() protected method

protected GetBufferStartIndex ( ) : int
return	int

LA() public method

public LA ( int i ) : int
i	int
return	int

LT() public method

public LT ( int i ) : IToken
i	int
return	IToken

Mark() public method

Return a marker that we can release later.

The specific marker value used for this class allows for some level of protection against misuse where seek() is called on a mark or release() is called in the wrong order.

public Mark ( ) : int
return	int

Release() public method

public Release ( int marker ) : void
marker	int
return	void

Seek() public method

public Seek ( int index ) : void
index	int
return	void

Sync() protected method

Make sure we have 'need' elements from current position p . Last valid p index is tokens.length-1 . p+need-1 is the tokens index 'need' elements ahead. If we need 1 element, (p+1-1)==p must be less than tokens.length .

protected Sync ( int want ) : void
want	int
return	void

UnbufferedTokenStream() public method

public UnbufferedTokenStream ( ITokenSource tokenSource ) : System
tokenSource	ITokenSource
return	System

UnbufferedTokenStream() public method

public UnbufferedTokenStream ( ITokenSource tokenSource, int bufferSize ) : System
tokenSource	ITokenSource
bufferSize	int
return	System

Property Details

currentTokenIndex protected property

Absolute token index.

Absolute token index. It's the index of the token about to be read via LT(1) . Goes from 0 to the number of tokens in the entire stream, although the stream size is unknown before the end is reached.

This value is used to set the token indexes if the stream provides tokens that implement IWritableToken .

protected int currentTokenIndex
return	int

lastToken protected property

This is the LT(-1) token for the current position.

protected IToken lastToken
return	IToken

lastTokenBufferStart protected property

When numMarkers > 0 , this is the LT(-1) token for the first token in tokens . Otherwise, this is .

protected IToken lastTokenBufferStart
return	IToken

n protected property

The number of tokens currently in tokens .

This is not the buffer capacity, that's tokens.length .

protected int n
return	int

numMarkers protected property

Count up with mark() and down with release() . When we release() the last mark, numMarkers reaches 0 and we reset the buffer. Copy tokens[p]..tokens[n-1] to tokens[0]..tokens[(n-1)-p] .

protected int numMarkers
return	int

p protected property

0..n-1 index into tokens of next token.

The LT(1) token is tokens[p] . If p == n , we are out of buffered tokens.

protected int p
return	int

tokens protected property

A moving window buffer of the data being scanned.

A moving window buffer of the data being scanned. While there's a marker, we keep adding to buffer. Otherwise, consume() resets so we start filling at index 0 again.

protected IToken[] tokens
return	IToken[]