C# Class Antlr4.Runtime.UnbufferedTokenStream

Inheritance: ITokenStream
Exibir arquivo Open project: antlr/antlr4

Protected Properties

Property Type Description
currentTokenIndex int
lastToken IToken
lastTokenBufferStart IToken
n int
numMarkers int
p int
tokens IToken[]

Public Methods

Method Description
Consume ( ) : void
Get ( int i ) : IToken
LA ( int i ) : int
LT ( int i ) : IToken
Mark ( ) : int

Return a marker that we can release later.

Return a marker that we can release later.

The specific marker value used for this class allows for some level of protection against misuse where seek() is called on a mark or release() is called in the wrong order.

Release ( int marker ) : void
Seek ( int index ) : void
UnbufferedTokenStream ( ITokenSource tokenSource ) : System
UnbufferedTokenStream ( ITokenSource tokenSource, int bufferSize ) : System

Protected Methods

Method Description
Add ( IToken t ) : void
Fill ( int n ) : int

Add n elements to the buffer. Returns the number of tokens actually added to the buffer. If the return value is less than n , then EOF was reached before n tokens could be added.

GetBufferStartIndex ( ) : int
Sync ( int want ) : void

Make sure we have 'need' elements from current position p . Last valid p index is tokens.length-1 . p+need-1 is the tokens index 'need' elements ahead. If we need 1 element, (p+1-1)==p must be less than tokens.length .

Private Methods

Method Description
GetText ( ) : string
GetText ( IToken start, IToken stop ) : string
GetText ( Interval interval ) : string
GetText ( RuleContext ctx ) : string

Method Details

Add() protected method

protected Add ( IToken t ) : void
t IToken
return void

Consume() public method

public Consume ( ) : void
return void

Fill() protected method

Add n elements to the buffer. Returns the number of tokens actually added to the buffer. If the return value is less than n , then EOF was reached before n tokens could be added.
protected Fill ( int n ) : int
n int
return int

Get() public method

public Get ( int i ) : IToken
i int
return IToken

GetBufferStartIndex() protected method

protected GetBufferStartIndex ( ) : int
return int

LA() public method

public LA ( int i ) : int
i int
return int

LT() public method

public LT ( int i ) : IToken
i int
return IToken

Mark() public method

Return a marker that we can release later.
Return a marker that we can release later.

The specific marker value used for this class allows for some level of protection against misuse where seek() is called on a mark or release() is called in the wrong order.

public Mark ( ) : int
return int

Release() public method

public Release ( int marker ) : void
marker int
return void

Seek() public method

public Seek ( int index ) : void
index int
return void

Sync() protected method

Make sure we have 'need' elements from current position p . Last valid p index is tokens.length-1 . p+need-1 is the tokens index 'need' elements ahead. If we need 1 element, (p+1-1)==p must be less than tokens.length .
protected Sync ( int want ) : void
want int
return void

UnbufferedTokenStream() public method

public UnbufferedTokenStream ( ITokenSource tokenSource ) : System
tokenSource ITokenSource
return System

UnbufferedTokenStream() public method

public UnbufferedTokenStream ( ITokenSource tokenSource, int bufferSize ) : System
tokenSource ITokenSource
bufferSize int
return System

Property Details

currentTokenIndex protected_oe property

Absolute token index.
Absolute token index. It's the index of the token about to be read via LT(1) . Goes from 0 to the number of tokens in the entire stream, although the stream size is unknown before the end is reached.

This value is used to set the token indexes if the stream provides tokens that implement IWritableToken .

protected int currentTokenIndex
return int

lastToken protected_oe property

This is the LT(-1) token for the current position.
protected IToken lastToken
return IToken

lastTokenBufferStart protected_oe property

When numMarkers > 0 , this is the LT(-1) token for the first token in tokens . Otherwise, this is .
protected IToken lastTokenBufferStart
return IToken

n protected_oe property

The number of tokens currently in tokens .

This is not the buffer capacity, that's tokens.length .

protected int n
return int

numMarkers protected_oe property

Count up with mark() and down with release() . When we release() the last mark, numMarkers reaches 0 and we reset the buffer. Copy tokens[p]..tokens[n-1] to tokens[0]..tokens[(n-1)-p] .
protected int numMarkers
return int

p protected_oe property

0..n-1 index into tokens of next token.

The LT(1) token is tokens[p] . If p == n , we are out of buffered tokens.

protected int p
return int

tokens protected_oe property

A moving window buffer of the data being scanned.
A moving window buffer of the data being scanned. While there's a marker, we keep adding to buffer. Otherwise, consume() resets so we start filling at index 0 again.
protected IToken[] tokens
return IToken[]