C# Class Antlr.Runtime.LegacyCommonTokenStream

Inheritance: ITokenStream
Show file Open project: antlr/antlrcs

Protected Properties

Property Type Description
channel int
channelOverrideMap int>.IDictionary
discardOffChannelTokens bool
discardSet List
lastMarker int
p int
tokens List

Public Methods

Method Description
Consume ( ) : void

* Move the input pointer to the next incoming token. The stream * must become active with LT(1) available. consume() simply * moves the input pointer so that LT(1) points at the next * input symbol. Consume at least one token. *

* Walk past any token not on the channel the parser is listening to. *

DiscardTokenType ( int ttype ) : void
FillBuffer ( ) : void

* Load all tokens from the token source and put in tokens. * This is done upon first LT request because you might want to * set some token type / channel overrides before filling buffer. *
Get ( int i ) : IToken

* Return absolute token i; ignore which channel the tokens are on; * that is, count all tokens not just on-channel tokens. *
Get ( int start, int count ) : List
GetTokens ( ) : IList
GetTokens ( int start, int stop ) : IList
GetTokens ( int start, int stop, BitSet types ) : IList

* Given a start and stop index, return a List of all tokens in * the token type BitSet. Return null if no tokens were found. This * method looks at both on and off channel tokens. *
GetTokens ( int start, int stop, IList types ) : IList
GetTokens ( int start, int stop, int ttype ) : IList
LA ( int i ) : int
LT ( int k ) : IToken

* Get the ith token from the current position 1..n where k=1 is the * first symbol of lookahead. *
LegacyCommonTokenStream ( ) : System.Collections.Generic
LegacyCommonTokenStream ( ITokenSource tokenSource ) : System.Collections.Generic
LegacyCommonTokenStream ( ITokenSource tokenSource, int channel ) : System.Collections.Generic
Mark ( ) : int
Release ( int marker ) : void
Reset ( ) : void
Rewind ( ) : void
Rewind ( int marker ) : void
Seek ( int index ) : void
SetDiscardOffChannelTokens ( bool discardOffChannelTokens ) : void
SetTokenSource ( ITokenSource tokenSource ) : void

Reset this token stream by setting its token source.
SetTokenTypeChannel ( int ttype, int channel ) : void

* A simple filter mechanism whereby you can tell this token stream * to force all tokens of type ttype to be on channel. For example, * when interpreting, we cannot exec actions so we need to tell * the stream to force all WS and NEWLINE to be a different, ignored * channel. *
ToString ( ) : string
ToString ( IToken start, IToken stop ) : string
ToString ( int start, int stop ) : string

Protected Methods

Method Description
LB ( int k ) : IToken

Look backwards k tokens on-channel tokens
SkipOffTokenChannels ( int i ) : int

Given a starting index, return the index of the first on-channel token.
SkipOffTokenChannelsReverse ( int i ) : int

Method Details

Consume() public method

* Move the input pointer to the next incoming token. The stream * must become active with LT(1) available. consume() simply * moves the input pointer so that LT(1) points at the next * input symbol. Consume at least one token. *
* Walk past any token not on the channel the parser is listening to. *
public Consume ( ) : void
return void

DiscardTokenType() public method

public DiscardTokenType ( int ttype ) : void
ttype int
return void

FillBuffer() public method

* Load all tokens from the token source and put in tokens. * This is done upon first LT request because you might want to * set some token type / channel overrides before filling buffer. *
public FillBuffer ( ) : void
return void

Get() public method

* Return absolute token i; ignore which channel the tokens are on; * that is, count all tokens not just on-channel tokens. *
public Get ( int i ) : IToken
i int
return IToken

Get() public method

public Get ( int start, int count ) : List
start int
count int
return List

GetTokens() public method

public GetTokens ( ) : IList
return IList

GetTokens() public method

public GetTokens ( int start, int stop ) : IList
start int
stop int
return IList

GetTokens() public method

* Given a start and stop index, return a List of all tokens in * the token type BitSet. Return null if no tokens were found. This * method looks at both on and off channel tokens. *
public GetTokens ( int start, int stop, BitSet types ) : IList
start int
stop int
types BitSet
return IList

GetTokens() public method

public GetTokens ( int start, int stop, IList types ) : IList
start int
stop int
types IList
return IList

GetTokens() public method

public GetTokens ( int start, int stop, int ttype ) : IList
start int
stop int
ttype int
return IList

LA() public method

public LA ( int i ) : int
i int
return int

LB() protected method

Look backwards k tokens on-channel tokens
protected LB ( int k ) : IToken
k int
return IToken

LT() public method

* Get the ith token from the current position 1..n where k=1 is the * first symbol of lookahead. *
public LT ( int k ) : IToken
k int
return IToken

LegacyCommonTokenStream() public method

public LegacyCommonTokenStream ( ) : System.Collections.Generic
return System.Collections.Generic

LegacyCommonTokenStream() public method

public LegacyCommonTokenStream ( ITokenSource tokenSource ) : System.Collections.Generic
tokenSource ITokenSource
return System.Collections.Generic

LegacyCommonTokenStream() public method

public LegacyCommonTokenStream ( ITokenSource tokenSource, int channel ) : System.Collections.Generic
tokenSource ITokenSource
channel int
return System.Collections.Generic

Mark() public method

public Mark ( ) : int
return int

Release() public method

public Release ( int marker ) : void
marker int
return void

Reset() public method

public Reset ( ) : void
return void

Rewind() public method

public Rewind ( ) : void
return void

Rewind() public method

public Rewind ( int marker ) : void
marker int
return void

Seek() public method

public Seek ( int index ) : void
index int
return void

SetDiscardOffChannelTokens() public method

public SetDiscardOffChannelTokens ( bool discardOffChannelTokens ) : void
discardOffChannelTokens bool
return void

SetTokenSource() public method

Reset this token stream by setting its token source.
public SetTokenSource ( ITokenSource tokenSource ) : void
tokenSource ITokenSource
return void

SetTokenTypeChannel() public method

* A simple filter mechanism whereby you can tell this token stream * to force all tokens of type ttype to be on channel. For example, * when interpreting, we cannot exec actions so we need to tell * the stream to force all WS and NEWLINE to be a different, ignored * channel. *
public SetTokenTypeChannel ( int ttype, int channel ) : void
ttype int
channel int
return void

SkipOffTokenChannels() protected method

Given a starting index, return the index of the first on-channel token.
protected SkipOffTokenChannels ( int i ) : int
i int
return int

SkipOffTokenChannelsReverse() protected method

protected SkipOffTokenChannelsReverse ( int i ) : int
i int
return int

ToString() public method

public ToString ( ) : string
return string

ToString() public method

public ToString ( IToken start, IToken stop ) : string
start IToken
stop IToken
return string

ToString() public method

public ToString ( int start, int stop ) : string
start int
stop int
return string

Property Details

channel protected property

Skip tokens on any channel but this one; this is how we skip whitespace...
protected int channel
return int

channelOverrideMap protected property

Map from token type to channel to override some Tokens' channel numbers
protected IDictionary channelOverrideMap
return int>.IDictionary

discardOffChannelTokens protected property

By default, track all incoming tokens
protected bool discardOffChannelTokens
return bool

discardSet protected property

Set of token types; discard any tokens with this type
protected List discardSet
return List

lastMarker protected property

Track the last mark() call result value for use in rewind().
protected int lastMarker
return int

p protected property

* The index into the tokens list of the current token (next token * to consume). p==-1 indicates that the tokens list is empty *
protected int p
return int

tokens protected property

* Record every single token pulled from the source so we can reproduce * chunks of it later. *
protected List tokens
return List