| Property | Type | Description | |
|---|---|---|---|
| HANGUL_TYPE | int | ||
| HIRAGANA_TYPE | int | ||
| IDEOGRAPHIC_TYPE | int | ||
| KATAKANA_TYPE | int | ||
| NUMERIC_TYPE | int | ||
| SOUTH_EAST_ASIAN_TYPE | int | ||
| WORD_TYPE | int | ||
| YYEOF | int |
| Method | Description | |
|---|---|---|
| GetNextToken ( ) : int |
Resumes scanning until the next regular expression is matched, the end of input is encountered or an I/O-Error occurs.
|
|
| GetText ( ICharTermAttribute t ) : void |
Fills CharTermAttribute with the current token text.
|
|
| StandardTokenizerImpl ( TextReader @in ) : Lucene.Net.Analysis.Tokenattributes |
Creates a new scanner
|
|
| YyBegin ( int newState ) : void |
Enters a new lexical state
|
|
| YyCharAt ( int pos ) : char |
Returns the character at position pos from the matched text. It is equivalent to YyText().charAt(pos), but faster
|
|
| YyClose ( ) : void |
Closes the input stream.
|
|
| YyPushBack ( int number ) : void |
Pushes the specified amount of characters back into the input stream. They will be read again by then next call of the scanning method
|
|
| YyReset ( TextReader reader ) : void |
Resets the scanner to read from a new input stream. Does not close the old reader. All internal variables are reset, the old input stream cannot be reused (internal buffer is discarded and lost). Lexical state is set to ZZ_INITIAL. Internal scan buffer is resized down to its initial length, if it has grown.
|
| Method | Description | |
|---|---|---|
| ZzRefill ( ) : bool |
Refills the input buffer.
|
|
| ZzScanError ( int errorCode ) : void |
Reports an error that occured while scanning. In a wellformed scanner (no or only correct usage of YyPushBack(int) and a match-all fallback rule) this method will only be called with things that "Can't Possibly Happen". If this method is called, something is seriously wrong (e.g. a JFlex bug producing a faulty scanner etc.). Usual syntax/scanner level error handling should be done in error fallback rules.
|
|
| ZzUnpackAction ( string packed, int offset, int result ) : int | ||
| ZzUnpackAction ( ) : int[] | ||
| ZzUnpackAttribute ( string packed, int offset, int result ) : int | ||
| ZzUnpackAttribute ( ) : int[] | ||
| ZzUnpackCMap ( string packed ) : char[] |
Unpacks the compressed character translation table.
|
|
| ZzUnpackRowMap ( string packed, int offset, int result ) : int | ||
| ZzUnpackRowMap ( ) : int[] | ||
| ZzUnpackTrans ( string packed, int offset, int result ) : int | ||
| ZzUnpackTrans ( ) : int[] |
| public GetText ( ICharTermAttribute t ) : void | ||
| t | ICharTermAttribute | |
| return | void | |
| public StandardTokenizerImpl ( TextReader @in ) : Lucene.Net.Analysis.Tokenattributes | ||
| @in | TextReader | |
| return | Lucene.Net.Analysis.Tokenattributes | |
| public YyBegin ( int newState ) : void | ||
| newState | int | the new lexical state |
| return | void | |
| public YyCharAt ( int pos ) : char | ||
| pos | int | the position of the character to fetch. /// A value from 0 to YyLength()-1. /// |
| return | char | |
| public YyPushBack ( int number ) : void | ||
| number | int | the number of characters to be read again. /// This number must not be greater than YyLength()! |
| return | void | |
| public YyReset ( TextReader reader ) : void | ||
| reader | TextReader | the new input stream |
| return | void | |