C# Class Bamboo.Prevalence.Indexing.FullText.Tokenizers.StringTokenizer

Inheritance: ITokenizer
Datei anzeigen Open project: bamboo/Bamboo.Prevalence Class Usage Examples

Public Methods

Method Description
Clone ( ITokenizer tail ) : ITokenizer

Returns a clone.
NextToken ( ) : Token

See Bamboo.Prevalence.Indexing.FullText.ITokenizer.NextToken for details.
StringTokenizer ( string text ) : System

Creates a new tokenizer for the string in the text argument.

Protected Methods

Method Description
StringTokenizer ( string text, int current ) : System

Creates a new tokenizer for the string in the text argument starting from the position indicated by the current argument.

Private Methods

Method Description
IsSeparator ( string text, int index ) : bool
SkipSeparators ( ) : void

Method Details

Clone() public method

Returns a clone.
public Clone ( ITokenizer tail ) : ITokenizer
tail ITokenizer must always be null
return ITokenizer

NextToken() public method

See Bamboo.Prevalence.Indexing.FullText.ITokenizer.NextToken for details.
public NextToken ( ) : Token
return Bamboo.Prevalence.Indexing.FullText.Token

StringTokenizer() public method

Creates a new tokenizer for the string in the text argument.
public StringTokenizer ( string text ) : System
text string token source
return System

StringTokenizer() protected method

Creates a new tokenizer for the string in the text argument starting from the position indicated by the current argument.
protected StringTokenizer ( string text, int current ) : System
text string token source
current int starting position
return System