C# Class Lucene.Net.Analysis.Core.WhitespaceTokenizer

A WhitespaceTokenizer is a tokenizer that divides text at whitespace. Adjacent sequences of non-Whitespace characters form tokens.

You must specify the required LuceneVersion compatibility when creating WhitespaceTokenizer:

  • As of 3.1, CharTokenizer uses an int based API to normalize and detect token characters. See CharTokenizer#isTokenChar(int) and CharTokenizer#normalize(int) for details.

Inheritance: Lucene.Net.Analysis.Util.CharTokenizer
Afficher le fichier Open project: apache/lucenenet Class Usage Examples

Méthodes publiques

Méthode Description
WhitespaceTokenizer ( LuceneVersion matchVersion, AttributeFactory factory, TextReader @in ) : System.IO

Construct a new WhitespaceTokenizer using a given org.apache.lucene.util.AttributeSource.AttributeFactory. @param matchVersion Lucene version to match See above
WhitespaceTokenizer ( LuceneVersion matchVersion, TextReader @in ) : System.IO

Méthodes protégées

Méthode Description
IsTokenChar ( int c ) : bool

Collects only characters which do not satisfy Character#isWhitespace(int).

Method Details

IsTokenChar() protected méthode

Collects only characters which do not satisfy Character#isWhitespace(int).
protected IsTokenChar ( int c ) : bool
c int
Résultat bool

WhitespaceTokenizer() public méthode

Construct a new WhitespaceTokenizer using a given org.apache.lucene.util.AttributeSource.AttributeFactory. @param matchVersion Lucene version to match See above
public WhitespaceTokenizer ( LuceneVersion matchVersion, AttributeFactory factory, TextReader @in ) : System.IO
matchVersion LuceneVersion
factory AttributeFactory /// the attribute factory to use for this
@in System.IO.TextReader
Résultat System.IO

WhitespaceTokenizer() public méthode

public WhitespaceTokenizer ( LuceneVersion matchVersion, TextReader @in ) : System.IO
matchVersion LuceneVersion Lucene version /// to match See above"/> ///
@in System.IO.TextReader
Résultat System.IO