C# Класс Lucene.Net.Analysis.Shingle.ShingleFilter

A ShingleFilter constructs shingles (token n-grams) from a token stream. In other words, it creates combinations of tokens as a single token.

For example, the sentence "please divide this sentence into shingles" might be tokenized into shingles "please divide", "divide this", "this sentence", "sentence into", and "into shingles".

This filter handles position increments > 1 by inserting filler tokens (tokens with termtext "_"). It does not handle a position increment of 0.

Наследование: Lucene.Net.Analysis.TokenFilter
Показать файл Открыть проект Примеры использования класса

Открытые методы

Метод Описание
End ( ) : void
IncrementToken ( ) : bool
Reset ( ) : void
ShingleFilter ( TokenStream input ) : System

Construct a ShingleFilter with default shingle size: 2.
ShingleFilter ( TokenStream input, int maxShingleSize ) : System

Constructs a ShingleFilter with the specified shingle size from the TokenStream input
ShingleFilter ( TokenStream input, int minShingleSize, int maxShingleSize ) : System

Constructs a ShingleFilter with the specified shingle size from the TokenStream input
ShingleFilter ( TokenStream input, string tokenType ) : System

Construct a ShingleFilter with the specified token type for shingle tokens and the default shingle size: 2

Приватные методы

Метод Описание
GetNextToken ( InputWindowToken target ) : InputWindowToken

Get the next token from the input stream.

If the next token has positionIncrement > 1, positionIncrement - 1 #fillerTokens are inserted first.

ShiftInputWindow ( ) : void

Fills #inputWindow with input stream tokens, if available, shifting to the right if the window was previously full.

Resets #gramSize to its minimum value.

Описание методов

End() публичный Метод

public End ( ) : void
Результат void

IncrementToken() публичный Метод

public IncrementToken ( ) : bool
Результат bool

Reset() публичный Метод

public Reset ( ) : void
Результат void

ShingleFilter() публичный Метод

Construct a ShingleFilter with default shingle size: 2.
public ShingleFilter ( TokenStream input ) : System
input Lucene.Net.Analysis.TokenStream input stream
Результат System

ShingleFilter() публичный Метод

Constructs a ShingleFilter with the specified shingle size from the TokenStream input
public ShingleFilter ( TokenStream input, int maxShingleSize ) : System
input Lucene.Net.Analysis.TokenStream input stream
maxShingleSize int maximum shingle size produced by the filter.
Результат System

ShingleFilter() публичный Метод

Constructs a ShingleFilter with the specified shingle size from the TokenStream input
public ShingleFilter ( TokenStream input, int minShingleSize, int maxShingleSize ) : System
input Lucene.Net.Analysis.TokenStream input stream
minShingleSize int minimum shingle size produced by the filter.
maxShingleSize int maximum shingle size produced by the filter.
Результат System

ShingleFilter() публичный Метод

Construct a ShingleFilter with the specified token type for shingle tokens and the default shingle size: 2
public ShingleFilter ( TokenStream input, string tokenType ) : System
input Lucene.Net.Analysis.TokenStream input stream
tokenType string token type for shingle tokens
Результат System