C# Class Lucene.Net.Analysis.Miscellaneous.LimitTokenCountFilter

This TokenFilter limits the number of tokens while indexing. It is a replacement for the maximum field length setting inside org.apache.lucene.index.IndexWriter.

By default, this filter ignores any tokens in the wrapped {@code TokenStream} once the limit has been reached, which can result in {@code reset()} being called prior to {@code incrementToken()} returning {@code false}. For most {@code TokenStream} implementations this should be acceptable, and faster then consuming the full stream. If you are wrapping a {@code TokenStream} which requires that the full stream of tokens be exhausted in order to function properly, use the #LimitTokenCountFilter(TokenStream,int,boolean) consumeAllTokens option.

Inheritance: TokenFilter
Show file Open project: apache/lucenenet Class Usage Examples

Public Methods

Method Description
IncrementToken ( ) : bool
LimitTokenCountFilter ( TokenStream @in, int maxTokenCount )

Build a filter that only accepts tokens up to a maximum number. This filter will not consume any tokens beyond the maxTokenCount limit
LimitTokenCountFilter ( TokenStream @in, int maxTokenCount, bool consumeAllTokens )

Build an filter that limits the maximum number of tokens per field.
Reset ( ) : void

Method Details

IncrementToken() public method

public IncrementToken ( ) : bool
return bool

LimitTokenCountFilter() public method

Build a filter that only accepts tokens up to a maximum number. This filter will not consume any tokens beyond the maxTokenCount limit
public LimitTokenCountFilter ( TokenStream @in, int maxTokenCount )
@in TokenStream
maxTokenCount int

LimitTokenCountFilter() public method

Build an filter that limits the maximum number of tokens per field.
public LimitTokenCountFilter ( TokenStream @in, int maxTokenCount, bool consumeAllTokens )
@in TokenStream
maxTokenCount int max number of tokens to produce
consumeAllTokens bool whether all tokens from the input must be consumed even if maxTokenCount is reached.

Reset() public method

public Reset ( ) : void
return void