C# Class org.apache.lucene.analysis.miscellaneous.LimitTokenCountFilter

This TokenFilter limits the number of tokens while indexing. It is a replacement for the maximum field length setting inside org.apache.lucene.index.IndexWriter.

By default, this filter ignores any tokens in the wrapped {@code TokenStream} once the limit has been reached, which can result in {@code reset()} being called prior to {@code incrementToken()} returning {@code false}. For most {@code TokenStream} implementations this should be acceptable, and faster then consuming the full stream. If you are wrapping a {@code TokenStream} which requires that the full stream of tokens be exhausted in order to function properly, use the #LimitTokenCountFilter(TokenStream,int,boolean) consumeAllTokens option.

Inheritance: TokenFilter
Show file Open project: paulirwin/lucene.net Class Usage Examples

Public Methods

Method Description
LimitTokenCountFilter ( TokenStream @in, int maxTokenCount )

Build a filter that only accepts tokens up to a maximum number. This filter will not consume any tokens beyond the maxTokenCount limit
LimitTokenCountFilter ( TokenStream @in, int maxTokenCount, bool consumeAllTokens )

Build an filter that limits the maximum number of tokens per field.
incrementToken ( ) : bool
reset ( ) : void

Method Details

LimitTokenCountFilter() public method

Build a filter that only accepts tokens up to a maximum number. This filter will not consume any tokens beyond the maxTokenCount limit
public LimitTokenCountFilter ( TokenStream @in, int maxTokenCount )
@in TokenStream
maxTokenCount int

LimitTokenCountFilter() public method

Build an filter that limits the maximum number of tokens per field.
public LimitTokenCountFilter ( TokenStream @in, int maxTokenCount, bool consumeAllTokens )
@in TokenStream
maxTokenCount int max number of tokens to produce
consumeAllTokens bool whether all tokens from the input must be consumed even if maxTokenCount is reached.

incrementToken() public method

public incrementToken ( ) : bool
return bool

reset() public method

public reset ( ) : void
return void