C# (CSharp) Lucene.Net.Search Namespace

this query will match documents that contain terms accepted by a given finite-state machine. The automaton can be constructed with the Lucene.Net.Util.Automaton API. Alternatively, it can be created from a regular expression with RegexpQuery or from the standard Lucene wildcard syntax with WildcardQuery.

When the query is executed, it will create an equivalent DFA of the finite-state machine, and will enumerate the term dictionary in an intelligent way to reduce the number of comparisons. For example: the regular expression of [dl]og? will make approximately four comparisons: do, dog, lo, and log.

 Query balancedQuery = new BoostingQuery(positiveQuery, negativeQuery, 0.01f); 

NOTE: this class consumes 4 (or 8 bytes, if scoring is cached) per collected document. If the result set is large this can easily be a very substantial amount of RAM!

NOTE: this class caches at least 128 documents before checking RAM limits.

See the Lucene modules/grouping module for more details including a full code example.

Expert: Collectors are primarily meant to be used to gather raw results from a search, and implement sorting or custom result filtering, collation, etc.

Lucene's core collectors are derived from Collector. Likely your application can use one of these classes, or subclass TopDocsCollector, instead of implementing Collector directly: TopDocsCollector is an abstract base class that assumes you will retrieve the top N docs, according to some criteria, after collection is done. TopScoreDocCollector is a concrete subclass TopDocsCollector and sorts according to score + docID. This is used internally by the IndexSearcher search methods that do not take an explicit Sort. It is likely the most frequently used collector. TopFieldCollector subclasses TopDocsCollector and sorts according to a specified Sort object (sort by field). This is used internally by the IndexSearcher search methods that take an explicit Sort. TimeLimitingCollector, which wraps any other Collector and aborts the search if it's taken too much time. PositiveScoresOnlyCollector wraps any other Collector and prevents collection of hits whose score is <= 0.0

Collector decouples the score from the collected doc: the score computation is skipped entirely if it's not needed. Collectors that do need the score should implement the SetScorer method, to hold onto the passed Scorer instance, and call Scorer.Score() within the collect method to compute the current hit's score. If your collector may request the score for a single hit multiple times, you should use ScoreCachingWrappingScorer.

NOTE: The doc that is passed to the collect method is relative to the current reader. If your collector needs to resolve this to the docID space of the Multi*Reader, you must re-base it by recording the docBase from the most recent setNextReader call. Here's a simple example showing how to collect docIDs into a BitSet:

Searcher searcher = new IndexSearcher(indexReader); final BitSet bits = new BitSet(indexReader.MaxDoc); searcher.search(query, new Collector() { private int docBase; // ignore scorer public void setScorer(Scorer scorer) { } // accept docs out of order (for a BitSet it doesn't matter) public boolean acceptsDocsOutOfOrder() { return true; } public void collect(int doc) { bits.set(doc + docBase); } public void setNextReader(IndexReader reader, int docBase) { this.docBase = docBase; } });

Not all collectors will need to rebase the docID. For example, a collector that simply counts the total number of hits would skip it.

NOTE: Prior to 2.9, Lucene silently filtered out hits with score <= 0. As of 2.9, the core Collectors no longer do that. It's very unusual to have such hits (a negative query boost, or function query returning negative custom scores, could cause it to happen). If you need that behavior, use PositiveScoresOnlyCollector .

NOTE: This API is experimental and might change in incompatible ways in the next release.

This means that IndexReader does not need to be held open. Useful for the majority of casses where we're only interested in the count and not the individual Documents.

Created: May 19, 2004 4:40:36 PM

This API is designed to achieve high performance sorting, by exposing a tight interaction with FieldValueHitQueue as it visits hits. Whenever a hit is competitive, it's enrolled into a virtual slot, which is an int ranging from 0 to numHits-1. The FieldComparator is made aware of segment transitions during searching in case any internal state it's tracking needs to be recomputed during these transitions.

A comparator must define these functions:

Compare Compare a hit at 'slot a' with hit 'slot b'. SetBottom This method is called by FieldValueHitQueue to notify the FieldComparator of the current weakest ("bottom") slot. Note that this slot may not hold the weakest value according to your comparator, in cases where your comparator is not the primary one (ie, is only used to break ties from the comparators before it). CompareBottom Compare a new hit (docID) against the "weakest" (bottom) entry in the queue. Copy Installs a new hit into the priority queue. The FieldValueHitQueue calls this method when a new hit is competitive. SetNextReader Invoked when the search is switching to the next segment. You may need to update internal state of the comparator, for example retrieving new values from the FieldCache. Value Return the sort value stored in the specified slot. This is only called at the end of the search, in order to populate FieldDoc.fields when returning the top results. NOTE: This API is experimental and might change in incompatible ways in the next release.

Created: Feb 11, 2004 1:23:38 PM @since lucene 1.4

Technically, this same functionality could be achieved with ChainedFilter (under queries/), however the benefit of this class is it never materializes the full bitset for the filter. Instead, the #match method is invoked on-demand, per docID visited during searching. If you know few docIDs will be visited, and the logic behind #match is relatively costly, this may be a better way to filter than ChainedFilter.

this query uses MultiTermQuery.TopTermsScoringBooleanQueryRewrite as default. So terms will be collected and scored according to their edit distance. Only the top terms are used for building the BooleanQuery. It is not recommended to change the rewrite mode for fuzzy queries.

At most, this query will match terms up to {@value Lucene.Net.Util.Automaton.LevenshteinAutomata#MAXIMUM_SUPPORTED_DISTANCE} edits. Higher distances (especially with transpositions enabled), are generally not useful and will match a significant amount of the term dictionary. If you really want this, consider using an n-gram indexing technique (such as the SpellChecker in the suggest module) instead.

NOTE: terms of length 1 or 2 will sometimes not match because of how the scaled distance between two terms is computed. For a term to match, the edit distance between the terms must be less than the minimum length term (either the input term, or the candidate term). For example, FuzzyQuery on term "abcd" with maxEdits=2 will not match an indexed term "ab", and FuzzyQuery on term "a" with maxEdits=2 will not match an indexed term "abc".

Term enumerations are always ordered by #getComparator. Each term in the enumeration is greater than all that precede it.

this is the fastest method as opposed to LinearFuzzyTermsEnum: as enumeration is logarithmic to the number of terms (instead of linear) and comparison is linear to length of the term (rather than quadratic)

Applications usually need only call the inherited {@link #Search(Query)} or {@link #Search(Query,Filter)} methods. For performance reasons it is recommended to open only one IndexSearcher and use it for all of your searches.

Note that you can only access Hits from an IndexSearcher as long as it is not yet closed, otherwise an IOException will be thrown.

Applications usually need only call the inherited Searcher.Search(Query, int) or Searcher.Search(Query,Filter, int) methods.

• Comparisons are fast
• The term dictionary is enumerated in an intelligent way, to avoid comparisons. See AutomatonQuery for more details.

The supported syntax is documented in the RegExp class. Note this might be different than other regular expression implementations. For some alternatives with different syntax, look under the sandbox.

Note this query can be slow, as it needs to iterate over many terms. In order to prevent extremely slow RegexpQueries, a Regexp term should not start with the expression .*

Note that you can only access Hits from a Searcher as long as it is not yet closed, otherwise an IOException will be thrown.

This can be used by MultiTermQuerys that need only visit one term, but want to preserve MultiTermQuery semantics such as RewriteMethod.

Created: Feb 11, 2004 1:25:29 PM @since lucene 1.4

Suitable for high cardinality, sparsely populated facets. i.e. There are a large number of facet values and each facet value is hit in a small percentage of documents. Especially if there are also a large number of documents. SimpleFacetedSearch holds a bitmap for each value representing whether that value is a hit is each document (approx 122KB per 1M documents per facet value). So this is an O(N*M) problem. The memory requirement can grow very quickly.

SparseFacetedSearcher records the DocID (Int32) for each value hit (memory cost = values * hits * 4). SimpleFacetedSearch record a bit for evey document per value (memory cost = values * documents / 8). So if the average number of hits for each value is less than 1/32 or 3.125% then Sparse is more memory efficient.

There are also some enumerable methods than mean there is much less pressure on the GC. Plus some bug fixes.

same as TestRankingSimilarity in TestRanking.zip from http://issues.apache.org/jira/browse/LUCENE-323