| Name |
Description |
| BigramTokenizer |
The standard bigram tokenizer that converting text into a sequence of tokens. |
| BigramTokenizer.BigramTokenBreaker |
|
| CharExtensions |
|
| Chunk |
|
| Dawg |
|
| DawgBuilder |
|
| DawgDecoder |
|
| DawgEncoder |
|
| DawgNode |
|
| FnvHash |
|
| MaximumMatchTokenBreaker |
|
| MaximumMatchTokenBreaker.LawlFilter |
The largest average word length filter. |
| MaximumMatchTokenBreaker.LsdmfocwFilter |
|
| MaximumMatchTokenBreaker.SvwlFilter |
The Smallest variance of word lengths filter. |
| RewindStringReader |
|
| StopwordTokenizer |
The tokenizer that stop a specified word. |
| Token |
|
| UnigramTokenizer |
The standard unigram tokenizer that converting text into a sequence of tokens. |
| UnigramTokenizer.UnigramTokenBreaker |
|
| VersionConflictException |
|
| WhiteSpaceTokenBreaker |
The tokenizer that used a whitespace to split text into tokens. |
| WordDict |
|
| WordPoint |
|