C# Класс org.apache.lucene.analysis.compound.hyphenation.HyphenationTree

This tree structure stores the hyphenation patterns in an efficient way for fast lookup. It provides the provides the method to hyphenate a word. This class has been taken from the Apache FOP project (http://xmlgraphics.apache.org/fop/). They have been slightly modified.

Наследование: TernaryTree, PatternConsumer

Показать файл Открыть проект Примеры использования класса

Защищенные свойства (Protected)

Свойство	Тип	Описание
classmap	TernaryTree
stoplist	List
vspace	ByteVector

Открытые методы

Метод	Описание
HyphenationTree ( ) : System
addClass ( string chargroup ) : void	Add a character class to the tree. It is used by PatternParser PatternParser as callback to add character classes. Character classes define the valid word characters for hyphenation. If a word contains a character not defined in any of the classes, it is not hyphenated. It also defines a way to normalize the characters in order to compare them with the stored patterns. Usually pattern files use only lower case characters, in this case a class for letter 'a', for example, should be defined as "aA", the first character being the normalization char.
addException ( string word, List	Add an exception to the tree. It is used by PatternParser PatternParser class as callback to store the hyphenation exceptions.
addPattern ( string pattern, string ivalue ) : void	Add a pattern to the tree. Mainly, to be used by PatternParser PatternParser class as callback to add a pattern to the tree.
findPattern ( string pat ) : string
hyphenate ( char w, int offset, int len, int remainCharCount, int pushCharCount ) : Hyphenation	Hyphenate word and return an array of hyphenation points.
hyphenate ( string word, int remainCharCount, int pushCharCount ) : Hyphenation	Hyphenate word and return a Hyphenation object.
loadPatterns ( File f ) : void	Read hyphenation patterns from an XML file.
loadPatterns ( InputSource source ) : void	Read hyphenation patterns from an XML file.
printStats ( PrintStream @out ) : void

Защищенные методы

Метод	Описание
getValues ( int k ) : sbyte[]
hstrcmp ( char s, int si, char t, int ti ) : int	String compare, returns 0 if equal or t is a substring of s
packValues ( string values ) : int	Packs the values by storing them in 4 bits, two values into a byte Values range is from 0 to 9. We use zero as terminator, so we'll add 1 to the value.
searchPatterns ( char word, int index, sbyte il ) : void	Search for all possible partial matches of word starting at index an update interletter values. In other words, it does something like: `for(i=0; i<patterns.length; i++) { if ( word.substring(index).startsWidth(patterns[i]) ) update_interletter_values(patterns[i]); }` But it is done in an efficient way since the patterns are stored in a ternary tree. In fact, this is the whole purpose of having the tree: doing this search without having to test every single pattern. The number of patterns for languages such as English range from 4000 to 10000. Thus, doing thousands of string comparisons for each word to hyphenate would be really slow without the tree. The tradeoff is memory, but using a ternary tree instead of a trie, almost halves the the memory used by Lout or TeX. It's also faster than using a hash table
unpackValues ( int k ) : string

Описание методов

HyphenationTree() публичный Метод

public HyphenationTree ( ) : System
Результат	System

addClass() публичный Метод

Add a character class to the tree. It is used by PatternParser PatternParser as callback to add character classes. Character classes define the valid word characters for hyphenation. If a word contains a character not defined in any of the classes, it is not hyphenated. It also defines a way to normalize the characters in order to compare them with the stored patterns. Usually pattern files use only lower case characters, in this case a class for letter 'a', for example, should be defined as "aA", the first character being the normalization char.

public addClass ( string chargroup ) : void
chargroup	string
Результат	void

addException() публичный Метод

Add an exception to the tree. It is used by PatternParser PatternParser class as callback to store the hyphenation exceptions.

public addException ( string word, List
word	string	normalized word
hyphenatedword	List	a vector of alternating strings and /// objects.
Результат	void

addPattern() публичный Метод

Add a pattern to the tree. Mainly, to be used by PatternParser PatternParser class as callback to add a pattern to the tree.

public addPattern ( string pattern, string ivalue ) : void
pattern	string	the hyphenation pattern
ivalue	string	interletter weight values indicating the desirability and /// priority of hyphenating at a given point within the pattern. It /// should contain only digit characters. (i.e. '0' to '9').
Результат	void

findPattern() публичный Метод

public findPattern ( string pat ) : string
pat	string
Результат	string

getValues() защищенный Метод

protected getValues ( int k ) : sbyte[]
k	int
Результат	sbyte[]

hstrcmp() защищенный Метод

String compare, returns 0 if equal or t is a substring of s

protected hstrcmp ( char s, int si, char t, int ti ) : int
s	char
si	int
t	char
ti	int
Результат	int

hyphenate() публичный Метод

Hyphenate word and return an array of hyphenation points.

public hyphenate ( char w, int offset, int len, int remainCharCount, int pushCharCount ) : Hyphenation
w	char	char array that contains the word
offset	int	Offset to first character in word
len	int	Length of word
remainCharCount	int	Minimum number of characters allowed before the /// hyphenation point.
pushCharCount	int	Minimum number of characters allowed after the /// hyphenation point.
Результат	Hyphenation

hyphenate() публичный Метод

Hyphenate word and return a Hyphenation object.

public hyphenate ( string word, int remainCharCount, int pushCharCount ) : Hyphenation
word	string	the word to be hyphenated
remainCharCount	int	Minimum number of characters allowed before the /// hyphenation point.
pushCharCount	int	Minimum number of characters allowed after the /// hyphenation point.
Результат	Hyphenation

loadPatterns() публичный Метод

Read hyphenation patterns from an XML file.

In case the parsing fails

public loadPatterns ( File f ) : void
f	File	the filename
Результат	void

loadPatterns() публичный Метод

Read hyphenation patterns from an XML file.

In case the parsing fails

public loadPatterns ( InputSource source ) : void
source	InputSource	the InputSource for the file
Результат	void

packValues() защищенный Метод

Packs the values by storing them in 4 bits, two values into a byte Values range is from 0 to 9. We use zero as terminator, so we'll add 1 to the value.

protected packValues ( string values ) : int
values	string	a string of digits from '0' to '9' representing the /// interletter values.
Результат	int

printStats() публичный Метод

public printStats ( PrintStream @out ) : void
@out	PrintStream
Результат	void

searchPatterns() защищенный Метод

Search for all possible partial matches of word starting at index an update interletter values. In other words, it does something like:

 for(i=0; i<patterns.length; i++) { if ( word.substring(index).startsWidth(patterns[i]) ) update_interletter_values(patterns[i]); }

But it is done in an efficient way since the patterns are stored in a ternary tree. In fact, this is the whole purpose of having the tree: doing this search without having to test every single pattern. The number of patterns for languages such as English range from 4000 to 10000. Thus, doing thousands of string comparisons for each word to hyphenate would be really slow without the tree. The tradeoff is memory, but using a ternary tree instead of a trie, almost halves the the memory used by Lout or TeX. It's also faster than using a hash table

protected searchPatterns ( char word, int index, sbyte il ) : void
word	char	null terminated word to match
index	int	start index from word
il	sbyte	interletter values array to update
Результат	void

unpackValues() защищенный Метод

protected unpackValues ( int k ) : string
k	int
Результат	string

Описание свойств

classmap защищенное свойство

This map stores the character classes

protected TernaryTree classmap
Результат	TernaryTree

stoplist защищенное свойство

This map stores hyphenation exceptions

protected Dictionary> stoplist
Результат	List

vspace защищенное свойство

value space: stores the interletter values

protected ByteVector,org.apache.lucene.analysis.compound.hyphenation vspace
Результат	ByteVector