C# Class Yamool.CWSharp.UnigramTokenizer

The standard unigram tokenizer that converting text into a sequence of tokens.
Inheritance: ITokenizer
Mostra file Open project: zhengchun/CWSharp

Public Methods

Method Description
Traverse ( string text ) : IEnumerable
UnigramTokenizer ( ) : System

Method Details

Traverse() public method

public Traverse ( string text ) : IEnumerable
text string
return IEnumerable

UnigramTokenizer() public method

public UnigramTokenizer ( ) : System
return System