C# Class Yamool.CWSharp.WhiteSpaceTokenBreaker

The tokenizer that used a whitespace to split text into tokens.
Inheritance: ITokenBreaker
Mostra file Open project: zhengchun/CWSharp

Public Methods

Method Description
Next ( ) : Token
WhiteSpaceTokenBreaker ( RewindStringReader stringReader ) : System

Private Methods

Method Description
WhiteSpaceTokenBreaker ( ) : System

Method Details

Next() public method

public Next ( ) : Token
return Token

WhiteSpaceTokenBreaker() public method

public WhiteSpaceTokenBreaker ( RewindStringReader stringReader ) : System
stringReader RewindStringReader
return System