C# Class NClassifier.DefaultTokenizer

Inheritance: ITokenizer

Mostra file Open project: colin-dumitru/Proiect-AI-2012---GUI

Public Properties

Property	Type	Description
BREAK_ON_WHITESPACE	int
BREAK_ON_WORD_BREAKS	int

Public Methods

Method	Description
DefaultTokenizer ( ) : System
DefaultTokenizer ( int tokenizerConfig ) : System
DefaultTokenizer ( string regularExpression ) : System
Tokenize ( string input ) : string[]

Method Details

DefaultTokenizer() public method

public DefaultTokenizer ( ) : System
return	System

DefaultTokenizer() public method

public DefaultTokenizer ( int tokenizerConfig ) : System
tokenizerConfig	int
return	System

DefaultTokenizer() public method

public DefaultTokenizer ( string regularExpression ) : System
regularExpression	string
return	System

Tokenize() public method

public Tokenize ( string input ) : string[]
input	string
return	string[]

Property Details

BREAK_ON_WHITESPACE public_oe static_oe property

Use a "\s" (whitespace) regex to split the string pass in to classify

public static int BREAK_ON_WHITESPACE
return	int

BREAK_ON_WORD_BREAKS public_oe static_oe property

Use a "\W" (non-word character) regex to split the string passed in to classify

public static int BREAK_ON_WORD_BREAKS
return	int