C# 클래스 Lucene.Net.Analysis.Wikipedia.WikipediaTokenizer

Extension of StandardTokenizer that is aware of Wikipedia syntax. It is based off of the Wikipedia tutorial available at http://en.wikipedia.org/wiki/Wikipedia:Tutorial, but it may not be complete.

@lucene.experimental

상속: Tokenizer

파일 보기 프로젝트 열기: apache/lucenenet 1 사용 예제들

공개 프로퍼티들

프로퍼티	타입	설명
TOKEN_TYPES	string[]

공개 메소드들

메소드	설명
Dispose ( ) : void
End ( ) : void
IncrementToken ( ) : bool
Reset ( ) : void
WikipediaTokenizer ( AttributeFactory factory, TextReader input, int tokenOutput, IEnumerable untokenizedTypes ) : System.Collections.Generic	Creates a new instance of the org.apache.lucene.analysis.wikipedia.WikipediaTokenizer. Attaches the `input` to a the newly created JFlex scanner. Uses the given org.apache.lucene.util.AttributeSource.AttributeFactory.
WikipediaTokenizer ( TextReader input ) : System.Collections.Generic	Creates a new instance of the WikipediaTokenizer. Attaches the `input` to a newly created JFlex scanner.
WikipediaTokenizer ( TextReader input, int tokenOutput, IEnumerable untokenizedTypes ) : System.Collections.Generic	Creates a new instance of the org.apache.lucene.analysis.wikipedia.WikipediaTokenizer. Attaches the `input` to a the newly created JFlex scanner.

비공개 메소드들

메소드	설명
Init ( int tokenOutput, IEnumerable untokenizedTypes ) : void
collapseAndSaveTokens ( int tokenType, string type ) : void
collapseTokens ( int tokenType ) : void
setupSavedToken ( int positionInc, string type ) : void
setupToken ( ) : void

메소드 상세

Dispose() 공개 메소드

public Dispose ( ) : void
리턴	void

End() 공개 메소드

public End ( ) : void
리턴	void

IncrementToken() 공개 메소드

public IncrementToken ( ) : bool
리턴	bool

Reset() 공개 메소드

public Reset ( ) : void
리턴	void

WikipediaTokenizer() 공개 메소드

Creates a new instance of the org.apache.lucene.analysis.wikipedia.WikipediaTokenizer. Attaches the input to a the newly created JFlex scanner. Uses the given org.apache.lucene.util.AttributeSource.AttributeFactory.

public WikipediaTokenizer ( AttributeFactory factory, TextReader input, int tokenOutput, IEnumerable untokenizedTypes ) : System.Collections.Generic
factory	AttributeFactory
input	TextReader	The input
tokenOutput	int	One of , ,
untokenizedTypes	IEnumerable
리턴	System.Collections.Generic

WikipediaTokenizer() 공개 메소드

Creates a new instance of the WikipediaTokenizer. Attaches the input to a newly created JFlex scanner.

public WikipediaTokenizer ( TextReader input ) : System.Collections.Generic
input	TextReader	The Input Reader
리턴	System.Collections.Generic

WikipediaTokenizer() 공개 메소드

Creates a new instance of the org.apache.lucene.analysis.wikipedia.WikipediaTokenizer. Attaches the input to a the newly created JFlex scanner.

public WikipediaTokenizer ( TextReader input, int tokenOutput, IEnumerable untokenizedTypes ) : System.Collections.Generic
input	TextReader	The input
tokenOutput	int	One of , ,
untokenizedTypes	IEnumerable
리턴	System.Collections.Generic

프로퍼티 상세

TOKEN_TYPES 공개적으로 정적으로 프로퍼티

String token types that correspond to token type int constants

public static string[] TOKEN_TYPES
리턴	string[]