Implements classification of text by using a ScriptEngine which supports the TokenCategorizer service. Languages should subclass this type and override the Engine property. They should then export the provider using MEF indicating the content type which it is applicable to. Languages can also export this under the the content type "DlrCode" and can then override the ContentType property to indicate a specific content type which they provide tokenization for. This specific content type should be a sub-type of DlrCode. This enables the languages to dynamicly register their own content type but statically export the provider.
