org.apache.lucene.analysis.wikipedia

Class WikipediaTokenizer

  • All Implemented Interfaces:
    Closeable, AutoCloseable


    public final class WikipediaTokenizer
    extends Tokenizer
    Extension of StandardTokenizer that is aware of Wikipedia syntax. It is based off of the Wikipedia tutorial available at http://en.wikipedia.org/wiki/Wikipedia:Tutorial, but it may not be complete.