public class JapaneseAnalyzer extends StopwordAnalyzerBase
JapaneseTokenizerAnalyzer.ReuseStrategy, Analyzer.TokenStreamComponentsstopwordsGLOBAL_REUSE_STRATEGY, PER_FIELD_REUSE_STRATEGY| Constructor and Description |
|---|
JapaneseAnalyzer() |
JapaneseAnalyzer(UserDictionary userDict,
JapaneseTokenizer.Mode mode,
CharArraySet stopwords,
Set<String> stoptags) |
| Modifier and Type | Method and Description |
|---|---|
protected Analyzer.TokenStreamComponents |
createComponents(String fieldName)
Creates a new
Analyzer.TokenStreamComponents instance for this analyzer. |
static CharArraySet |
getDefaultStopSet() |
static Set<String> |
getDefaultStopTags() |
protected TokenStream |
normalize(String fieldName,
TokenStream in)
Wrap the given
TokenStream in order to apply normalization filters. |
getStopwordSet, loadStopwordSet, loadStopwordSet, loadStopwordSetattributeFactory, close, getOffsetGap, getPositionIncrementGap, getReuseStrategy, getVersion, initReader, initReaderForNormalization, normalize, setVersion, tokenStream, tokenStreampublic JapaneseAnalyzer()
public JapaneseAnalyzer(UserDictionary userDict, JapaneseTokenizer.Mode mode, CharArraySet stopwords, Set<String> stoptags)
public static CharArraySet getDefaultStopSet()
public static Set<String> getDefaultStopTags()
protected Analyzer.TokenStreamComponents createComponents(String fieldName)
AnalyzerAnalyzer.TokenStreamComponents instance for this analyzer.createComponents in class AnalyzerfieldName - the name of the fields content passed to the
Analyzer.TokenStreamComponents sink as a readerAnalyzer.TokenStreamComponents for this analyzer.protected TokenStream normalize(String fieldName, TokenStream in)
AnalyzerTokenStream in order to apply normalization filters.
The default implementation returns the TokenStream as-is. This is
used by Analyzer.normalize(String, String).