org.apache.lucene.analysis.nl: public final class: DutchStemFilter

org.apache.lucene.analysis.nl
public final class: DutchStemFilter [javadoc | source]

java.lang.Object
   org.apache.lucene.analysis.TokenStream
      org.apache.lucene.analysis.TokenFilter
         org.apache.lucene.analysis.nl.DutchStemFilter

A filter that stems Dutch words. It supports a table of words that should not be stemmed at all. The stemmer used can be changed at runtime after the filter object is created (as long as it is a DutchStemmer).

author: Edwin - de Jonge

Fields inherited from org.apache.lucene.analysis.TokenFilter:
input

Constructor:

Constructor:
public DutchStemFilter(TokenStream _in) { super(_in); stemmer = new DutchStemmer(); }
public DutchStemFilter(TokenStream _in, Set exclusiontable) { this(_in); exclusions = exclusiontable; } Builds a DutchStemFilter that uses an exclusiontable.
public DutchStemFilter(TokenStream _in, Set exclusiontable, Map stemdictionary) { this(_in, exclusiontable); stemmer.setStemDictionary(stemdictionary); } Parameters: `stemdictionary` - Dictionary of word stem pairs, that overrule the algorithm

 public DutchStemFilter(TokenStream _in) {
     
    super(_in);
    stemmer = new DutchStemmer();
}

 public DutchStemFilter(TokenStream _in,
    Set exclusiontable) {
    this(_in);
    exclusions = exclusiontable;
}

Builds a DutchStemFilter that uses an exclusiontable.

 public DutchStemFilter(TokenStream _in,
    Set exclusiontable,
    Map stemdictionary) {
    this(_in, exclusiontable);
    stemmer.setStemDictionary(stemdictionary);
}

Parameters:

stemdictionary - Dictionary of word stem pairs, that overrule the algorithm

Method from org.apache.lucene.analysis.nl.DutchStemFilter Summary:
next, setExclusionTable, setStemDictionary, setStemmer

Methods from org.apache.lucene.analysis.TokenFilter:
close

Methods from org.apache.lucene.analysis.TokenStream:
close, next, next, reset

Methods from java.lang.Object:
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method from org.apache.lucene.analysis.nl.DutchStemFilter Detail:
public Token next() throws IOException { if ((token = input.next()) == null) { return null; } // Check the exclusiontable else if (exclusions != null && exclusions.contains(token.termText())) { return token; } else { String s = stemmer.stem(token.termText()); // If not stemmed, dont waste the time creating a new token if (!s.equals(token.termText())) { return new Token(s, token.startOffset(), token.endOffset(), token.type()); } return token; } }
public void setExclusionTable(HashSet exclusiontable) { exclusions = exclusiontable; } Set an alternative exclusion list for this filter.
public void setStemDictionary(HashMap dict) { if (stemmer != null) stemmer.setStemDictionary(dict); } Set dictionary for stemming, this dictionary overrules the algorithm, so you can correct for a particular unwanted word-stem pair.
public void setStemmer(DutchStemmer stemmer) { if (stemmer != null) { this.stemmer = stemmer; } } Set a alternative/custom DutchStemmer for this filter.

Method from org.apache.lucene.analysis.nl.DutchStemFilter Detail:

 public Token next() throws IOException {
    if ((token = input.next()) == null) {
      return null;
    }
    // Check the exclusiontable
    else if (exclusions != null && exclusions.contains(token.termText())) {
      return token;
    } else {
      String s = stemmer.stem(token.termText());
      // If not stemmed, dont waste the time creating a new token
      if (!s.equals(token.termText())) {
        return new Token(s, token.startOffset(),
            token.endOffset(), token.type());
      }
      return token;
    }
}

 public  void setExclusionTable(HashSet exclusiontable) {
    exclusions = exclusiontable;
}

Set an alternative exclusion list for this filter.

 public  void setStemDictionary(HashMap dict) {
    if (stemmer != null)
      stemmer.setStemDictionary(dict);
}

Set dictionary for stemming, this dictionary overrules the algorithm, so you can correct for a particular unwanted word-stem pair.

 public  void setStemmer(DutchStemmer stemmer) {
    if (stemmer != null) {
      this.stemmer = stemmer;
    }
}

Set a alternative/custom DutchStemmer for this filter.