Home » lucene-2.3.2-src » org.apache » lucene » analysis » [javadoc | source]

org.apache.lucene.analysis
public class: WhitespaceTokenizer [javadoc | source]

java.lang.Object
   org.apache.lucene.analysis.TokenStream
      org.apache.lucene.analysis.Tokenizer
         org.apache.lucene.analysis.CharTokenizer
            org.apache.lucene.analysis.WhitespaceTokenizer

A WhitespaceTokenizer is a tokenizer that divides text at whitespace. Adjacent sequences of non-Whitespace characters form tokens.

Fields inherited from org.apache.lucene.analysis.Tokenizer:
input

Constructor:
public WhitespaceTokenizer(Reader in) { super(in); } Construct a new WhitespaceTokenizer.

Method from org.apache.lucene.analysis.WhitespaceTokenizer Summary:
isTokenChar

Methods from org.apache.lucene.analysis.CharTokenizer:
isTokenChar,   next,   normalize,   reset

Methods from org.apache.lucene.analysis.Tokenizer:
close,   reset

Methods from org.apache.lucene.analysis.TokenStream:
close,   next,   next,   reset

Methods from java.lang.Object:
equals,   getClass,   hashCode,   notify,   notifyAll,   toString,   wait,   wait,   wait

Method from org.apache.lucene.analysis.WhitespaceTokenizer Detail:
protected boolean isTokenChar(char c) { return !Character.isWhitespace(c); } Collects only characters which do not satisfy Character#isWhitespace(char) .