Package weka.core.tokenizers
Class CharacterDelimitedTokenizer
- java.lang.Object
-
- weka.core.tokenizers.Tokenizer
-
- weka.core.tokenizers.CharacterDelimitedTokenizer
-
- All Implemented Interfaces:
java.io.Serializable
,java.util.Enumeration
,OptionHandler
,RevisionHandler
- Direct Known Subclasses:
NGramTokenizer
,WordTokenizer
public abstract class CharacterDelimitedTokenizer extends Tokenizer
Abstract superclass for tokenizers that take characters as delimiters.- Version:
- $Revision: 1.3 $
- Author:
- fracpete (fracpete at waikato dot ac dot nz)
- See Also:
- Serialized Form
-
-
Constructor Summary
Constructors Constructor Description CharacterDelimitedTokenizer()
-
Method Summary
All Methods Instance Methods Concrete Methods Modifier and Type Method Description java.lang.String
delimitersTipText()
Returns the tip text for this propertyjava.lang.String
getDelimiters()
Get the value of delimiters (not backquoted).java.lang.String[]
getOptions()
Gets the current option settings for the OptionHandler.java.util.Enumeration
listOptions()
Returns an enumeration of all the available options..void
setDelimiters(java.lang.String value)
Set the value of delimiters.void
setOptions(java.lang.String[] options)
Sets the OptionHandler's options using the given list.-
Methods inherited from class weka.core.tokenizers.Tokenizer
globalInfo, hasMoreElements, nextElement, runTokenizer, tokenize, tokenize
-
Methods inherited from class java.lang.Object
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
-
Methods inherited from interface weka.core.RevisionHandler
getRevision
-
-
-
-
Method Detail
-
listOptions
public java.util.Enumeration listOptions()
Returns an enumeration of all the available options..- Specified by:
listOptions
in interfaceOptionHandler
- Overrides:
listOptions
in classTokenizer
- Returns:
- an enumeration of all available options.
-
getOptions
public java.lang.String[] getOptions()
Gets the current option settings for the OptionHandler.- Specified by:
getOptions
in interfaceOptionHandler
- Overrides:
getOptions
in classTokenizer
- Returns:
- the list of current option settings as an array of strings
-
setOptions
public void setOptions(java.lang.String[] options) throws java.lang.Exception
Sets the OptionHandler's options using the given list. All options will be set (or reset) during this call (i.e. incremental setting of options is not possible).- Specified by:
setOptions
in interfaceOptionHandler
- Overrides:
setOptions
in classTokenizer
- Parameters:
options
- the list of options as an array of strings- Throws:
java.lang.Exception
- if an option is not supported
-
getDelimiters
public java.lang.String getDelimiters()
Get the value of delimiters (not backquoted).- Returns:
- Value of delimiters.
-
setDelimiters
public void setDelimiters(java.lang.String value)
Set the value of delimiters. For convenienve, the strings "\r", "\n", "\t", "\'", "\\" get automatically translated into their character representations '\r', '\n', '\t', '\'', '\\'. This means, one can either usesetDelimiters("\r\n\t\\");
orsetDelimiters("\\r\\n\\t\\\\");
.- Parameters:
value
- Value to assign to delimiters.- See Also:
Utils.unbackQuoteChars(String)
-
delimitersTipText
public java.lang.String delimitersTipText()
Returns the tip text for this property- Returns:
- tip text for this property suitable for displaying in the explorer/experimenter gui
-
-