public class Token extends Object
Thread Safety: Objects of this class are NOT thread safe and should not be accessed simultaneously by multiple threads.
CAUTION: Morphemes are implemented as lazy proxies onto a
Dictionary, and care should be taken not to access the same
Dictionary from multiple threads. Once any member of a
Morpheme has been read, its link to the Dictionary is broken
and this restriction is relaxed
| Constructor and Description |
|---|
Token()
Creates a blank Token
|
Token(String surface,
int cost,
int start,
int length,
Morpheme morpheme)
Creates a Token with explicit parameters
|
Token(String surface,
Node node)
Creates a Token from a Node
|
| Modifier and Type | Method and Description |
|---|---|
int |
end()
Gets the end of the character range of this Token within the underlying
sentence
|
boolean |
equals(Object object) |
int |
getCost()
Gets the Viterbi cost of this Token
|
int |
getLength()
Gets the length of the character range of this Token within the
underlying sentence
|
Morpheme |
getMorpheme()
Gets the morpheme data for this Token
|
int |
getStart()
Gets the start of the character range of this Token within the
underlying sentence
|
String |
getSurface()
Gets the character range of this Token within the underlying sentence
|
boolean |
isSentenceStart()
Returns whether or not this Token begins a new sentence.
|
void |
setCost(int cost)
Sets the Viterbi cost of this Token
|
void |
setLength(int length)
Sets the length of the character range of this Token within the
underlying sentence
|
void |
setMorpheme(Morpheme morpheme)
Sets the morpheme data for this Token
|
void |
setSentenceStart(boolean sentenceStart)
Sets whether or not this token begins a new sentence.
|
void |
setStart(int start)
Sets the start of the character range of this Token within the
underlying sentence
|
void |
setSurface(String surface)
Sets the character range of this Token within the underlying sentence
|
String |
toString()
Returns the character range of this Token within the underlying sentence
|
public Token(String surface, Node node)
surface - The underlying sentence stringnode - The Node to create frompublic Token(String surface, int cost, int start, int length, Morpheme morpheme)
surface - The character range within the underlying sentencecost - The Viterbi coststart - The start of the character range within the underlying
sentencelength - The length of the character range within the underlying
sentencemorpheme - The morpheme datapublic Token()
public boolean isSentenceStart()
public void setSentenceStart(boolean sentenceStart)
public int getStart()
public void setStart(int start)
start - The start of the character range of this Token within the
underlying sentencepublic int getLength()
public void setLength(int length)
length - The length of the character range of this Token within the
underlying sentencepublic String getSurface()
public void setSurface(String surface)
surface - The character range of this Token within the underlying
sentencepublic int getCost()
public void setCost(int cost)
cost - The Viterbi cost of this Tokenpublic Morpheme getMorpheme()
public void setMorpheme(Morpheme morpheme)
morpheme - new Morpheme for this Tokenpublic int end()