ESC/Java2
© 2003,2004,2005 David Cok and Joseph Kiniry
© 2005 UCD Dublin
© 2003,2004 Radboud University Nijmegen
© 1999,2000 Compaq Computer Corporation
© 1997,1998,1999 Digital Equipment Corporation
All Rights Reserved

javafe.parser
Class Token

java.lang.Object
  extended byjavafe.parser.Token
Direct Known Subclasses:
Lex

public class Token
extends java.lang.Object

The Token class defines a set of fields that describe lexical tokens.

Allocating an individual Token object for every token consumes a lot of memory. Instead, our front end tries to reuse Token instances as much as possible. Thus, for example, Lex is a subclass of Token; Lex returns information about the current token not by allocating a new Token object but rather by filling in the Token fields of itself. To facilitate the reuse of Token instances, Token has a copyInto method.


Field Summary
 java.lang.Object auxVal
          Auxillary information about the token.
static int CLEAR
           
 int endingLoc
          The location of the last character of the token.
 Identifier identifierVal
          Identifier represented by the token.
 int startingLoc
          The location of the first character of the token.
 int ttype
          Integer code giving the kind of token.
 
Constructor Summary
Token()
          * Creation: * *
 
Method Summary
 void clear()
           
 Token copyInto(Token dst)
          Copy all the fields of this into dst.
 java.lang.String ztoString()
          Return a representation of this suitable for debug output.
 void zzz()
          Check the invariants of this.
 
Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
 

Field Detail

ttype

public int ttype
Integer code giving the kind of token.

To clear a token, set this field to Token.CLEAR and set identifierVal and auxVal to null, as is done in the clear() method.

See Also:
TagConstants

CLEAR

public static final int CLEAR
See Also:
Constant Field Values

startingLoc

public int startingLoc
The location of the first character of the token.


endingLoc

public int endingLoc
The location of the last character of the token. (This value isn't "off-by-one" right now.)


identifierVal

public Identifier identifierVal
Identifier represented by the token. Must be non-null if ttype is TagConstants.IDENT.


auxVal

public java.lang.Object auxVal
Auxillary information about the token. In the case of literal tokens, this field holds the value of token. In particular, if ttype is one of the codes on the left of the following table, then auxVal must be an instance of the class on the right:
TagConstants.INTLIT Integer
TagConstants.LONGLIT Long
TagConstants.FLOATLIT Float
TagConstants.DOUBLELIT Double
TagConstants.STRINGLIT String
TagConstants.CHARLIT Integer
TagConstants.LEXICALPRAGMA LexicalPragma
TagConstants.MODIFIERPRAGMA ModifierPragma
TagConstants.STMTPRAGMA StmtPragma
TagConstants.TYPEDECLELEMPRAGMA TypeDeclElemPragma

For the various pragmas, auxVal may be null, but for the literals it may not be.

Constructor Detail

Token

public Token()
* Creation: * *

Method Detail

clear

public void clear()

copyInto

public final Token copyInto(Token dst)
Copy all the fields of this into dst. For convenience, returns dst.


ztoString

public java.lang.String ztoString()
Return a representation of this suitable for debug output.


zzz

public void zzz()
Check the invariants of this.


ESC/Java2
© 2003,2004,2005 David Cok and Joseph Kiniry
© 2005 UCD Dublin
© 2003,2004 Radboud University Nijmegen
© 1999,2000 Compaq Computer Corporation
© 1997,1998,1999 Digital Equipment Corporation
All Rights Reserved

The ESC/Java2 Project Homepage