Token class abstract

A token that was scanned from the input. Each token knows which tokens precede and follow it, acting as a link in a doubly linked list of tokens.

Clients may not extend, implement or mix-in this class.

Implemented types

Constructors

Token(TokenType type, int offset, [CommentToken? preceedingComment])
Initialize a newly created token to have the given type and offset.
factory
Token.eof(int offset, [CommentToken? precedingComments])
Initialize a newly created end-of-file token to have the given offset.
factory

Properties

beforeSynthetic ↔ Token?
The token before this synthetic token, or null if this is not a synthetic ), ], }, or > token.
getter/setter pair
charCount → int
The number of characters parsed by this token.
no setter
charEnd → int
The character offset of the end of this token within the source text.
no setter
charOffset → int
The character offset of the start of this token within the source text.
no setter
end → int
Return the offset from the beginning of the file to the character after the last character of the syntactic entity.
no setteroverride
endGroup → Token?
The token that corresponds to this token, or null if this token is not the first of a pair of matching tokens (such as parentheses).
no setter
hashCode → int
The hash code for this object.
no setterinherited
isEof → bool
Return true if this token represents an end of file.
no setter
isIdentifier → bool
True if this token is an identifier. Some keywords allowed as identifiers, see implementation in KeywordToken.
no setter
isKeyword → bool
True if this token is a keyword. Some keywords allowed as identifiers, see implementation in KeywordToken.
no setter
isKeywordOrIdentifier → bool
True if this token is a keyword or an identifier.
no setter
isModifier → bool
Return true if this token is a modifier such as abstract or const.
no setter
isOperator → bool
Return true if this token represents an operator.
no setter
isSynthetic → bool
Return true if this token is a synthetic token. A synthetic token is a token that was introduced by the parser in order to recover from an error in the code.
no setter
isTopLevelKeyword → bool
Return true if this token is a keyword starting a top level declaration such as class, enum, import, etc.
no setter
isUserDefinableOperator → bool
Return true if this token represents an operator that can be defined by users.
no setter
keyword → Keyword?
Return the keyword, if a keyword token, or null otherwise.
no setter
kind → int
The kind enum of this token as determined by its type.
no setter
length → int
Return the number of characters in the syntactic entity's source range.
no setteroverride
lexeme → String
Return the lexeme that represents this token.
no setter
next ↔ Token?
Return the next token in the token stream.
getter/setter pair
offset ↔ int
Return the offset from the beginning of the file to the first character in the syntactic entity.
getter/setter pairoverride-getter
precedingComments → CommentToken?
Return the first comment in the list of comments that precede this token, or null if there are no comments preceding this token. Additional comments can be reached by following the token stream using next until null is returned.
no setter
previous ↔ Token?
Return the previous token in the token stream.
getter/setter pair
runtimeType → Type
A representation of the runtime type of the object.
no setterinherited
stringValue → String?
For symbol and keyword tokens, returns the string value represented by this token. For StringTokens this method returns null.
no setter
type → TokenType
Return the type of the token.
no setter

Methods

copy() → Token
Return a newly created token that is a copy of this tokens including any preceedingComment tokens, but that is not a part of any token stream.
copyComments(CommentToken? token) → CommentToken?
Copy a linked list of comment tokens identical to the given comment tokens.
matchesAny(List<TokenType> types) → bool
Return true if this token has any one of the given types.
noSuchMethod(Invocation invocation) → dynamic
Invoked when a nonexistent method or property is accessed.
inherited
setNext(Token token) → Token
Set the next token in the token stream to the given token. This has the side-effect of setting this token to be the previous token for the given token. Return the token that was passed in.
setNextWithoutSettingPrevious(Token? token) → Token?
Set the next token in the token stream to the given token without changing which token is the previous token for the given token. Return the token that was passed in.
toString() → String
Returns a textual representation of this token to be used for debugging purposes. The resulting string might contain information about the structure of the token, for example 'StringToken(foo)' for the identifier token 'foo'.
override
value() → Object
Return the value of this token. For keyword tokens, this is the keyword associated with the token, for other tokens it is the lexeme associated with the token.

Operators

operator ==(Object other) → bool
The equality operator.
inherited

Static Methods

lexicallyFirst([Token? t1, Token? t2, Token? t3, Token? t4, Token? t5]) → Token?
Compare the given tokens to find the token that appears first in the source being parsed. That is, return the left-most of all of the tokens. Return the token with the smallest offset, or null if all of the tokens are null.