GenericSymbolState class

The idea of a symbol is a character that stands on its own, such as an ampersand or a parenthesis.

For example, when tokenizing the expression (isReady)& (isWilling), a typical tokenizer would return 7 tokens, including one for each parenthesis and one for the ampersand. Thus a series of symbols such as )&( becomes three tokens, while a series of letters such as isReady becomes a single word token.

Multi-character symbols are an exception to the rule that a symbol is a standalone character. For example, a tokenizer may want less-than-or-equals to tokenize as a single token. This class provides a method for establishing which multi-character symbols an object of this class should treat as single symbols. This allows, for example, "cat <= dog" to tokenize as three tokens, rather than splitting the less-than and equals symbols into separate tokens.

By default, this state recognizes the following multi-character symbols: !=, :-, <=, >=

Implemented types

• ISymbolState

Implementers

• CsvSymbolState
• ExpressionSymbolState

Constructors

GenericSymbolState()

Properties

hashCode → int

The hash code for this object.

no setter, inherited

runtimeType → Type

A representation of the runtime type of the object.

no setter, inherited

Methods

add(String value, TokenType tokenType) → void

Add a multi-character symbol.

override

nextToken(IScanner scanner, ITokenizer? tokenizer) → Token

Return a symbol token from a scanner.

override

noSuchMethod(Invocation invocation) → dynamic

Invoked when a nonexistent method or property is accessed.

inherited

toString() → String

A string representation of this object.

inherited

Operators

operator ==(Object other) → bool

The equality operator.

inherited