ITokenizerState class abstract

A tokenizerState returns a token, given a scanner, an initial character read from the scanner, and a tokenizer that is conducting an overall tokenization of the scanner. The tokenizer will typically have a character state table that decides which state to use, depending on an initial character. If a single character is insufficient, a state such as SlashState will read a second character, and may delegate to another state, such as SlashStarState. This prospect of delegation is the reason that the nextToken() method has a tokenizer argument.

Implementers

• ICommentState
• INumberState
• IQuoteState
• ISymbolState
• IWhitespaceState
• IWordState
• MustacheSpecialState

Constructors

ITokenizerState()

Properties

hashCode → int

The hash code for this object.

no setter, inherited

runtimeType → Type

A representation of the runtime type of the object.

no setter, inherited

Methods

nextToken(IScanner scanner, ITokenizer? tokenizer) → Token?

Gets the next token from the stream started from the character linked to this state.

noSuchMethod(Invocation invocation) → dynamic

Invoked when a nonexistent method or property is accessed.

inherited

toString() → String

A string representation of this object.

inherited

Operators

operator ==(Object other) → bool

The equality operator.

inherited