TokenSource class abstract
A source of tokens must provide a sequence of tokens via {@link #nextToken()} and also must reveal it's source of characters; CommonToken's text is computed from a CharStream; it only store indices into the char stream.
Errors from the lexer are never passed to the parser. Either you want to keep going or you do not upon token recognition error. If you do not want to continue lexing then you do not want to continue parsing. Just throw an exception not under [RecognitionException] and Java will naturally toss you all the way out of the recognizers. If you want to continue lexing then you should not throw an exception to the parser--it has already requested a token. Keep lexing until you get a valid one. Just report errors and keep going, looking for a valid token.
- Implementers
Constructors
Properties
- charPositionInLine → int
-
Get the index into the current line for the current position in the input
stream. The first character on a line has position 0.
no setter
- hashCode → int
-
The hash code for this object.
no setterinherited
- inputStream → CharStream?
-
Get the CharStream from which this token source is currently
providing tokens.
no setter
- line → int?
-
Get the line number for the current position in the input stream. The
first line in the input is line 1.
no setter
- runtimeType → Type
-
A representation of the runtime type of the object.
no setterinherited
- sourceName → String
-
Gets the name of the underlying input source. This method returns a
non-null, non-empty string. If such a name is not known, this method
returns {@link IntStream#UNKNOWN_SOURCE_NAME}.
no setter
-
tokenFactory
↔ TokenFactory<
Token> -
Gets the TokenFactory this token source is currently using for
creating
Token
objects from the input.getter/setter pair
Methods
-
nextToken(
) → Token - Return a Token object from your input stream (usually a CharStream). Do not fail/return upon lexing error; keep chewing on the characters until you get a good one; errors are not passed through to the parser.
-
noSuchMethod(
Invocation invocation) → dynamic -
Invoked when a nonexistent method or property is accessed.
inherited
-
toString(
) → String -
A string representation of this object.
inherited
Operators
-
operator ==(
Object other) → bool -
The equality operator.
inherited