Parser class

Parses a stream of Tokens (produced by a Lexer) into an Abstract Syntax Tree (AST) consisting of Stmt and Expr nodes.

This parser uses a recursive descent strategy, where each grammar rule typically corresponds to a method in the class. It handles operator precedence, control flow structures (if, while, for), function definitions, calls, assignments, and various expressions based on the Python subset grammar.

The parser also includes error handling and synchronization capabilities to attempt recovery after a syntax error and report multiple errors in one pass.

Constructors

Parser(List<Token> tokens, [dynamic errorCallback(String)?])

Creates a new Parser instance.

Properties

current ↔ int

The index of the next token to be consumed from the tokens list.

getter/setter pair

errorCallback ↔ dynamic Function(String)?

An optional callback function to report error messages during parsing. Used primarily when synchronize is called.

getter/setter pair

hashCode → int

The hash code for this object.

no setter, inherited

runtimeType → Type

A representation of the runtime type of the object.

no setter, inherited

tokens → List<Token>

The list of tokens provided by the Lexer.

final

Methods

addsub() → Expr

Parses addition and subtraction expressions. ('+', '-') addsub ::= factor ( ("+" | "-") factor )* ;

advance() → Token

Consumes and returns the current token, advancing the parser state. Returns the previous token (the one just consumed).

assignment() → Expr

Parses an assignment expression (including augmented assignment). assignment ::= ( target (augAssignOp | "=") )? logicOr ; target ::= IDENTIFIER | callOrGet "" expression "" ; // Simplified target augAssignOp::= "+=" | "-=" | "*=" | "/=" | "//=" | "%=" | "**=" | "&=" | "|=" | "^=" | "<<=" | ">>=" ;

bitwiseAnd() → Expr

Parses bitwise AND expressions. ('&') bitwiseAnd ::= shift ( "&" shift )* ;

bitwiseOr() → Expr

Parses bitwise OR expressions. ('|') bitwiseOr ::= bitwiseXor ( "|" bitwiseXor )* ;

bitwiseXor() → Expr

Parses bitwise XOR expressions. ('^') bitwiseXor ::= bitwiseAnd ( "^" bitwiseAnd )* ;

block() → List<Stmt>

Parses a block of statements, typically following a colon (:). Expects a TokenType.NEWLINE, followed by an TokenType.INDENT, then a sequence of statements, and finally a TokenType.DEDENT. Handles nested blocks. block ::= NEWLINE INDENT statement* DEDENT ;

breakStatement() → Stmt

Parses a break statement. breakStmt ::= "break" ;

call() → Expr

Parses function calls and indexing/subscription expressions. Handles chaining like f()[0]. call ::= primary ( "(" arguments? ")" | "" expression "" )* ;

check(TokenType type) → bool

Checks if the current token's type matches the given type without consuming it. Returns false if at the end of the token stream.

classDeclaration() → Stmt

Parses a class definition. classDecl ::= "class" IDENTIFIER ( "(" IDENTIFIER ")" )? ":" block ; Note: block for classes currently only contains methods (DEF).

comparison() → Expr

Parses comparison expressions. (==, !=, <, <=, >, >=) comparison ::= bitwiseOr ( compOp bitwiseOr )* ; compOp ::= "==" | "!=" | "<" | "<=" | ">" | ">=" ;

comprehension() → List<ComprehensionClause>

Parses a list/dict/set comprehension (without the brackets) comprehension ::= expression 'for' IDENTIFIER 'in' iterable 'if' condition ( 'if' condition )*

consume(TokenType type, String message) → Token

Checks if the current token is of the expected type. If yes, consumes and returns the token. If no, throws a ParseError with the given message.

continueStatement() → Stmt

Parses a continue statement. continueStmt ::= "continue" ;

declaration() → Stmt?

Parses a declaration, which can be a function definition or any other statement. declaration ::= classDecl | functionDecl | statement ;

dictComprehension() → Expr

Parses a dict comprehension (including the braces)

error(Token token, String message) → ParseError

Reports a parsing error by creating a ParseError associated with the given token. It also calls the optional errorCallback if provided. Returns the created ParseError object, which should typically be thrown by the caller.

exponentiation() → Expr

Parses exponentiation expressions. ('') Right-associative. exponentiation ::= call ( "" unary )? ; // Right operand has higher precedence (unary)

expression() → Expr

Parses an expression (entry point, lowest precedence). expression ::= assignment ;

expressionStatement() → Stmt

Parses an expression statement. exprStmt ::= expression ;

factor() → Expr

Parses multiplication, division, floor division, and modulo expressions. ('', '/', '//', '%') factor ::= unary ( ("" | "/" | "//" | "%") unary )* ;

finishCall(Expr callee) → Expr

Finishes parsing a function call after the opening parenthesis '(' has been matched. Parses the argument list (positional and keyword). arguments ::= argument ( "," argument )* ; argument ::= expression | IDENTIFIER "=" expression ; // Add *expr, **expr later?

finishIndexOrSlice(Expr object) → Expr

Finishes parsing an indexing operation after the opening bracket '[' has been matched. index ::= "" expression "" ;

forStatement() → Stmt

Parses a for loop statement. forStmt ::= "for" IDENTIFIER "in" expression ":" block ;

fromImportStatement() → Stmt

Parses a from <module> import <name> [as alias], ... statement FromImportStmt ::= "from" module "import" name ("as" alias)? (, name ("as" alias)?)*

functionDeclaration(String kind) → Stmt

Parses a function definition. functionDecl ::= "def" IDENTIFIER "(" parameters? ")" ":" block ; parameters ::= parameter ("," parameter)* ; parameter ::= IDENTIFIER ("=" expression)? | "*" IDENTIFIER | "**" IDENTIFIER ;

globalStatement() → Stmt

Parses a global statement. globalStmt ::= "global" IDENTIFIER ("," IDENTIFIER)* ;

ifStatement() → Stmt

Parses an if-elif-else statement. ifStmt ::= "if" expression ":" block ( "elif" expression ":" block )* ( "else" ":" block )? ;

importStatement() → Stmt

Parses an import <module> [as alias] statement importStmt ::= "import" module ("as" alias)?

isAtEnd() → bool

Returns true if all tokens have been consumed (i.e., current token is EOF).

lambda() → Expr

Parses a lambda expression or delegates to the next precedence level. lambdaExpr ::= "lambda" parameters? ":" expression ;

listComprehension() → Expr

Parses a list comprehension (including the brackets)

logicAnd() → Expr

Parses logical AND expressions. ('and') logicAnd ::= negation ( "and" negation )* ;

logicOr() → Expr

Parses logical OR expressions. ('or') logicOr ::= logicAnd ( "or" logicAnd )* ;

match(List<TokenType> types) → bool

Checks if the current token's type is one of the given types. If a match is found, consumes the token (advances) and returns true. Otherwise, returns false without consuming the token.

negation() → Expr

Parses logical NOT expressions. ('not') negation ::= ( "not" negation ) | comparison ;

nonlocalStatement() → Stmt

Parses a nonlocal statement. nonlocalStmt ::= "nonlocal" IDENTIFIER ("," IDENTIFIER)* ;

noSuchMethod(Invocation invocation) → dynamic

Invoked when a nonexistent method or property is accessed.

inherited

parse() → List<Stmt>

Parses the entire list of tokens and returns a list of top-level Stmt nodes, representing the program's structure.

parseSlice(Expr listExpr, Token bracket, Expr? start) → Expr

Finishes parsing of a slice expression (start:end, start:end:step, end], [start or : ) The start expression has been parsed already by the calling function, and the current token should be the colon.

passStatement() → Stmt

Parses a pass statement. passStmt ::= "pass" ;

peek() → Token

peekNextToken() → Token

Returns the token after the current one, without consuming anything. Useful for lookahead.

previous() → Token

Returns the current token without consuming it.

primary() → Expr

Parses the highest-precedence expressions: literals, variables, parenthesized expressions, list literals, and dictionary literals. primary ::= NUMBER | STRING | "True" | "False" | "None" | IDENTIFIER | "(" expression ")" | "" (expression ("," expression)*)? "" | "{" (pair ("," pair)*)? "}" ; pair ::= expression ":" expression ;

raiseStatement() → Stmt

returnStatement() → Stmt

Parses a return statement. returnStmt ::= "return" expression? ;

setComprehension() → Expr

shift() → Expr

Parses bitwise shift expressions. ('<<', '>>') shift ::= addsub ( ("<<" | ">>") addsub )* ;

statement() → Stmt

Parses a statement. statement ::= ifStmt | returnStmt | whileStmt | forStmt | passStmt | breakStmt | continueStmt | exprStmt ;

synchronize() → void

Attempts to recover from a parsing error to allow finding subsequent errors.

ternary() → Expr

Parses a ternary expression- ternary ::= expression "if" condition "else" expression

toString() → String

A string representation of this object.

inherited

tryStatement() → Stmt

unary() → Expr

Parses unary expressions. ('-', '+', '') Note: 'not' handled in negation. unary ::= ("+" | "-" | "") unary | exponentiation ;

whileStatement() → Stmt

Parses a while loop statement. whileStmt ::= "while" expression ":" block ;

Operators

operator ==(Object other) → bool

The equality operator.

inherited