getExpectedTokens method

IntervalSet getExpectedTokens(int stateNumber, RuleContext? context)

Computes the set of input symbols which could follow ATN state number stateNumber in the specified full context. This method considers the complete parser context, but does not evaluate semantic predicates (i.e. all predicates encountered during the calculation are assumed true). If a path in the ATN exists from the starting state to the RuleStopState of the outermost context without matching any symbols, {@link Token#EOF} is added to the returned set.

If [context] is null, it is treated as {@link ParserRuleContext#EMPTY}.

Note that this does NOT give you the set of all tokens that could appear at a given token position in the input phrase. In other words, it does not answer:

"Given a specific partial input phrase, return the set of all tokens that can follow the last token in the input phrase."

The big difference is that with just the input, the parser could land right in the middle of a lookahead decision. Getting all possible tokens given a partial input stream is a separate computation. See https://github.com/antlr/antlr4/issues/1428

For this function, we are specifying an ATN state and call stack to compute what token(s) can come next and specifically: outside of a lookahead decision. That is what you want for error reporting and recovery upon parse error.

@param stateNumber the ATN state number @param context the full parse context @return The set of potentially valid input symbols which could follow the specified state in the specified context. @throws IllegalArgumentException if the ATN does not contain a state with number stateNumber

Implementation

IntervalSet getExpectedTokens(int stateNumber, RuleContext? context) {
  if (stateNumber < 0 || stateNumber >= states.length) {
    throw RangeError.index(stateNumber, states, 'stateNumber');
  }

  var ctx = context;
  final s = states[stateNumber]!;
  var following = nextTokens(s);
  if (!following.contains(Token.EPSILON)) {
    return following;
  }

  final expected = IntervalSet();
  expected.addAll(following);
  expected.remove(Token.EPSILON);
  while (ctx != null &&
      ctx.invokingState >= 0 &&
      following.contains(Token.EPSILON)) {
    final invokingState = states[ctx.invokingState]!;

    final rt = invokingState.transition(0) as RuleTransition;
    following = nextTokens(rt.followState);
    expected.addAll(following);
    expected.remove(Token.EPSILON);
    ctx = ctx.parent;
  }

  if (following.contains(Token.EPSILON)) {
    expected.addOne(Token.EOF);
  }

  return expected;
}