scan method

Token scan()

Consumes and returns the next token.

Implementation

Token scan() {
  if (_streamEndProduced) throw StateError('Out of tokens.');
  if (!_tokenAvailable) _fetchMoreTokens();

  var token = _tokens.removeFirst();
  _tokenAvailable = false;
  _tokensParsed++;
  _streamEndProduced = token.type == TokenType.streamEnd;
  return token;
}