nextToken method

@override Token? nextToken(IScanner scanner, ITokenizer? tokenizer)

Either delegate to a comment-handling state, or return a token with just a slash in it.

• scanner A textual string to be tokenized.
• tokenizer A tokenizer class that controls the process. Returns The next token from the top of the stream.

Implementation

@override
Token? nextToken(IScanner scanner, ITokenizer? tokenizer) {
  var firstSymbol = scanner.read();
  var line = scanner.line();
  var column = scanner.column();

  if (firstSymbol != SLASH) {
    scanner.unread();
    throw Exception('Incorrect usage of CppCommentState.');
  }

  var secondSymbol = scanner.read();
  if (secondSymbol == STAR) {
    return Token(
        TokenType.Comment, '/*' + getMultiLineComment(scanner), line, column);
  } else if (secondSymbol == SLASH) {
    return Token(TokenType.Comment, '//' + getSingleLineComment(scanner),
        line, column);
  } else {
    if (!CharValidator.isEof(secondSymbol)) {
      scanner.unread();
    }
    if (!CharValidator.isEof(firstSymbol)) {
      scanner.unread();
    }
    return tokenizer?.symbolState?.nextToken(scanner, tokenizer);
  }
}