tokenize method

  1. @override
List<CodeToken> tokenize(
  1. String code
)
override

Tokenizes code into a flat list of tokens covering the full input.

Implementation

@override
List<CodeToken> tokenize(String code) => _tokenize(
  code,
  keywords: _keywords,
  lineComment: '#',
  blockComment: null,
);