tokenize method

List<List<Token>> tokenize(
  1. String code, {
  2. required String language,
})

Raw tokens per line, without theme information.

Implementation

List<List<Token>> tokenize(String code, {required String language}) =>
    grammar(language).tokenize(code);