tokenize abstract method

List<CodeToken> tokenize(
  1. String code
)

Tokenizes code into a flat list of tokens covering the full input.

Implementation

List<CodeToken> tokenize(String code);