Tokenizes a string buffer into a list of tokens structures.
buffer
List<Token> tokenizeBuffer(String buffer);