tokenise function

List<String> tokenise(
  1. String normalised
)

Split an already-normaliseForMatch'd string on single-space boundaries into a token bag.

Implementation

List<String> tokenise(String normalised) {
  if (normalised.isEmpty) return const <String>[];
  return normalised
      .split(' ')
      .where((t) => t.isNotEmpty)
      .toList(growable: false);
}