idDroppedTokens function

List<String> idDroppedTokens(
  1. String text
)

The data-bearing tokens idTextSlug drops from text (every whitespace-separated token whose first ASCII alphanumeric is a digit), in order: 15 Oct ₹500 paid → [15, ₹500].

Implementation

List<String> idDroppedTokens(String text) {
  final dropped = <String>[];
  for (final tok in text.split(RegExp(r'\s+'))) {
    if (tok.isEmpty) continue;
    for (var i = 0; i < tok.length; i++) {
      final c = tok.codeUnitAt(i);
      final isLetter = (c >= 0x41 && c <= 0x5A) || (c >= 0x61 && c <= 0x7A);
      final isDigit = c >= 0x30 && c <= 0x39;
      if (isDigit) dropped.add(tok);
      if (isLetter || isDigit) break;
    }
  }
  return dropped;
}