ftsTerm function
Prepares a String for use as an extended FTS token.
This function replaces ftsTermJoiners, if present in the input string,
with spaces, performs NFC normalization, and replaces inner sequences of
characters that do not match reFtsTermToken with ftsTermJoiner. The
resulting token can be used as an indivisible unit in strings assigned to
FTS fields and in FTS queries. It is "extended" because it can contain
characters from the P and Sm categories in addition to the standard
token characters. Such tokens may be used as identifiers, tags, etc.
The input string should be free from invisible characters that interfere
with tokens by either causing unwanted cuts (e.g., U+00AD SHY) or by
changing equality (e.g., U+FE0F VS16). Additionally, such characters may
block NFC composition.
See also: ftsText.
Implementation
String ftsTerm(String string) {
var norm = nfc(string.replaceAll(ftsTermJoiner, " "));
var tokens = reFtsTermToken.allMatches(norm).map((var m) => m.group(0)!);
return tokens.join(ftsTermJoiner);
}