ftsText function

String ftsText(
  1. String string
)

Prepares ordinary text for assignment to an FTS field.

This function preprocesses the given string containing ordinary text before it can be assigned to an FTS field to become the subject of FTS queries. The library configures the "unicode61" tokenizer to accept an extended set of characters (see ftsTokenCategories); therefore, the preprocessing step is required to retain standard FTS query behavior for ordinary text. This preprocessing NFC-normalizes the input, extracts the character sequences that match standard FTS tokens (defined by reFtsTextToken), and joins them with ASCII spaces.

The input string should be free from invisible characters that interfere with tokens by either causing unwanted cuts (e.g., U+00AD SHY) or by changing equality (e.g., U+FE0F VS16). Additionally, such characters may block NFC composition.

See also: ftsTerm.

Implementation

String ftsText(String string) => ftsWords(string).join(" ");