sizeInTokens abstract method

Future<int> sizeInTokens(
  1. String text
)

Runs an invocation of only the tokenization for the LLM, and returns the size (in tokens) of the result.

Implementation

Future<int> sizeInTokens(String text);