sizeInTokens method

@override Future<int> sizeInTokens(String text)

Runs an invocation of only the tokenization for the LLM, and returns the size (in tokens) of the result.

Implementation

@override
Future<int> sizeInTokens(String text) async {
  _log.fine('Counting tokens of "${text.shorten()}"');
  await _ready;
  _sendPort.send(_LlmInferenceTask.countTokens(text));
  while (true) {
    final response = await _events.next;
    if (response is int) {
      return response;
    } else if (response is String) {
      _log.fine(response);
    } else {
      throw Exception(
        'Unexpected sizeInTokens result of type ${response.runtimeType} : $response',
      );
    }
  }
}