estimateTokens function

int estimateTokens(
  1. Message message
)

Estimate token count for one message using pi's character heuristic.

  • user / toolResult: text chars + estimatedImageChars per image.
  • assistant: text + thinking chars, plus tool call name and JSON-encoded arguments.

Implementation

int estimateTokens(Message message) {
  final chars = switch (message) {
    UserMessage(:final content) => _textAndImageChars(content),
    AssistantMessage(:final content) => _assistantChars(content),
    ToolResultMessage(:final content) => _textAndImageChars(content),
    _ => 0,
  };
  return (chars / _charsPerToken).ceil();
}