countTokens method

Future<int> countTokens(List<ChatMessage> messages, { List<Tool>? tools, })

Count tokens in messages using Anthropic's token counting API

API Reference: https://docs.anthropic.com/en/api/messages-count-tokens

This uses Anthropic's dedicated endpoint to accurately count tokens for messages, system prompts, tools, and thinking configurations without sending an actual chat request. Useful for:

• Cost estimation before sending requests
• Staying within model token limits
• Optimizing prompt length

Implementation

Future<int> countTokens(List<ChatMessage> messages,
    {List<Tool>? tools}) async {
  final requestBody = _buildTokenCountRequestBody(messages, tools);

  try {
    final responseData =
        await client.postJson('messages/count_tokens', requestBody);
    return responseData['input_tokens'] as int? ?? 0;
  } catch (e) {
    client.logger.warning('Failed to count tokens: $e');
    // Fallback to rough estimation (4 chars per token)
    final totalChars =
        messages.map((m) => m.content.length).fold(0, (a, b) => a + b);
    return (totalChars / 4).ceil();
  }
}