tiktoken_tokenizer_gpt4o_o1 1.2.1 copy "tiktoken_tokenizer_gpt4o_o1: ^1.2.1" to clipboard
tiktoken_tokenizer_gpt4o_o1: ^1.2.1 copied to clipboard

OpenAI's Tiktoken tokenizer for models: GPT-4, GPT-4o, GPT-4o-mini, o1, o1-mini, and o1-preview.

1.2.1 #

  • Dart-only.

1.1.0 #

  • Enums OpenAiModel and TiktokenEncodingType now have toJson and fromJson methods.

1.0.2 #

  • Supports: Gpt-4, Gpt-4o, Gpt-4o-mini, o1, o1-mini, and o1-preview.
2
likes
140
points
134
downloads

Publisher

verified publisherglasberg.dev

Weekly Downloads

OpenAI's Tiktoken tokenizer for models: GPT-4, GPT-4o, GPT-4o-mini, o1, o1-mini, and o1-preview.

Repository (GitHub)

Topics

#tokenizer #openai #gpt #gpt-4o #tiktoken

Documentation

API reference

License

BSD-2-Clause (license)

Dependencies

characters

More

Packages that depend on tiktoken_tokenizer_gpt4o_o1