tiktoken 1.0.1 copy "tiktoken: ^1.0.1" to clipboard
tiktoken: ^1.0.1 copied to clipboard

Published 19 months ago • verified publishermarinvitlov.com• Latest: 1.0.3
[unknown platforms]
→

Metadata

Tiktoken is a BPE tokeniser for use with OpenAI's models. It exposes APIs used for processing text using tokens.

More...

1.0.1 #

  • Expose encodingForModel mapping function

1.0.0 #

  • Initial version.

← Metadata

20
likes
0
pub points
88%
popularity

Publisher

verified publishermarinvitlov.com

Metadata

Tiktoken is a BPE tokeniser for use with OpenAI's models. It exposes APIs used for processing text using tokens.

Homepage
Repository (GitHub)
View/report issues

License

unknown (license)

More

Packages that depend on tiktoken

Back