mishamosher_tiktoken 1.0.1
mishamosher_tiktoken: ^1.0.1 copied to clipboard

Published 18 months ago •

→

Tiktoken is a BPE tokeniser for use with OpenAI's models. It exposes APIs used for processing text using tokens.

2 versions

Stable versions of mishamosher_tiktoken

Version		Min Dart SDK	Uploaded	Documentation	Archive
1.0.1		2.15	18 months ago
1.0.0		2.15	18 months ago

Tiktoken is a BPE tokeniser for use with OpenAI's models. It exposes APIs used for processing text using tokens.