dart_sentencepiece_tokenizer 1.3.2 
dart_sentencepiece_tokenizer: ^1.3.2 copied to clipboard
Published 17 days ago • 
brodykim.work
brodykim.workA lightweight, pure Dart implementation of SentencePiece tokenizer. Supports BPE (Gemma) and Unigram (Llama) algorithms.
8 versions
Stable versions of dart_sentencepiece_tokenizer
| Version | Min Dart SDK | Uploaded | Documentation | Archive | |
|---|---|---|---|---|---|
| 1.3.2 | 3.10 | 17 days ago |  |  | |
| 1.3.1 | 3.10 | 21 days ago | | | |
| 1.3.0 | 3.10 | 2 months ago | | ||
| 1.2.2 | 3.10 | 2 months ago | | ||
| 1.2.1 | 3.10 | 2 months ago | | ||
| 1.2.0 | 3.10 | 3 months ago | | ||
| 1.1.0 | 3.10 | 3 months ago | | ||
| 1.0.0 | 3.10 | 3 months ago | |
Metadata
4
likes
160
points
5.35k
downloads
Documentation
Publisher
Weekly Downloads
Metadata
A lightweight, pure Dart implementation of SentencePiece tokenizer. Supports BPE (Gemma) and Unigram (Llama) algorithms.
Repository (GitHub)
View/report issues
Topics
#nlp #sentencepiece #tokenizer #machine-learning #llm
License
MIT (license)
