dart_sentencepiece_tokenizer 1.4.1 
dart_sentencepiece_tokenizer: ^1.4.1 copied to clipboard
Published 5 days ago • 
brodykim.work
brodykim.workA lightweight pure Dart SentencePiece tokenizer supporting BPE (Gemma), Unigram (Llama), and Hugging Face tokenizer.json pipelines.
11 versions
Stable versions of dart_sentencepiece_tokenizer
| Version | Min Dart SDK | Uploaded | Documentation | Archive | |
|---|---|---|---|---|---|
| 1.4.1 | 3.10 | 5 days ago |  |  | |
| 1.4.0 | 3.10 | 8 days ago | | | |
| 1.3.3 | 3.10 | 9 days ago | | ||
| 1.3.2 | 3.10 | 5 months ago | | ||
| 1.3.1 | 3.10 | 5 months ago | | ||
| 1.3.0 | 3.10 | 7 months ago | | ||
| 1.2.2 | 3.10 | 7 months ago | | ||
| 1.2.1 | 3.10 | 7 months ago | | ||
| 1.2.0 | 3.10 | 7 months ago | | ||
| 1.1.0 | 3.10 | 8 months ago | | ||
| 1.0.0 | 3.10 | 8 months ago | |
Metadata
6
likes
160
points
24.2k
downloads
Documentation
Publisher
Weekly Downloads
Metadata
A lightweight pure Dart SentencePiece tokenizer supporting BPE (Gemma), Unigram (Llama), and Hugging Face tokenizer.json pipelines.
Repository (GitHub)
View/report issues
Topics
#nlp #sentencepiece #tokenizer #machine-learning #llm
License
MIT (license)