langchain_tiktoken 1.0.1 
langchain_tiktoken: ^1.0.1 copied to clipboard
dragonx.cloudA BPE tokeniser for use with OpenAI's models. It exposes APIs used for processing text using tokens.
We analyzed this package 8 days ago, and awarded it 150 pub points (of a possible 160):
10/10 points: Provide a valid pubspec.yaml
Issue tracker URL doesn't exist.
At the time of the analysis https://github.com/orenagiv/langchain_tiktoken/issues?q=is%3Aissue+is%3Aopen+ was unreachable. Make sure that the website is reachable via HEAD requests.
5/5 points: Provide a valid README.md
5/5 points: Provide a valid CHANGELOG.md
10/10 points: Use an OSI-approved license
Detected license: MIT.
10/10 points: 20% or more of the public API has dartdoc comments
20 out of 28 API elements (71.4 %) have documentation comments.
Some symbols that are missing documentation: tiktoken.SpecialTokensSet.SpecialTokensSet.all, tiktoken.SpecialTokensSet.SpecialTokensSet.custom, tiktoken.SpecialTokensSet.SpecialTokensSet.empty, tiktoken.SpecialTokensSet.isAll, tiktoken.SpecialTokensSet.set.
10/10 points: Package has an example
20/20 points: Supports 6 of 6 possible platforms (iOS, Android, Web, Windows, macOS, Linux)
-
✓ Android
-
✓ iOS
-
✓ Windows
-
✓ Linux
-
✓ macOS
-
✓ Web
0/0 points: WASM compatibility
This package is compatible with runtime wasm, and will be rewarded additional points in a future version of the scoring model.
See https://dart.dev/web/wasm for details.
40/50 points: code has no errors, warnings, lints, or formatting issues
Found 5 issues. Showing the first 2:
INFO: A value for optional parameter 'explicitNVocab' isn't ever given.
lib/src/core_bpe_constructor.dart:40:10
╷
40 │ this.explicitNVocab,
│ ^^^^^^^^^^^^^^
╵
To reproduce make sure you are using the lints_core and run dart analyze lib/src/core_bpe_constructor.dart
INFO: Dangling library doc comment.
lib/src/ranks/cl100k_base.tiktoken.dart:1:1
╷
1 │ ┌ /// Placeholder file
2 │ │ ///
3 │ │ /// Use `download_ranks.dart` script under the `\tools` directory
4 │ └ /// to download required files.
╵
To reproduce make sure you are using the lints_core and run dart analyze lib/src/ranks/cl100k_base.tiktoken.dart
10/10 points: All of the package dependencies are supported in the latest version
No dependencies.
To reproduce run dart pub outdated --no-dev-dependencies --up-to-date --no-dependency-overrides.
10/10 points: Package supports latest stable Dart and Flutter SDKs
20/20 points: Compatible with dependency constraint lower bounds
pub downgrade does not expose any static analysis error.
Analyzed with Pana 0.22.15, Dart 3.5.4.
Check the analysis log for details.
Metadata
Publisher
Metadata
A BPE tokeniser for use with OpenAI's models. It exposes APIs used for processing text using tokens.
Repository (GitHub)
View/report issues
Documentation
License
MIT (license)
