dart_bert_tokenizer 1.0.1 copy "dart_bert_tokenizer: ^1.0.1" to clipboard
dart_bert_tokenizer: ^1.0.1 copied to clipboard

A lightweight, pure Dart implementation of BERT WordPiece tokenizer. 100% compatible with HuggingFace tokenizers.

145/ 160
pub points
78
downloads

We analyzed this package 2 days ago, and awarded it 145 pub points (of a possible 160):

Failed report section
Follow Dart file conventions
25 / 30trigger folding of the section

Passed check 10/10 points: Provide a valid pubspec.yaml

Passed check 5/5 points: Provide a valid README.md

Failed check 0/5 points: Provide a valid CHANGELOG.md

`CHANGELOG.md` does not contain reference to the current version ("1.0.1").

Changelog entries help developers follow the progress of your package. Check out the Dart conventions for Maintaining a package changelog.

Passed check 10/10 points: Use an OSI-approved license

Detected license: MIT.

Partially passed report section
Provide documentation
10 / 20trigger folding of the section

Partially passed check 0/10 points: 20% or more of the public API has dartdoc comments

4 out of 147 API elements (2.7 %) have documentation comments.

Providing good documentation for libraries, classes, functions, and other API elements improves code readability and helps developers find and use your API. Document at least 20% of the public API elements.

To highlight public API members missing documentation consider enabling the public_member_api_docs lint.

Some symbols that are missing documentation: dart_bert_tokenizer, dart_bert_tokenizer.BertPreTokenizer, dart_bert_tokenizer.BertPreTokenizer.BertPreTokenizer.new, dart_bert_tokenizer.BertPreTokenizer.handleChineseChars, dart_bert_tokenizer.BertPreTokenizer.lowercase.

Passed check 10/10 points: Package has an example

Passed report section
Platform support
20 / 20trigger folding of the section

Passed check 20/20 points: Supports 5 of 6 possible platforms (iOS, Android, Web, Windows, macOS, Linux)

  • ✓ Android

  • ✓ iOS

  • ✓ Windows

  • ✓ Linux

  • ✓ macOS

These platforms are not supported:

Package not compatible with platform Web

Because:

  • package:dart_bert_tokenizer/dart_bert_tokenizer.dart that imports:
  • package:dart_bert_tokenizer/src/wordpiece_tokenizer.dart that imports:
  • package:dart_bert_tokenizer/src/vocabulary.dart that imports:
  • dart:io
Passed report section
Pass static analysis
50 / 50trigger folding of the section

Passed check 50/50 points: code has no errors, warnings, lints, or formatting issues

Passed report section
Support up-to-date dependencies
40 / 40trigger folding of the section

Passed check 10/10 points: All of the package dependencies are supported in the latest version

No dependencies.

To reproduce run dart pub outdated --no-dev-dependencies --up-to-date --no-dependency-overrides.

Passed check 10/10 points: Package supports latest stable Dart and Flutter SDKs

Passed check 20/20 points: Compatible with dependency constraint lower bounds

pub downgrade does not expose any static analysis error.

Analyzed with Pana 0.23.3, Dart 3.11.0-200.1.beta.

Check the analysis log for details.

Weekly downloads

Display as:
By versions:
1
likes
145
points
78
downloads

Publisher

verified publisherbrodykim.work

Weekly Downloads

A lightweight, pure Dart implementation of BERT WordPiece tokenizer. 100% compatible with HuggingFace tokenizers.

Repository (GitHub)
View/report issues

Topics

#nlp #bert #tokenizer #machine-learning #wordpiece

Documentation

API reference

License

MIT (license)

More

Packages that depend on dart_bert_tokenizer