tiktoken_tokenizer_gpt4o_o1 1.1.0 copy "tiktoken_tokenizer_gpt4o_o1: ^1.1.0" to clipboard
tiktoken_tokenizer_gpt4o_o1: ^1.1.0 copied to clipboard

OpenAI's Tiktoken tokenizer for models: GPT-4, GPT-4o, GPT-4o-mini, o1, o1-mini, and o1-preview.

1
likes
150/ 160
pub points
28%
popularity

We analyzed this package 45 hours ago, and awarded it 150 pub points (of a possible 160):

OK
Follow Dart file conventions
30 / 30trigger folding of the section

10/10 points: Provide a valid pubspec.yaml

5/5 points: Provide a valid README.md

5/5 points: Provide a valid CHANGELOG.md

10/10 points: Use an OSI-approved license

Detected license: BSD-2-Clause.

OK
Provide documentation
20 / 20trigger folding of the section

10/10 points: 20% or more of the public API has dartdoc comments

28 out of 43 API elements (65.1 %) have documentation comments.

Some symbols that are missing documentation: tiktoken_tokenizer_gpt4o_o1, tiktoken_tokenizer_gpt4o_o1.OpenAiModel.encoding, tiktoken_tokenizer_gpt4o_o1.SpecialTokensSet.SpecialTokensSet.all, tiktoken_tokenizer_gpt4o_o1.SpecialTokensSet.SpecialTokensSet.custom, tiktoken_tokenizer_gpt4o_o1.SpecialTokensSet.SpecialTokensSet.empty.

10/10 points: Package has an example

OK
Platform support
20 / 20trigger folding of the section

20/20 points: Supports 6 of 6 possible platforms (iOS, Android, Web, Windows, macOS, Linux)

  • ✓ Android

  • ✓ iOS

  • ✓ Windows

  • ✓ Linux

  • ✓ macOS

  • ✓ Web

0/0 points: WASM compatibility

This package is compatible with runtime wasm, and will be rewarded additional points in a future version of the scoring model.

See https://dart.dev/web/wasm for details.

partial
Pass static analysis
40 / 50trigger folding of the section

40/50 points: code has no errors, warnings, lints, or formatting issues

Found 24 issues. Showing the first 2:

INFO: The variable name '_CACHE' isn't a lowerCamelCase identifier.

lib/tiktoken_tokenizer_gpt4o_o1.dart:117:16

    ╷
117 │   static final _CACHE = <TiktokenEncodingType, TiktokenEncoder>{};
    │                ^^^^^^
    ╵

To reproduce make sure you are using the lints_core and run flutter analyze lib/tiktoken_tokenizer_gpt4o_o1.dart

/tmp/pana_KSEBDI/lib/src/common/byte_array.dart doesn't match the Dart formatter.

To format your files run: dart format .

OK
Support up-to-date dependencies
40 / 40trigger folding of the section

10/10 points: All of the package dependencies are supported in the latest version

Package Constraint Compatible Latest
characters ^1.3.0 1.3.0 1.3.1
flutter flutter 0.0.0 0.0.0
Transitive dependencies
Package Constraint Compatible Latest
collection - 1.18.0 1.19.1
material_color_utilities - 0.11.1 0.12.0
meta - 1.15.0 1.16.0
sky_engine - 0.0.99 0.0.99
vector_math - 2.1.4 2.1.4

To reproduce run dart pub outdated --no-dev-dependencies --up-to-date --no-dependency-overrides.

10/10 points: Package supports latest stable Dart and Flutter SDKs

20/20 points: Compatible with dependency constraint lower bounds

pub downgrade does not expose any static analysis error.

Analyzed with Pana 0.22.15, Flutter 3.24.5, Dart 3.5.4.

Check the analysis log for details.

1
likes
150
pub points
28%
popularity

Publisher

verified publisherglasberg.dev

OpenAI's Tiktoken tokenizer for models: GPT-4, GPT-4o, GPT-4o-mini, o1, o1-mini, and o1-preview.

Repository (GitHub)
View/report issues

Topics

#tokenizer #openai #gpt #gpt-4o #tiktoken

Documentation

API reference

License

BSD-2-Clause (license)

Dependencies

characters, flutter

More

Packages that depend on tiktoken_tokenizer_gpt4o_o1