flutter_gemma_onnx 0.3.0 copy "flutter_gemma_onnx: ^0.3.0" to clipboard
flutter_gemma_onnx: ^0.3.0 copied to clipboard

ONNX Runtime engine for flutter_gemma: GenAI inference on 5 native platforms + web generation (Transformers.js) & embeddings (onnxruntime-web).

0.3.0 #

  • Add web text generation via Transformers.js (@huggingface/transformers) — OnnxEngine web arm (WebGPU/WASM, HF-repo-id models).
  • Fix iOS embeddings: probe ORT GetApi down from v27 (the ORT inside onnxruntime-genai predates 1.27).
  • Fix Windows embeddings: pass the model path to OrtCreateSession as UTF-16 (ORTCHAR_T is wchar_t on Windows).
  • Device-verify embeddings on macOS + Linux + Windows + Android + iOS (unified onnx_embedding_device_test).

0.2.0 #

  • Add web support for OnnxEmbeddingBackend via onnxruntime-web (WebGPU/WASM, WordPiece models).
  • OnnxEngine (text generation) declines on web for now — Transformers.js support is a fast-follow.

0.1.0 #

  • Initial release: OnnxEngine — ORT-GenAI text generation over dart:ffi, macOS/Linux/Windows/Android/iOS arm64.
  • Add OnnxEmbeddingBackend: plain ONNX Runtime embedding forward pass over dart:ffi (no ORT GenAI).
  • Fix ORT/GenAI co-location via ORT_LIB_PATH (dladdr) — no Podfile/packaging step needed.
  • Harden GenAiFfiClient lifecycle: cancel/stop/close races can no longer wedge the mutex or hang a caller.
  • Platform-gate both engines to macOS/Linux/Windows/Android arm64 + iOS arm64; other hosts fail loud, not with a confusing native error.
  • Add OrtClient/OrtFfiClient: injectable ORT 1.27.0 C API seam, fake-testable with zero dlopen.
  • Verified against real all-MiniLM-L6-v2 (WordPiece) and EmbeddingGemma-300M-ONNX (SentencePiece) models on macOS.
0
likes
140
points
0
downloads

Documentation

API reference

Publisher

unverified uploader

Weekly Downloads

ONNX Runtime engine for flutter_gemma: GenAI inference on 5 native platforms + web generation (Transformers.js) & embeddings (onnxruntime-web).

Homepage
Repository (GitHub)
View/report issues

Topics

#gemma #llm #onnx #onnxruntime #on-device

License

MIT (license)

Dependencies

code_assets, crypto, ffi, flutter, flutter_gemma, flutter_gemma_embeddings, hooks, mutex

More

Packages that depend on flutter_gemma_onnx