flutter_gemma_embeddings 2.0.0 
flutter_gemma_embeddings: ^2.0.0 copied to clipboard
sashadenisov.devRuntime-agnostic on-device text embedding pipeline (tokenization, isolate worker, pooling/normalization) for flutter_gemma. Pair with an engine package (e.g. flutter_gemma_litertlm) for a concrete Emb [...]
flutter_gemma_embeddings #
Runtime-agnostic on-device text embedding pipeline for
flutter_gemma: tokenization,
task-type prefixing, a background-isolate worker, and pooling/normalization,
over the EmbeddingForwardPass seam. Android, iOS, macOS, Linux, Windows, Web.
Since 2.0.0 this package ships no concrete embedding backend — it depends
only on flutter_gemma. Pair it with an engine package that implements
EmbeddingForwardPass and registers an EmbeddingBackendProvider, e.g.
flutter_gemma_litertlm's
LiteRtEmbeddingBackend (Gecko / EmbeddingGemma .tflite via the LiteRT C
API + dart:ffi). Most apps only ever interact with flutter_gemma_litertlm
directly — it re-exports the pieces you register.
Usage #
import 'package:flutter_gemma/flutter_gemma.dart';
import 'package:flutter_gemma_litertlm/flutter_gemma_litertlm.dart';
await FlutterGemma.initialize(
embeddingBackends: [LiteRtEmbeddingBackend()],
);
LiteRtEmbeddingBackend provides the embedding model used by the auto-embedding
RAG methods (addDocument / searchSimilar) and by createEmbeddingModel. Pair
it with a vector store from flutter_gemma_rag_sqlite or
flutter_gemma_rag_qdrant.
Web setup #
On web, flutter_gemma_litertlm's embedding backend runs via LiteRT.js, using
this package's web/litert_embeddings.js. Add the loader script to your app's
web/index.html <head>. Pin a release tag and include a Subresource Integrity
hash so a CDN compromise cannot inject code:
<script type="module"
src="https://cdn.jsdelivr.net/gh/DenisovAV/flutter_gemma@<tag>/packages/flutter_gemma_embeddings/web/litert_embeddings.js"
integrity="sha384-<hash>"
crossorigin="anonymous"></script>
Compute the hash for the tag you pin (the browser rejects the script if
integritydoesn't match, so don't ship a placeholder):openssl dgst -sha384 -binary web/litert_embeddings.js | openssl base64 -A
Native platforms need no setup — the LiteRT native library is bundled at build
time by flutter_gemma_litertlm's Native-Assets hook.
Platforms #
| Platform | Support |
|---|---|
| Android / iOS | ✅ (via flutter_gemma_litertlm's FFI backend) |
| macOS / Linux / Windows | ✅ (via flutter_gemma_litertlm's FFI backend) |
| Web | ✅ (via flutter_gemma_litertlm's LiteRT.js backend, CDN) |
This package itself is pure Dart with no native/FFI code — the concrete backend (and its native library) is owned by whichever engine package you add.
Building a new engine backend #
Implement EmbeddingForwardPass (load/run/close/outputDimension/
inputSequenceLength) for your engine, expose a top-level factory tear-off
for it, and build an EmbeddingBackendProvider that calls
CommonEmbeddingModel.create(descriptor: ForwardPassDescriptor(...), tokenizerPath: ...).
Declare EmbeddingOutputContract.pooledFinal if your engine's forward pass
already returns the final embedding, or .tokenLevel if it returns raw
per-token hidden states for this package's meanPoolAndNormalize to pool.
See flutter_gemma_litertlm's lib/src/embedding/ for a worked example.
Troubleshooting #
dlopen / "library not found" (libLiteRtLm) #
flutter_gemma_litertlm is the sole owner of the shared native library and
bundles it via its build hook. A stale Native-Assets cache after a native
version bump can leave the library unbundled, surfacing as an opaque dlopen
"no such file" on the first embedding call. Fix with a clean rebuild:
flutter clean
rm -rf ~/Library/Caches/flutter_gemma/native # macOS / Linux
# Windows: rmdir /s "%LOCALAPPDATA%\flutter_gemma\native" (path may vary)
flutter pub get
Metadata
Documentation
Publisher
Weekly Downloads
Metadata
Runtime-agnostic on-device text embedding pipeline (tokenization, isolate worker, pooling/normalization) for flutter_gemma. Pair with an engine package (e.g. flutter_gemma_litertlm) for a concrete EmbeddingBackendProvider.
Homepage
Repository (GitHub)
View/report issues
Topics
#embeddings #gemma #litert #rag #on-device
License
MIT (license)
Dependencies
dart_sentencepiece_tokenizer, flutter, flutter_gemma