flutter_gemma_embeddings

Runtime-agnostic on-device text embedding pipeline for flutter_gemma: tokenization, task-type prefixing, a background-isolate worker, and pooling/normalization, over the EmbeddingForwardPass seam. Android, iOS, macOS, Linux, Windows, Web.

Since 2.0.0 this package ships no concrete embedding backend — it depends only on flutter_gemma. Pair it with an engine package that implements EmbeddingForwardPass and registers an EmbeddingBackendProvider, e.g. flutter_gemma_litertlm's LiteRtEmbeddingBackend (Gecko / EmbeddingGemma .tflite via the LiteRT C API + dart:ffi). Most apps only ever interact with flutter_gemma_litertlm directly — it re-exports the pieces you register.

Usage

import 'package:flutter_gemma/flutter_gemma.dart';
import 'package:flutter_gemma_litertlm/flutter_gemma_litertlm.dart';

await FlutterGemma.initialize(
  embeddingBackends: [LiteRtEmbeddingBackend()],
);

LiteRtEmbeddingBackend provides the embedding model used by the auto-embedding RAG methods (addDocument / searchSimilar) and by createEmbeddingModel. Pair it with a vector store from flutter_gemma_rag_sqlite or flutter_gemma_rag_qdrant.

Web setup

On web, flutter_gemma_litertlm's embedding backend runs via LiteRT.js, using this package's web/litert_embeddings.js. Add the loader script to your app's web/index.html <head>. Pin a release tag and include a Subresource Integrity hash so a CDN compromise cannot inject code:

<script type="module"
        src="https://cdn.jsdelivr.net/gh/DenisovAV/flutter_gemma@<tag>/packages/flutter_gemma_embeddings/web/litert_embeddings.js"
        integrity="sha384-<hash>"
        crossorigin="anonymous"></script>

Compute the hash for the tag you pin (the browser rejects the script if integrity doesn't match, so don't ship a placeholder): openssl dgst -sha384 -binary web/litert_embeddings.js | openssl base64 -A

Native platforms need no setup — the LiteRT native library is bundled at build time by flutter_gemma_litertlm's Native-Assets hook.

Platforms

Platform	Support
Android / iOS	✅ (via flutter_gemma_litertlm's FFI backend)
macOS / Linux / Windows	✅ (via flutter_gemma_litertlm's FFI backend)
Web	✅ (via flutter_gemma_litertlm's LiteRT.js backend, CDN)

This package itself is pure Dart with no native/FFI code — the concrete backend (and its native library) is owned by whichever engine package you add.

Building a new engine backend

Implement EmbeddingForwardPass (load/run/close/outputDimension/ inputSequenceLength) for your engine, expose a top-level factory tear-off for it, and build an EmbeddingBackendProvider that calls CommonEmbeddingModel.create(descriptor: ForwardPassDescriptor(...), tokenizerPath: ...). Declare EmbeddingOutputContract.pooledFinal if your engine's forward pass already returns the final embedding, or .tokenLevel if it returns raw per-token hidden states for this package's meanPoolAndNormalize to pool. See flutter_gemma_litertlm's lib/src/embedding/ for a worked example.

Troubleshooting

`dlopen` / "library not found" (`libLiteRtLm`)

flutter_gemma_litertlm is the sole owner of the shared native library and bundles it via its build hook. A stale Native-Assets cache after a native version bump can leave the library unbundled, surfacing as an opaque dlopen "no such file" on the first embedding call. Fix with a clean rebuild:

flutter clean
rm -rf ~/Library/Caches/flutter_gemma/native        # macOS / Linux
# Windows: rmdir /s "%LOCALAPPDATA%\flutter_gemma\native"  (path may vary)
flutter pub get

Libraries

embedding_tokenizer: Public, native-only export of the Gemma SentencePiece EmbeddingTokenizer adapter (loadGemmaSentencePieceEmbeddingTokenizer, loadEmbeddingTokenizer, encodeForEmbedding) for native-only leaves that need to import it from a file that is itself native-only (never reached on web) — engine packages' embedding backends (e.g. flutter_gemma_litertlm's LiteRtEmbeddingBackend).
flutter_gemma_embeddings: Runtime-agnostic on-device text embedding pipeline for flutter_gemma.
web_embedding_model: Public export of the web (LiteRT.js) EmbeddingModel — WebEmbeddingModel — for the engine package that builds the web embedding backend (flutter_gemma_litertlm's litert_embedding_backend_web.dart).
wordpiece_embedding_tokenizer: Public, native-only export of WordPieceEmbeddingTokenizer for native-only leaves that need to import it from a file that is itself native-only (never reached on web) — engine packages' embedding backends (e.g. flutter_gemma_onnx's ONNX tokenizer loader, which sniffs a tokenizer.json and routes to either this WordPiece adapter or the Gemma SentencePiece one in embedding_tokenizer.dart).