Kalan DB

Embedded, offline hybrid search + on-device embedding for Flutter (Android).

HNSW vector search + BM25 keyword search, fused with Reciprocal Rank Fusion (RRF), over a single memory-mapped .vdb file. Queries are embedded on the device at runtime by a small int8 ONNX model — no server, no network, no pre-computed query vectors. Pair it with an LLM for fully local retrieval + (optionally cloud) generation (RAG).

Status: proof-of-concept, exercised end-to-end on a physical arm64 device (Realme RMX3870, Android 16) with bge-small-en-v1.5 int8 embeddings. Android arm64-v8a only in this release.

What's in the box

Layer	Component	Tech
Embedding	TextEmbedder + WordPieceTokenizer (lib/src/embedding/)	flutter_onnxruntime, pure-Dart BERT tokenizer
Dart API	lib/vectordb.dart (KalanDB, EmbeddedKalanDB)	dart:ffi
C++ engine	android/src/main/cpp/vectordb/ → libkalandb.so	C++17, HNSW, BM25, mmap
Storage	single .vdb file	custom binary format, xxHash64 integrity

Install

dependencies:
  kalan_db: ^0.1.0

You also need an embedding model on the device. This package does not bundle one (large binaries don't belong in a pub package); supply model.onnx + vocab.txt yourself — bundle them as assets, or download on first launch. The example app uses bge-small-en-v1.5 int8 (~34 MB, 384-d). Fetch it with:

tools/fetch_model.sh        # downloads bge-small-en-v1.5 int8 + vocab

Usage

import 'package:kalan_db/kalan_db.dart';

// One handle that embeds queries on-device, then hybrid-searches.
final db = await EmbeddedKalanDB.open(
  dbPath:   '$dir/textbooks.vdb',
  modelPath:'$dir/model_quantized.onnx',
  vocabPath:'$dir/vocab.txt',
  // defaults are tuned for bge-small-en-v1.5 (CLS pooling + query instruction)
);

final hits = await db.search(
  'how do you find the roots of a quadratic equation?',
  topK: 5,
  mode: SearchMode.hybrid,         // or vectorOnly / keywordOnly
  filter: const MetadataFilter(subject: 'MATHS', grade: 10),
);
for (final r in hits) {
  print('${r.score.toStringAsFixed(3)}  ${r.chunk.text}');
}
await db.close();

Lower-level pieces are available too: KalanDB (search with a vector/text you provide) and TextEmbedder (embed text yourself). For a different model, set pooling: Pooling.mean and the appropriate instruction prefixes (e.g. e5).

RAG (retrieve → generate)

The example app shows an on-device RAG agent: embed + retrieve locally, then ground a Gemini 2.5 Flash Lite answer on the retrieved chunks. Retrieval is fully offline; only generation calls the network. See example/lib/rag_agent.dart.

Security: never hardcode/commit API keys. The example reads the key from --dart-define=GEMINI_API_KEY=…. Any key shipped in an APK is extractable — for production, proxy LLM calls through your backend.

Build a .vdb (offline)

# host build of the native index builder
cmake -S tools/builder/native -B build/native -G Ninja && cmake --build build/native

# PDF -> chunks -> bge embeddings -> .vdb (build-time embedding MUST match the
# runtime model, so use the same bge model dir)
python3 tools/builder/builder.py \
  --input ../data/Class_10_Mathematics_English.pdf \
  --output example/assets/textbooks.vdb \
  --subject MATHS --grade 10 --book-id 1 --dim 384 \
  --chunk-size 200 --overlap 20 \
  --model tools/builder/models/bge-small-en-v1.5 \
  --vdb-build build/native/vdb_build

Run the example + benchmark (physical device)

cd example
flutter run -d <device-id> --dart-define=GEMINI_API_KEY=<key>   # Search + Ask tabs
flutter test integration_test/vectordb_test.dart    -d <device-id>   # correctness
flutter test integration_test/benchmark_test.dart   -d <device-id>   # KPIs + latency

KPIs

Measured on a Realme RMX3870 (arm64-v8a, Android 16) over a 896-chunk Class-10 Maths corpus, against Gemini-graded relevance judgments (tools/eval/gen_qrels.py), via example/integration_test/benchmark_test.dart. Quality is the mean over 29 eval queries; hybrid wins on every metric:

mode	P@5	R@10	NDCG@10	MRR
vector	0.724	0.625	0.758	0.952
keyword	0.669	0.592	0.707	0.943
hybrid	0.752	0.653	0.781	0.954

stage	median	p95
query embedding (on-device, int8)	37.0 ms	74.1 ms
vector search	1.6 ms	2.3 ms
keyword search	0.5 ms	1.0 ms
hybrid search	2.4 ms	5.3 ms
end-to-end (embed + hybrid)	39.3 ms	—

Model+DB cold load 472 ms; model 34 MB; .vdb 2.63 MB. NDCG@10 meets the PRD §9 target (≥0.78) and MRR far exceeds it (≥0.75). R@10 is bounded here by the pooled-qrels protocol (top-25 candidates graded per query). Regenerate the report any time with the benchmark test; it also writes benchmark_report.json.

Deferred

Tamil pipeline, AES-256-GCM encryption, ARM NEON SIMD, armeabi-v7a/iOS, and int8-vs-fp32 embedding-drift reporting.