PhoenixDB

An ACID-compliant embedded database engine written in Rust and exposed to Dart/Flutter through a zero-overhead dart:ffi layer. Use it as a key/value store, as typed preferences, for vector similarity search, or through a small SQL front end.

Features

Pillar	Implementation
Index	B+Tree, 4 KiB slotted pages, configurable fill factors (min 50%, max 100%)
Integrity	CRC32 on every page, verified on every read; corruption returns `Err`, never bad data
Persistence	`mmap` for zero-copy reads + write-ahead log with `fsync`/`sync_all` at commit
Concurrency	MVCC snapshot isolation — many concurrent readers, single writer via `parking_lot::RwLock`
Transactions	`Begin` / `Insert` / `Delete` / `Commit` / `Rollback` WAL records with crash recovery
SQL	Optional: `CREATE`/`INSERT`/`SELECT`/`UPDATE`/`DELETE` with `WHERE`, `ORDER BY`, `LIMIT`
Vector search	HNSW k-NN over memory-mapped `f32` vectors; cosine / L2 / dot product; runtime-dispatched AVX2+FMA and NEON kernels
FFI	Pure C ABI (`extern "C"`), header auto-generated by `cbindgen`
Dart	`NativeFinalizer` for automatic cleanup, `Uint8List` keys/values, isolate-based async API

Available as libraries, not yet in the engine's path

These modules are implemented and tested, but the storage engine does not route through them yet. They are usable directly from Rust; they do not change how PhoenixDatabase behaves today.

Module	State
`lsm`	MemTable, block-based SSTables with Bloom filters, leveled compaction, crash-safe durable manifest. `Database` still writes through the B+Tree only.
`security::encryption`	AES-256-GCM page encryption (encrypt-then-checksum). Not wired into the pager.
`security::rbac`, `security::audit`	Role checks with constant-time comparison; append-only audit log. Not enforced at the FFI boundary.
`observability`	p50/p99/p999 histograms and tracing spans. Not emitted by the engine.

Key and value length validation from security is enforced on every FFI call.

Feature flags

The default build stays small for Flutter — it pulls in no heavy dependencies. Opt into what you need:

phoenixdb = { version = "2.0", features = ["sql", "encryption"] }

Flag	Adds
`sql`	SQL lexer, parser, and executor
`encryption`	AES-256-GCM encryption at rest
`json`	`serde_json`, for structured-value indexing
`metrics`	Latency histograms and counters
`full`	Everything above

Architecture

  Dart (dart:ffi)  ->  ffi.rs (C ABI, validation)  ->  Database
                                                          |
                                       +------------------+------------------+
                                       |                  |                  |
                                    txn.rs             btree.rs            wal.rs
                                   (MVCC)             (index)           (durability)
                                       |                  |
                                       +------ pager.rs --+  (cache + CRC + mmap)

Page layout (4096 bytes)

Offset	Size	Field
0	4	`crc32` — checksum of bytes `[4..4096]`
4	4	`page_id`
8	4	`parent_id`
12	4	`extra` (next leaf / leftmost child / next overflow)
16	2	`num_keys`
18	2	`cell_start`
20	1	`page_type`
21	1	`is_leaf`
22	2	`flags`
24	8	`lsn`

A slot directory grows upward from byte 32 while variable-length cells grow downward from byte 4096.

Platform support

Platform	Supported	Prebuilt binary ships	Rust needed to build your app
Android — arm64-v8a, armeabi-v7a, x86_64, x86	✅	✅	No
iOS — device + simulator	✅	❌	Yes (+ Xcode)
macOS — Apple silicon + Intel	✅	❌	Yes (+ Xcode)
Linux — x86_64, aarch64	✅	❌	Yes
Windows — x86_64	✅	❌	Yes
Web	❌	—	—

Web is not supported and cannot be. The engine is native code reached over dart:ffi, which does not exist in a browser, and the browser has no filesystem for a durable page store. A web build fails to compile rather than degrading at runtime.

Android is the one platform that needs nothing extra: the NDK cross-compiles cleanly from any host, so the four ABIs ship prebuilt and the Gradle plugin packages them into your APK/AAB. Every other platform compiles the engine from source during your app build — Apple platforms because linking needs an Xcode SDK and code signing that only exist on a developer's Mac, desktop because the prebuilt libraries in native/ target the host rather than your app bundle.

Install the toolchain once on any machine that builds for iOS, macOS, Linux, or Windows:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh   # or rustup.rs on Windows

For plain Dart (no Flutter), the prebuilt libraries in native/<triple>/ are used directly and no Rust toolchain is required — see Plain Dart.

Installation

flutter pub add phoenixdb

Or add it to pubspec.yaml:

dependencies:
  phoenixdb: ^2.0.0

Use flutter pub, not dart pub. PhoenixDB is a Flutter FFI plugin, so its pubspec declares flutter.plugin.platforms. pub only accepts that key alongside a flutter: SDK bound, and that bound makes standalone dart pub refuse the package:
Because phoenixdb requires the Flutter SDK, version solving failed.
This is a packaging rule, not a code dependency — nothing under lib/ imports package:flutter. Flutter bundles a Dart SDK, so a pure Dart CLI or server works fine: resolve with flutter pub get, then run it with dart run as usual.

How the native library is delivered

PhoenixDB is an FFI plugin, so the engine ships as a native artifact rather than Dart source. What happens per platform:

Platform	Delivery	Built when
Android	Prebuilt `.so` in `android/src/main/jniLibs/<abi>/`, packaged into your APK/AAB	Shipped prebuilt
iOS	Static archive compiled by `rust/build-apple.sh`, linked into the app binary	`flutter build ios`
macOS	Universal static archive (arm64 + x86_64)	`flutter build macos`
Linux	`cargo build` driven by CMake, bundled next to the executable	`flutter build linux`
Windows	`cargo build` driven by CMake, DLL bundled with the app	`flutter build windows`

Android ships prebuilt because the NDK cross-compiles cleanly from any host. Apple platforms build from source at app-build time because linking requires an Xcode SDK and code signing that only exist on the developer's Mac.

Flutter apps need no extra setup — the platform folders are wired up automatically. Building for iOS/macOS/Linux/Windows additionally requires a Rust toolchain on the build machine:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

Plain Dart (non-Flutter)

There is no app bundle to carry the library, so build it once and leave it where the loader can find it:

git clone https://github.com/ayoubzulfiqar/phoenixdb && cd phoenixdb
./build.sh                    # installs into native/

The loader searches native/<target-triple>/, then native/, then rust/target/{release,debug}/. You can also point at it explicitly:

final db = PhoenixDatabase.open('data.pdb', libraryPath: '/opt/lib/libphoenixdb.so');

Building

./build.sh              # release build, installs into native/
./build.sh --debug
./build.sh --all        # every supported platform (needs cargo-zigbuild)
./build.sh --target aarch64-apple-darwin

.\build.ps1
.\build.ps1 -Target x86_64-pc-windows-msvc

Cross-compilation

Cross builds need a linker that emits the target's object format. The most portable option is cargo-zigbuild, which uses Zig's bundled clang and needs no Xcode installation for macOS:

cargo install cargo-zigbuild
winget install zig.zig     # or: brew install zig / apt install zig

rustup target add x86_64-unknown-linux-gnu aarch64-unknown-linux-gnu \
                  x86_64-apple-darwin aarch64-apple-darwin
./build.sh --all

build.sh detects cargo-zigbuild and uses it automatically for any target that is not the host. Without it, the script falls back to plain cargo and honours the standard CC / AR variables:

CC=aarch64-linux-gnu-gcc AR=aarch64-linux-gnu-ar \
  ./build.sh --target aarch64-unknown-linux-gnu

Prebuilt binaries

Cross-compiled libraries are installed into native/<triple>/, and the Dart loader prefers the directory matching the running architecture before falling back to the flat native/ layout:

Target triple	Artifact	Format
`x86_64-unknown-linux-gnu`	`libphoenixdb.so`	ELF 64-bit x86-64
`aarch64-unknown-linux-gnu`	`libphoenixdb.so`	ELF 64-bit ARM aarch64
`x86_64-apple-darwin`	`libphoenixdb.dylib`	Mach-O 64-bit x86_64
`aarch64-apple-darwin`	`libphoenixdb.dylib`	Mach-O 64-bit arm64
`x86_64-pc-windows-gnu`	`phoenixdb.dll`	PE32+ x86-64

All five export the same 19-function C ABI. Binaries are build output and are not tracked in git — run ./build.sh --all to regenerate them.

The C header is regenerated into native/include/phoenixdb.h on every build.

Usage

Synchronous

import 'package:phoenixdb/phoenixdb.dart';

final db = PhoenixDatabase.open('data.pdb');
db.insert(utf8Key('hello'), utf8Value('world'));
print(utf8Decode(db.getOrThrow(utf8Key('hello'))));  // world

db.transaction((txn) {
  db.insert(utf8Key('a'), utf8Value('1'), txnId: txn);
  db.insert(utf8Key('b'), utf8Value('2'), txnId: txn);
});

db.close();

Asynchronous (recommended for Flutter)

Every call runs on a dedicated worker isolate, so disk I/O never blocks the UI:

final db = await AsyncPhoenixDB.open('data.pdb');
await db.insert(utf8Key('k'), utf8Value('v'));
final value = await db.get(utf8Key('k'));
await db.close();

Preferences

A shared_preferences-style typed facade over the same store — durable, transactional, and with no platform channel:

final prefs = await PhoenixPrefs.open('settings.pdb');

await prefs.setString('theme', 'dark');
await prefs.setInt('launches', 42);
await prefs.setBool('onboarded', true);

print(await prefs.getString('theme'));   // dark
print(await prefs.getInt('missing'));    // null

await prefs.close();

SQL

Requires a library built with the sql feature; check db.supportsSql if you target lean embedded builds.

final db = PhoenixDatabase.open('app.pdb');

db.query('CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT NOT NULL)');
db.query("INSERT INTO users VALUES (1, 'alice'), (2, 'bob')");

final result = db.query('SELECT name FROM users WHERE id > 1 ORDER BY name');
print(result.rows);           // [[bob]]
print(result.scalar);         // bob   (single cell shorthand)
print(result.asMaps);         // [{name: bob}]

db.query("UPDATE users SET name = 'carol' WHERE id = 2").affected;  // 1
db.close();

On a UI isolate, use the async form so parsing and execution stay off the main thread:

final db = await AsyncPhoenixDB.open('app.pdb');
final r = await db.query('SELECT name FROM users WHERE id = 1');
print(r.scalar);  // alice

SQL rows and raw key/value entries live in the same database without interfering — the SQL layer namespaces its keys.

Supported: CREATE TABLE / DROP TABLE (with IF [NOT] EXISTS), INSERT, SELECT (projection, WHERE with AND/OR, ORDER BY, LIMIT), UPDATE, DELETE. Not supported: joins, subqueries, aggregates, indexes on expressions.

Vector search

An embedded HNSW k-NN index for local-first semantic search — no server, no second database.

final index = PhoenixVectorDB.open(
  'vectors.pvec',
  dimensions: 384,          // fixed at creation; must match on reopen
  metric: VectorMetric.cosine,
);

index.insert('doc-1', embedding);           // Float32List
index.insertAll({'doc-2': a, 'doc-3': b});

for (final match in index.search(VectorQuery(query, k: 5))) {
  print('${match.id}  ${match.score.toStringAsFixed(4)}');
}

index.save();   // syncs vectors + writes the HNSW snapshot
index.close();

On a UI isolate, use the async form — every call runs on a worker isolate, so a search cannot drop a frame:

final index = await AsyncPhoenixVectorDB.open('vectors.pvec', dimensions: 384);
final hits = await index.search(VectorQuery(query, k: 5));
await index.close();

Metric	`distance`	`score` (higher is better)
`VectorMetric.cosine`	`1 - cos(θ)`, in `[0, 2]`	cosine similarity, `[-1, 1]`
`VectorMetric.euclidean`	true L2 distance	`1 / (1 + d)`, in `(0, 1]`
`VectorMetric.dotProduct`	`-(a·b)`	the inner product

Notes:

Vectors are the source of truth. The graph is snapshotted separately and rebuilt automatically if the snapshot is missing, stale or corrupt.
Small indexes are exact. Below 512 live vectors the engine scans exhaustively, so a handful of documents gives exact answers, not approximate ones.
Removal is a tombstone, so ids in the graph stay stable. Call compact() once stats().deletedRatio passes roughly 0.3.
efSearch trades latency for recall. The default (64) suits k ≤ 10; raise it for higher recall on large, high-dimensional collections.
SIMD is automatic. index.kernel reports avx2+fma, neon or portable. AVX2 is selected at runtime, never baked in, so a published binary still runs on pre-2013 x86 hardware.

Security model

FFI guardrails — every entry point validates pointer non-nullness and length limits (key ≤ 1 MiB, value ≤ 10 MiB) before any dereference, and returns -2 for violations.
Handle tagging — each handle carries a magic tag checked in constant time, so use-after-free and foreign pointers are rejected instead of dereferenced.
Constant-time comparison — security::ct_eq / ct_eq_u64 fold differences with bitwise XOR masking, with no early exit for equal-length inputs.
Panic containment — every FFI body runs inside catch_unwind; a panic becomes status -7 and never unwinds into Dart.
Single free path — Rust-allocated memory is released only by phoenix_buffer_free / phoenix_string_free, wired into Dart's NativeFinalizer.

Status codes

Code	Meaning
`0`	Success
`-1`	Unclassified error
`-2`	Invalid argument (null pointer / length limit)
`-3`	Key not found
`-4`	Corruption (CRC mismatch)
`-5`	I/O error
`-6`	Write-write conflict — retry
`-7`	Panic caught at the boundary
`-8`	Unknown transaction
`-9`	Capacity exceeded

Testing

cd rust && cargo test          # 86 tests: unit + FFI safety + ACID integration
dart test                      # 33 tests across the sync and async APIs
dart analyze --fatal-infos     # clean

Fuzzing

Three libfuzzer harnesses live in rust/fuzz/:

cargo +nightly fuzz run fuzz_btree -- -max_total_time=60   # vs a BTreeMap oracle
cargo +nightly fuzz run fuzz_page  -- -max_total_time=60   # arbitrary page bytes
cargo +nightly fuzz run fuzz_ffi   -- -max_total_time=60   # hostile FFI inputs

Limits

Limit	Value
Page size	4096 bytes
Max key (FFI)	1 MiB
Max key (B+Tree structural)	1 KiB
Max value	10 MiB
Max inline value	1 KiB (larger spills to overflow pages)
Max vector dimensions	65 536
Max vector id	128 bytes
Max `k` per search	4096

License

BSD-3-Clause