phoenixdb 2.1.0

ACID-compliant embedded database for Dart and Flutter, written in Rust with dart:ffi. B+Tree storage, MVCC transactions, a write-ahead log, HNSW vector search and optional SQL.

PhoenixDB

An ACID-compliant embedded database engine written in Rust and exposed to Dart/Flutter through a zero-overhead dart:ffi layer. Use it as a key/value store, as typed preferences, for vector similarity search, or through a small SQL front end.

Features

Pillar	Implementation
Index	B+Tree, 4 KiB slotted pages, configurable fill factors (min 50%, max 100%)
Integrity	CRC32 on every page, verified on every read; corruption returns Err, never bad data
Persistence	mmap for zero-copy reads + write-ahead log with fsync/sync_all at commit
Concurrency	MVCC snapshot isolation — many concurrent readers, single writer via parking_lot::RwLock
Transactions	Begin / Insert / Delete / Commit / Rollback WAL records with crash recovery
SQL	Optional: CREATE/INSERT/SELECT/UPDATE/DELETE with WHERE, ORDER BY, LIMIT
Vector search	HNSW k-NN over memory-mapped f32 vectors; cosine / L2 / dot product; runtime-dispatched AVX2+FMA and NEON kernels
FFI	Pure C ABI (extern "C"), header auto-generated by cbindgen
Dart	NativeFinalizer for automatic cleanup, Uint8List keys/values, isolate-based async API

Available as libraries, not yet in the engine's path

These modules are implemented and tested, but the storage engine does not route through them yet. They are usable directly from Rust; they do not change how PhoenixDatabase behaves today.

Module	State
lsm	MemTable, block-based SSTables with Bloom filters, leveled compaction, crash-safe durable manifest. Database still writes through the B+Tree only.
security::encryption	AES-256-GCM page encryption (encrypt-then-checksum). Not wired into the pager.
security::rbac, security::audit	Role checks with constant-time comparison; append-only audit log. Not enforced at the FFI boundary.
observability	p50/p99/p999 histograms and tracing spans. Not emitted by the engine.

Key and value length validation from security is enforced on every FFI call.

Feature flags

The default build stays small for Flutter — it pulls in no heavy dependencies. Opt into what you need:

phoenixdb = { version = "2.0", features = ["sql", "encryption"] }

Flag	Adds
sql	SQL lexer, parser, and executor
encryption	AES-256-GCM encryption at rest
json	serde_json, for structured-value indexing
metrics	Latency histograms and counters
full	Everything above

Architecture

  Dart (dart:ffi)  ->  ffi.rs (C ABI, validation)  ->  Database
                                                          |
                                       +------------------+------------------+
                                       |                  |                  |
                                    txn.rs             btree.rs            wal.rs
                                   (MVCC)             (index)           (durability)
                                       |                  |
                                       +------ pager.rs --+  (cache + CRC + mmap)

Page layout (4096 bytes)

Offset	Size	Field
0	4	crc32 — checksum of bytes [4..4096]
4	4	page_id
8	4	parent_id
12	4	extra (next leaf / leftmost child / next overflow)
16	2	num_keys
18	2	cell_start
20	1	page_type
21	1	is_leaf
22	2	flags
24	8	lsn

A slot directory grows upward from byte 32 while variable-length cells grow downward from byte 4096.

Platform support

Platform	Supported	Prebuilt binary ships	Rust needed to build your app
Android — arm64-v8a, armeabi-v7a, x86_64, x86	✅	✅	No
iOS — device + simulator	✅	❌	Yes (+ Xcode)
macOS — Apple silicon + Intel	✅	❌	Yes (+ Xcode)
Linux — x86_64, aarch64	✅	❌	Yes
Windows — x86_64	✅	❌	Yes
Web	❌	—	—

Web is not supported and cannot be. The engine is native code reached over dart:ffi, which does not exist in a browser, and the browser has no filesystem for a durable page store. A web build fails to compile rather than degrading at runtime.

Android is the one platform that needs nothing extra: the NDK cross-compiles cleanly from any host, so the four ABIs ship prebuilt and the Gradle plugin packages them into your APK/AAB. Every other platform compiles the engine from source during your app build — Apple platforms because linking needs an Xcode SDK and code signing that only exist on a developer's Mac, desktop because the prebuilt libraries in native/ target the host rather than your app bundle.

Install the toolchain once on any machine that builds for iOS, macOS, Linux, or Windows:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh   # or rustup.rs on Windows

For plain Dart (no Flutter), the prebuilt libraries in native/<triple>/ are used directly and no Rust toolchain is required — see Plain Dart.

Installation

flutter pub add phoenixdb

Or add it to pubspec.yaml:

dependencies:
  phoenixdb: ^2.0.0

Use flutter pub, not dart pub. PhoenixDB is a Flutter FFI plugin, so its pubspec declares flutter.plugin.platforms. pub only accepts that key alongside a flutter: SDK bound, and that bound makes standalone dart pub refuse the package:

Because phoenixdb requires the Flutter SDK, version solving failed.

This is a packaging rule, not a code dependency — nothing under lib/ imports package:flutter. Flutter bundles a Dart SDK, so a pure Dart CLI or server works fine: resolve with flutter pub get, then run it with dart run as usual.

How the native library is delivered

PhoenixDB is an FFI plugin, so the engine ships as a native artifact rather than Dart source. What happens per platform:

Platform	Delivery	Built when
Android	Prebuilt .so in android/src/main/jniLibs/<abi>/, packaged into your APK/AAB	Shipped prebuilt
iOS	Static archive compiled by rust/build-apple.sh, linked into the app binary	flutter build ios
macOS	Universal static archive (arm64 + x86_64)	flutter build macos
Linux	cargo build driven by CMake, bundled next to the executable	flutter build linux
Windows	cargo build driven by CMake, DLL bundled with the app	flutter build windows

Android ships prebuilt because the NDK cross-compiles cleanly from any host. Apple platforms build from source at app-build time because linking requires an Xcode SDK and code signing that only exist on the developer's Mac.

Flutter apps need no extra setup — the platform folders are wired up automatically. Building for iOS/macOS/Linux/Windows additionally requires a Rust toolchain on the build machine:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

Plain Dart (non-Flutter)

There is no app bundle to carry the library, so build it once and leave it where the loader can find it:

git clone https://github.com/ayoubzulfiqar/phoenixdb && cd phoenixdb
./build.sh                    # installs into native/

The loader searches native/<target-triple>/, then native/, then rust/target/{release,debug}/. You can also point at it explicitly:

final db = PhoenixDatabase.open('data.pdb', libraryPath: '/opt/lib/libphoenixdb.so');

Building

./build.sh              # release build, installs into native/
./build.sh --debug
./build.sh --all        # every supported platform (needs cargo-zigbuild)
./build.sh --target aarch64-apple-darwin

.\build.ps1
.\build.ps1 -Target x86_64-pc-windows-msvc

Cross-compilation

Cross builds need a linker that emits the target's object format. The most portable option is cargo-zigbuild, which uses Zig's bundled clang and needs no Xcode installation for macOS:

cargo install cargo-zigbuild
winget install zig.zig     # or: brew install zig / apt install zig

rustup target add x86_64-unknown-linux-gnu aarch64-unknown-linux-gnu \
                  x86_64-apple-darwin aarch64-apple-darwin
./build.sh --all

build.sh detects cargo-zigbuild and uses it automatically for any target that is not the host. Without it, the script falls back to plain cargo and honours the standard CC / AR variables:

CC=aarch64-linux-gnu-gcc AR=aarch64-linux-gnu-ar \
  ./build.sh --target aarch64-unknown-linux-gnu

Prebuilt binaries

Cross-compiled libraries are installed into native/<triple>/, and the Dart loader prefers the directory matching the running architecture before falling back to the flat native/ layout:

Target triple	Artifact	Format
x86_64-unknown-linux-gnu	libphoenixdb.so	ELF 64-bit x86-64
aarch64-unknown-linux-gnu	libphoenixdb.so	ELF 64-bit ARM aarch64
x86_64-apple-darwin	libphoenixdb.dylib	Mach-O 64-bit x86_64
aarch64-apple-darwin	libphoenixdb.dylib	Mach-O 64-bit arm64
x86_64-pc-windows-gnu	phoenixdb.dll	PE32+ x86-64

All five export the same 19-function C ABI. Binaries are build output and are not tracked in git — run ./build.sh --all to regenerate them.

The C header is regenerated into native/include/phoenixdb.h on every build.

Usage

Synchronous

import 'package:phoenixdb/phoenixdb.dart';

final db = PhoenixDatabase.open('data.pdb');
db.insert(utf8Key('hello'), utf8Value('world'));
print(utf8Decode(db.getOrThrow(utf8Key('hello'))));  // world

db.transaction((txn) {
  db.insert(utf8Key('a'), utf8Value('1'), txnId: txn);
  db.insert(utf8Key('b'), utf8Value('2'), txnId: txn);
});

db.close();

Asynchronous (recommended for Flutter)

Every call runs on a dedicated worker isolate, so disk I/O never blocks the UI:

final db = await AsyncPhoenixDB.open('data.pdb');
await db.insert(utf8Key('k'), utf8Value('v'));
final value = await db.get(utf8Key('k'));
await db.close();

Preferences

A shared_preferences-style typed facade over the same store — durable, transactional, and with no platform channel:

final prefs = await PhoenixPrefs.open('settings.pdb');

await prefs.setString('theme', 'dark');
await prefs.setInt('launches', 42);
await prefs.setBool('onboarded', true);

print(await prefs.getString('theme'));   // dark
print(await prefs.getInt('missing'));    // null

await prefs.close();

SQL

Requires a library built with the sql feature; check db.supportsSql if you target lean embedded builds.

final db = PhoenixDatabase.open('app.pdb');

db.query('CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT NOT NULL)');
db.query("INSERT INTO users VALUES (1, 'alice'), (2, 'bob')");

final result = db.query('SELECT name FROM users WHERE id > 1 ORDER BY name');
print(result.rows);           // [[bob]]
print(result.scalar);         // bob   (single cell shorthand)
print(result.asMaps);         // [{name: bob}]

db.query("UPDATE users SET name = 'carol' WHERE id = 2").affected;  // 1
db.close();

On a UI isolate, use the async form so parsing and execution stay off the main thread:

final db = await AsyncPhoenixDB.open('app.pdb');
final r = await db.query('SELECT name FROM users WHERE id = 1');
print(r.scalar);  // alice

SQL rows and raw key/value entries live in the same database without interfering — the SQL layer namespaces its keys.

Supported: CREATE TABLE / DROP TABLE (with IF [NOT] EXISTS), INSERT, SELECT (projection, WHERE with AND/OR, ORDER BY, LIMIT), UPDATE, DELETE. Not supported: joins, subqueries, aggregates, indexes on expressions.

Vector search

An embedded HNSW k-NN index for local-first semantic search — no server, no second database.

final index = PhoenixVectorDB.open(
  'vectors.pvec',
  dimensions: 384,          // fixed at creation; must match on reopen
  metric: VectorMetric.cosine,
);

index.insert('doc-1', embedding);           // Float32List
index.insertAll({'doc-2': a, 'doc-3': b});

for (final match in index.search(VectorQuery(query, k: 5))) {
  print('${match.id}  ${match.score.toStringAsFixed(4)}');
}

index.save();   // syncs vectors + writes the HNSW snapshot
index.close();

On a UI isolate, use the async form — every call runs on a worker isolate, so a search cannot drop a frame:

final index = await AsyncPhoenixVectorDB.open('vectors.pvec', dimensions: 384);
final hits = await index.search(VectorQuery(query, k: 5));
await index.close();

Metric	distance	score (higher is better)
VectorMetric.cosine	1 - cos(θ), in [0, 2]	cosine similarity, [-1, 1]
VectorMetric.euclidean	true L2 distance	1 / (1 + d), in (0, 1]
VectorMetric.dotProduct	-(a·b)	the inner product

Notes:

• Vectors are the source of truth. The graph is snapshotted separately and rebuilt automatically if the snapshot is missing, stale or corrupt.
• Small indexes are exact. Below 512 live vectors the engine scans exhaustively, so a handful of documents gives exact answers, not approximate ones.
• Removal is a tombstone, so ids in the graph stay stable. Call compact() once stats().deletedRatio passes roughly 0.3.
• efSearch trades latency for recall. The default (64) suits k ≤ 10; raise it for higher recall on large, high-dimensional collections.
• SIMD is automatic. index.kernel reports avx2+fma, neon or portable. AVX2 is selected at runtime, never baked in, so a published binary still runs on pre-2013 x86 hardware.

Security model

• FFI guardrails — every entry point validates pointer non-nullness and length limits (key ≤ 1 MiB, value ≤ 10 MiB) before any dereference, and returns -2 for violations.
• Handle tagging — each handle carries a magic tag checked in constant time, so use-after-free and foreign pointers are rejected instead of dereferenced.
• Constant-time comparison — security::ct_eq / ct_eq_u64 fold differences with bitwise XOR masking, with no early exit for equal-length inputs.
• Panic containment — every FFI body runs inside catch_unwind; a panic becomes status -7 and never unwinds into Dart.
• Single free path — Rust-allocated memory is released only by phoenix_buffer_free / phoenix_string_free, wired into Dart's NativeFinalizer.

Status codes

Code	Meaning
0	Success
-1	Unclassified error
-2	Invalid argument (null pointer / length limit)
-3	Key not found
-4	Corruption (CRC mismatch)
-5	I/O error
-6	Write-write conflict — retry
-7	Panic caught at the boundary
-8	Unknown transaction
-9	Capacity exceeded

Testing

cd rust && cargo test          # 86 tests: unit + FFI safety + ACID integration
dart test                      # 33 tests across the sync and async APIs
dart analyze --fatal-infos     # clean

Fuzzing

Three libfuzzer harnesses live in rust/fuzz/:

cargo +nightly fuzz run fuzz_btree -- -max_total_time=60   # vs a BTreeMap oracle
cargo +nightly fuzz run fuzz_page  -- -max_total_time=60   # arbitrary page bytes
cargo +nightly fuzz run fuzz_ffi   -- -max_total_time=60   # hostile FFI inputs

Limits

Limit	Value
Page size	4096 bytes
Max key (FFI)	1 MiB
Max key (B+Tree structural)	1 KiB
Max value	10 MiB
Max inline value	1 KiB (larger spills to overflow pages)
Max vector dimensions	65 536
Max vector id	128 bytes
Max k per search	4096

License

BSD-3-Clause