ob_dump_reader 0.1.0-beta.3

Minimal ObjectBox LMDB reader toolkit for Dart and Flutter: reads data.mdb directly through a bundled LMDB (build hook).

ob_dump_reader

Minimal ObjectBox LMDB reader toolkit for Dart. Its core job: walk a data.mdb file and hand you each stored object's raw FlatBuffers table bytes, plus its entity id and object id — no per-field FlatBuffers or schema knowledge for that part, just a small purpose-built dart:ffi binding over vendored LMDB C source (see "Why this shape" below). It also covers the handful of things flatc --dart output can't: ToMany relations (a separate LMDB structure, not a table field) and Flex/ ExternalPropertyType fields (flatc only knows the base FlatBuffers type, not ObjectBox's semantic annotation on top) — see "Beyond flatc" below.

Decoding those raw bytes into typed objects is left to the official flatc --dart compiler — not this package, and not ob-dump's own C++ core either. This is a deliberate toolkit split, not a missing feature: see the parent project's docs/BACKLOG.md for the reasoning.

Dart and Flutter, one package

dart pub add ob_dump_reader      # or: flutter pub add ob_dump_reader

The native LMDB library is compiled for you by the package's build hook (hook/build.dart, hooks + native_toolchain_c) from the vendored source in src/lmdb/ — no network access, no prebuilt binary, no separate build step. It works the same under dart run, dart test, flutter run/build/test and in AOT release builds on Android, iOS, Linux, macOS and Windows (there is no web support: LMDB needs a file system). A C compiler is all a build machine needs (the Android NDK / Xcode / Visual Studio that Flutter already requires).

Sandboxed macOS apps: LMDB synchronises processes with named POSIX semaphores on macOS (sem_open), which the macOS App Sandbox forbids — opening a store fails with mdb_env_open ...: Operation not permitted. Apps that read a store from inside the sandbox (e.g. Mac App Store builds) have to run without it (or use an entitlement that allows it); unsandboxed apps, dart run and flutter test are unaffected. CI runs the macOS example unsandboxed for this reason.

If the library cannot be built or loaded, the build fails (the hook checks what it produced), and reading throws an LmdbException that names the problem.

Migrating from ob_dump_reader_flutter (discontinued): depend on ob_dump_reader instead and import package:ob_dump_reader/ob_dump_reader.dart. dart run ob_dump_reader:build is gone — the build hook replaces it. Requires Dart SDK 3.10+.

Testing

dart test runs the unit tests (and the build hook). CI additionally builds and runs example/ — a Flutter app that writes an ObjectBox-shaped store and reads it back — for Linux, macOS, Windows, Android (also on an emulator) and iOS, and as an offline Flatpak (flatpak.yml, started by hand); see scripts/ci/flutter-smoke.sh. To run the Flutter checks locally:

cd example && flutter test
OB_DUMP_EXAMPLE_SELFTEST=1 flutter run -d linux   # headless check, exits 0/1

Workflow

# 1. Generate a schema JSON (entityId -> table name/shape) and a .fbs from
#    your ObjectBox model, using the ob_dump CLI (see ../README.md):
ob_dump --schema objectbox-model.json -o schema.json
ob_dump --fbs    objectbox-model.json -o schema.fbs

# 2. Generate typed Dart classes with the *official* FlatBuffers compiler.
#    (This is `flatc`, not `flatcc` — flatcc is a separate, C-only
#    implementation with no Dart backend.) flatc always names its output
#    `<input>_generated.dart`; rename to the `.g.dart` suffix Dart tooling
#    conventionally uses for generated code (e.g. so a `*.g.dart` entry in
#    .gitignore or an analyzer exclusion rule already covers it).
flatc --dart -o lib/generated schema.fbs
mv lib/generated/schema_generated.dart lib/generated/schema.g.dart

import 'package:ob_dump_reader/ob_dump_reader.dart';
import 'generated/schema.g.dart'; // from flatc --dart, see step 2 above

const ammoEntityId = 1; // from schema.json

Future<void> main() async {
  // Walks data.mdb, calling this callback once per stored ObjectBox object
  // with its raw (still-undecoded) FlatBuffers bytes — see [ObRecord].
  await readObjectBoxRecords('/path/to/objectbox/dir', (record) {
    if (record.entityId == ammoEntityId) {
      final ammo = Ammo(record.data); // flatc-generated class decodes the bytes
      print('${ammo.name}: ${ammo.bcG1}');
      // ... insert into whatever your new database is.
    }
  });
}

Reads are in place, no copy

readObjectBoxRecords reads data.mdb/lock.mdb directly, with no temporary copy step — it opens the LMDB environment itself MDB_RDONLY (matching ob-dump's own C++ LmdbReader), so it never needs a write-capable handle. This is exactly what LMDB's MVCC design is for: any number of readers can safely run alongside one concurrent writer, in any process, with zero risk to the original data — even while a live ObjectBox process has the same store open.

Beyond flatc: ToMany relations and Flex/ExternalPropertyType fields

ob_dump --schema lists each property's externalType and each entity's relations (id, name, target entity) when present — use that to know which of your fields need one of these:

import 'package:ob_dump_reader/ob_dump_reader.dart';

// ToMany: not part of the FlatBuffers table at all, so flatc has no
// accessor for it — relationId/sourceObjectId come from `ob_dump --schema`
// and the record you're looking at (record.objectId).
final authorIds = await readToManyTargets(dbDir, relationId, record.objectId);

// Flex: flatc gives you the raw bytes (a List<int>/Uint8List field) —
// decode with decodeFlex.
final value = decodeFlex(ammo.someFlexField);

// ExternalPropertyType: flatc gives you the base type's plain value
// (a byte list for Uuid/Int128/Decimal128/Bson, a string for
// Json/JavaScript/JsonToNative) — decode with the matching helper.
final uuid = bytesToUuidString(ammo.someUuidField);
final blob = bytesToHex(ammo.someBsonField);
final parsed = tryParseJsonString(ammo.someJsonField);

Every one of these mirrors ob-dump's own C++ decode (src/fb_decode.cpp) exactly — same hex/UUID formatting, same JSON-parse-with-string-fallback for JavaScript, same forward-only relation direction (see docs/BACKLOG.md "ToMany relations" for why: the backward direction is an auto-maintained index for ObjectBox's own query engine, not needed for a one-directional dump).

Why this shape

The alternative — an FFI wrapper around ob-dump's full C++ core — would mean vendoring/building that much larger codebase from a pub.dev package for every platform. Not needed: LMDB itself is a single small C library (src/lmdb/, vendored from the same upstream commit ob-dump's own C++ core builds), so this package binds directly to it via a purpose-built dart:ffi layer (lib/src/lmdb_bindings.dart), and flatc --dart already gives an officially generated, correct FlatBuffers decoder. This package is only the small piece connecting the two — LMDB traversal and the ObjectBox key format (docs/BACKLOG.md in the parent project) — and stays that small on purpose.

An earlier version of this package depended on dart_lmdb2 instead of vendoring LMDB directly. Dropped because its native-library path resolution (Isolate.resolvePackageUriSync()) throws UnsupportedError outright in any compiled AOT release build — confirmed empirically against a real flutter build linux --release, run from an isolated copy of just the output bundle — and because it required a write-capable LMDB environment just to open a store, forcing a copy-to-temp step this package no longer needs (see "Reads are in place, no copy" above).

Verified end-to-end

Confirmed against a real ObjectBox database, independent of every other verification already done in this repo (the Dart PoC this was ported from, the C++ decoder, and a C++ flatc-generated consumer): generated schema.fbs from the real model, ran flatc --dart on it, read a real record with readObjectBoxRecords, decoded it with the generated Ammo class — every field matched exactly.

Integrity & Licensing

ob-dump was developed as an independent implementation for reading data stored in the ObjectBox format. It adheres to a "Clean Room Design" approach regarding binary software:

• Purpose-Limited: Built solely to support data recovery and migration to another database, for projects whose own license is incompatible with objectbox-c's (a closed-source binary — see "Why this exists" in the parent project's docs/BACKLOG.md). Not intended as, and not pursued as, a competing product to ObjectBox itself — no write support, no query engine, no ongoing-database use case, strictly a one-time read-only export path out of an existing store.
• No Reverse Engineering: We have performed no decompilation, disassembly, or any other analysis of the closed-source objectbox-c binary.
• Open Specification: Data parsing is based exclusively on public formats (LMDB and FlatBuffers) and the open-source code of the official schema generator (objectbox_generator, licensed under Apache 2.0).
• Model-Driven: The decoding process is driven by the user-provided objectbox-model.json file, which is an open, user-accessible schema definition.

This approach ensures full licensing integrity: ob-dump is an independent software project that contains no proprietary or misappropriated code, making it suitable for integration into projects with any licensing requirements.

License

MIT — see LICENSE.