ffastdb 0.3.5

A high-performance, pure-Dart embedded NoSQL database with B-Tree indexes, WAL crash recovery, and reactive queries.

Changelog

0.3.5

Fixed — performance

• BufferedStorageStrategy could be unexpectedly slow when many different offsets are written in a single flush() call. If the pending writes are widely scattered (e.g. each writes to a different index block or document offset), the mergeChunks logic could create many individual _writeChunk sub-ranges, resulting in more I/O operations than necessary.

0.3.4

Fixed — data loss / correctness

• FTS and Composite indexes could come back silently empty after closing and reopening the database. Three compounding bugs in the index-persistence path:
  • SortedIndex.deserialize() corrupted its read cursor on any string-valued entry (off += readInt32() discarded 4 bytes due to Dart's compound-assignment evaluation order — the RHS's side effect on off isn't visible to the addition), throwing a RangeError once entries existed.
  • That exception was swallowed by loadIndexes(), silently aborting the load of every index serialized after the failing one — including FTS and Composite.
  • The fallback path (reindex(field:), used for any index whose persisted blob failed to load) extracted the wrong document field for FTS (used its _secondaryIndexes map key, e.g. _fts_content, instead of the actual field content) and didn't handle Composite's multi-field extraction at all — so even the fallback silently produced empty indexes.
• loadIndexes() also looked up an index's "already registered" check by its plain fieldName, which for FtsIndex can collide with an unrelated Hash/Sorted index registered on the same field name — could incorrectly discard a valid loaded FTS index as a "type mismatch".
• compact() (single-file mode) could corrupt live document data: it didn't clear the stale pre-compact index-block pointer in the header, so saveIndexes() could reuse an offset from the old (larger) file layout that now overlapped freshly rewritten documents in the smaller, compacted file.

Fixed — performance

• insertAll() and compact()'s rebuild path indexed documents one at a time (SecondaryIndex.add()), which is an O(n) array shift per call for SortedIndex — making indexing a batch O(n²) in its own size. Both now batch field values per index and apply them with one addAll() call per index per chunk.
• WalStorageStrategy (the default native storage) applied every buffered write to the main file individually — one syscall per document. Writes are now coalesced into a handful of large sequential writes per transaction, with correct "last write wins" ordering preserved for overlapping ranges.
• For transactions larger than the in-RAM buffering threshold (huge insertAll calls), each streamed write hit the WAL file with its own syscall — up to ~1M for a 1M-document batch. These are now accumulated in a rolling buffer and flushed every few MB instead.
• BufferedStorageStrategy.commit()'s write-coalescing was O(n²) in the number of pending writes (re-copied the whole accumulated buffer on every additional write).
• reindex() / rebuildSecondaryIndexes() read each document with its own read() call — mostly Dart-side Future/lock overhead rather than actual I/O. Densely-packed batches now fetch their whole byte span in one read and slice records out of the shared buffer in memory.
• Net effect: 1M-document insertAll went from not completing in 5+ minutes to ~34s; reindex() on 100k documents went from ~6s to ~2.7s; a 100k-document insertAll with Hash + Sorted + FTS + Composite indexes together went from visibly quadratic per-batch growth to flat.

Notes

• The two fixes above to WalStorageStrategy/BufferedStorageStrategy write-coalescing were themselves iterated on during development after being found to zero out untouched bytes between non-adjacent writes, and to break "last write wins" for writes that overlap out of offset order — both are covered by the final implementation and its tests.
• Individual (non-batched) insert()/update() calls are unaffected by the coalescing changes: transactions with a single write (the common case) take a direct fast path with no extra allocation.

0.3.3

Fix - Delete - Clear all the docs in the database works well Fix - Rebuild Indexes - Rebuild all the indexes in the database works well

0.3.2

Bugfix release — crash safety and reactive consistency

Critical:

• FIX - _readAt silently returned null on CRC32 mismatch — corrupted documents were indistinguishable from missing ones. Now throws StateError with offset and checksum values so corruption is immediately visible instead of surfacing as phantom findById nulls.
• FIX - deleteWhere did not notify reactive watchers — db.watch() streams were never updated after a batch delete, leaving UIs showing stale data. _notifyWatchersBatch() is now called after the WAL commit on the success path.

High:

• FIX - OperationLog was written before the WAL commit in all 4 CRUD methods — a crash between opLog.log() and wal.commit() caused _replayOpLog() to re-apply operations that had never actually completed, potentially duplicating documents or corrupting _nextId. The log entry is now written only after a successful commit.
• FIX - _serialize used a lazy Map.cast<String,dynamic>() view — when a document's map had non-String keys (e.g. Map<dynamic,dynamic> from Firebase or untyped jsonDecode), the CastError appeared deep inside the serializer instead of at the insert() call site. Replaced with Map<String,dynamic>.from(doc) for eager validation.

Medium:

• FIX - _replayOpLog silently swallowed all replay errors — a bare catch (_) {} discarded failures with no logging. Now emits a diagnostic print in debug mode via assert() (zero overhead in release builds).
• FIX - _applySort used ids.length * 4 < idx.size — on Dart Web (int = JS 53-bit number) this multiplication could overflow with very large collections. Replaced with ids.length < idx.size ~/ 4 and added a idx.size > 0 guard.

0.3.1

Bugfix and Documentation release

Fixes:

• fix documentation.

0.3.0

Durability release. Breaking for custom StorageStrategy implementations only.

Critical — crash atomicity:

• compact() destroyed docs on crash/ENOSPC — truncated the file first, rewrote from RAM after. The truncate is now a WAL transaction entry; compact is all-or-nothing.
• Rollback after a failed commit-apply deleted a committed transaction — it checkpointed the WAL past the atomic point. Rollback now preserves the WAL and the strategy refuses further operations until reopened.
• Schema migrations applied twice after a crash — version header was saved last, so a kill re-ran everything. Migrations now run inside a WAL transaction.
• Uint8List in documents came back as List<dynamic> — JsonEncoder never calls toEncodable for List subclasses, so the blob path was dead code (as was the 0.2.8 \u0000 escaping). Now pre-encoded recursively; round-trips correctly.
• Concurrent find() threw FileSystemException on native storage — concurrent setPosition/read on one RandomAccessFile. Handle operations are now serialized in IoStorageStrategy.

High:

• Opening a locked DB hung forever (FileLock.blockingExclusive) — now fails fast; also retries ~2 s after an owner dies (Windows lock-release lag, errno 33).
• close() always releases the file lock, even when a flush fails; op log kept for replay on unclean close.
• StorageStrategy.innerStorage lets capability detection (WAL) see through any wrapper.

Breaking: custom StorageStrategy implementations must add innerStorage (one line, return null); opening a DB held by another process now throws instead of waiting forever.

Tests: physical durability suite — SIGKILL mid-write (insert/batch/update/compact/reindex/blob/encrypted), SIGKILL mid-migration, fault injection on the commit-apply path, cross-version migration test (old package code → new code), concurrent writes/transactions, multi-process lock, ENOSPC, LRU eviction, watcher stress, query fuzzing vs linear oracle, index edge cases, opt-in soak test (FFASTDB_SOAK=1), manual on-device checklist (test/manual/).

0.2.8

Bugfix & Performance Release — full audit (see FIX_PLAN.md)

Data correctness (critical):

• FIX - B-Tree rangeSearch off-by-one: the last document was lost when the internal separator equaled the upper bound (e.g. after insertAll of 229/458/... docs, getAll() returned N-1).
• FIX - BufferedStorageStrategy: coalescing discarded recent writes contained within another write (silent corruption).
• FIX - PageManager: a stale dirty page could overwrite newer data; batch rollbacks no longer bypass the WAL.
• FIX - EncryptedStorageStrategy(WalStorageStrategy): the WAL was not detected through the encrypted wrapper (no real atomicity and ~2.5× fsyncs).
• FIX - Per-instance QueryCache: no more result contamination between FastDB instances.
• FIX - Unindexed conditions: now perform a full scan with in-memory filtering (previously silently dropped, returning incorrect results).

Durability/consistency (high): read-your-writes in WAL transactions (and _findById consults _batchEntries), delete() flushes on web, header saved before flush, IndexedDB fail-closed with dirty chunk retries, cache keys free of type collisions, aliasing removed (immutable query results), int64 serialization in SortedIndex/BitmaskIndex, _writeValue fails fast on unsupported types, CompositeIndex with collision-free keys (requires automatic rebuild), count() consistent with pagination, upsert respects manual ids.

API/indexes (medium): watch() without duplicate emissions and with wildcard for condition-less queries, not() propagated to isNull/isNotNull/alwaysTrue, put() rejects ids < 1, unified numeric comparator (1 == 1.0) without crashes on mixed types, idempotent add() across all indexes, bulkLoad tolerates out-of-order/duplicate input, transaction() rollback reverts secondary indexes, page free-list (file no longer grows monotonically on merges/rebuilds).

Performance: eliminated the fsync storm (double fsync + checkpoint per commit), streaming WAL (large imports without OOM), autoCompactThreshold actually disabled by default (was ~22 ms/delete in benchmarks), amortized operation log checkpoint and cleanup on close, table-driven CRC32 (4-8×), bulk index deserialize (startup without O(n²)), dead Bloom filter removed, _applySort O(k log k), concurrent find(), page free-list.

Serialization/security: DateTime preserves isUtc in TypeAdapters, \u0000 sentinels escaped, aes256KeyFromPassword rejects empty passwords, typeId 256 reserved.

Minor breaking changes: findIds() results are immutable (copy with .toList() if you need to mutate them); put() throws ArgumentError on ids < 1; HashIndex.mightContainValue removed along with the Bloom filter; CompositeIndex blobs persisted with the old format are discarded and automatically rebuilt on open.

0.2.7

Usage Update

• CHANGED — Native Internal ID Migrated to ffdbID: The database now stores its internal numeric ID in the ffdbID key instead of id. This safely fully segregates FastDB IDs from user IDs (like Firebase id), eliminating the need for _originalId workarounds. Legacy documents with id are migrated automatically on read.

0.2.6

Reliability and Usage Updates

• NEW — Sequential operation log for write recovery: Added a .log sidecar file that records insert, put, update, and delete operations before applying them to the main database. Pending operations are replayed automatically on startup after an interrupted write.
• IMPROVED — Native Uint8List storage: Binary payloads are now stored as raw bytes in the main database format instead of being encoded as Base64 strings, reducing storage overhead and improving large-binary performance.
• CHANGED — Single-owner native access model: Removed the transparent multi-isolate socket proxy path. On native platforms, a database file is now documented and treated as having one active owner at a time.
• FIX — Safer close/write synchronization: Close now waits behind the write lock so shutdown cannot race with pending exclusive writes.
• FIX — Safer storage opening semantics: Native file opening now avoids truncation while still resetting the cursor for random-access writes.

0.2.5

Bug Fixes (B-Tree Corruption & Recovery)

• FIX — Added cycle detection to B-Tree traversal: B-Tree traversal functions (searchSync, search, _countLeafEntries, _extractLeafEntries) now detect circular page references (cycles). If a cycle is detected, the function immediately returns null or throws a StateError with a helpful message, preventing infinite loops and StackOverflow errors caused by corrupt index pages.
• FIX — Improved WAL recovery error messages: Enhanced _recover() to provide more context when encountering corrupted or truncated WAL entries. Errors now include the offset, size, and type of the problematic entry, aiding in diagnosis.
• MODERATE — Updated B-Tree rebuilding logic: Refined _rebuild() to use iterative traversals instead of recursive ones for counting and extracting leaf entries. This prevents StackOverflow errors on very deep index trees.

0.2.4

Bug Fixes (WAL Corruption & Multi-Isolate Stability)

• CRITICAL — Fixed WAL recovery replaying uncommitted transactions: The _recover() method used a single hasCommit flag across the entire WAL file. If TX1 was committed and TX2 was interrupted mid-flight (e.g. app killed), recovery would set hasCommit = true and replay all entries — including the uncommitted ones from TX2 — silently corrupting the B-Tree. Fixed by tracking pendingEntries per transaction: a COMMIT marker moves pending → committed; entries still pending at EOF are discarded.
• CRITICAL — WAL now checkpoints after every commit: Previously the WAL only truncated when it exceeded 1 MB, leaving dozens of committed transactions accumulated in a single file. Combined with the multi-tx recovery bug above, this was the primary source of database corruption on mobile. The WAL is now truncated after every commit() call, ensuring it never holds more than one transaction at a time and recovery is instantaneous.
• FIX — Stale .fdb.port file causes SocketException on reopen: If the Owner isolate crashed or closed the database without calling coordinator.stop(), the .fdb.port sidecar file was left on disk. The next openDatabase() call would find the port, try to connect, and throw a SocketException. Fixed by adding IsolateCoordinator.isPortAlive() — a 200 ms probe that verifies the socket is accepting connections before entering proxy mode. Stale port files are deleted automatically.
• FIX — .fdb.port not cleaned up in tests: persistence_test.dart tearDown/setUp did not include .fdb.port in the file cleanup list, causing cross-test contamination when running the full suite. Added to the cleanup set.

---

0.1.2

Bug Fixes (WAL & B-Tree Stability on Mobile)

• CRITICAL — Fixed infinite loop in WAL binary recovery on Android/iOS: When a COMMIT marker's trailing 4-byte CRC was truncated (i.e. the file ended between the 25-byte header and its checksum), _recover() advanced offset only inside the if (offset + 4 <= raw.length) guard, then continue-d back to the while — re-matching the same magic bytes forever. Fixed by moving offset += 4 outside the if so the parser always advances past the COMMIT entry, even when truncated.
• CRITICAL — Fixed Stack Overflow in BTree._insertNonFull on corrupt databases: The recursive descent could cause a Dart VM stack overflow (~55 000 frames) when B-Tree page pointers formed a cycle due to a previously corrupt WAL recovery. Converted _insertNonFull to an iterative loop with a visited page-index set that detects cycles immediately and throws a clear StateError instead of crashing the VM.
• FIX — _WalEntry truncation check now includes CRC bytes: The _kEntryWrite truncation guard previously checked offset + length > raw.length, ignoring the 4 trailing CRC bytes. A write entry whose data fit exactly but left no room for the CRC would throw a RangeError inside the catch (_) silently. Now checks offset + length + 4 > raw.length.
• FIX — Unknown WAL entry types now break the parse loop: Previously an unrecognized type byte would leave offset unchanged and spin the while loop forever. Added an explicit break after the _kEntryWrite block for unknown types.

---

0.1.1

Bug Fixes

• FIX — Fixed cumulative sum stream: Resolved an issue where the reactive stream returned by aggregation watchers was accumulating values across emissions instead of replacing them, causing incorrect cumulative totals on repeated events.

---

0.1.0

Stable Public API Release

• BREAKING — Public index types: Removed @internal from SecondaryIndex, HashIndex, SortedIndex, FtsIndex, BitmaskIndex, and CompositeIndex. These are now part of the stable public API.
• BREAKING — IndexManager is now public: The db.indexes getter previously returned a private _IndexManager type. It now returns the public IndexManager class, enabling proper static typing in user code.
• FIX — SortedIndex.startsWith is now O(log n): Replaced the O(n) _reverse scan with a true binary prefix search using _lowerBound(prefix) / _lowerBound(nextPrefix). Prefix queries on addSortedIndex fields are now as fast as range queries.
• FIX — HashIndex.startsWith was broken (returned []): Added startsWith and contains support to HashIndex.search(). O(n) scan as expected for a hash structure — use addSortedIndex for prefix-heavy workloads.
• CLEANUP — 0 dart analyze warnings: Removed all unused imports, dead code (_readAtSync), unnecessary null assertions, and redundant type checks.

---

0.0.27

Bug Fixes (B-Tree Stability & Indexing)

• CRITICAL — Data corruption in single-file mode fixed: Resolved a critical issue where document data could overwrite B-Tree index pages. Implemented _syncDataOffset() to ensure the document write pointer is always synchronized with the underlying storage size after every structural index operation. This prevents structural corruption under high load.

• MODERATE — Fixed Indexing Type Strictness: Corrected a bug where documents were silently skipped during indexing because of strict Map<String, dynamic> runtime type checks. The library now correctly supports all Map variants (including Map<String, Object> literals used in common application code).

• MODERATE — Fixed Composite Index Population: Fixed a bug where CompositeIndex fields were not being correctly extracted and indexed during standard insert() and put() operations.

• CRITICAL — Fixed Index Persistence & Hot Restart failures: Fixed an issue where FtsIndex and CompositeIndex instances were left empty in memory if they were registered manually after opening the database, causing searches to fail after a Flutter Hot Restart. _StorageManager.loadIndexes was updated to properly restore complex indexes and map them correctly to the query engine.

• MODERATE — Missing indexes automatically rebuilt: FastDB.open() now actively checks if any registered secondary indexes are missing from the loaded storage payload. If missing indexes are detected (e.g. after a schema update), FastDB automatically triggers a background rebuild.

• MODERATE — Fixed FtsIndex search signature: Updated FtsIndex to correctly implement the SecondaryIndex.search(String operator, dynamic value) interface, preventing runtime Too few positional arguments errors.

New Features & Improvements

• NEW — Declarative Index Registration: Extended openDatabase and FfastDb.init to directly accept indexes, sortedIndexes, ftsIndexes, and compositeIndexes. This declarative approach guarantees that indexes are fully initialized before disk loading, resolving schema state issues during app restarts.
• IMPROVED — Case-Insensitive Search: The contains() and startsWith() operators are now case-insensitive by default. This provides a much more intuitive user experience for string-based filtering.
• NEW — Index Diagnostics API: Exposed db.indexes.all to allow developers to inspect active indexes, check their sizes, and retrieve internal statistics (useful for debugging and performance tuning).
• NEW — Public Index Types: Exported HashIndex, SortedIndex, FtsIndex, BitmaskIndex, and CompositeIndex as public types.

---

0.0.26

Bug Fixes (Index Rebuild Corruption on Startup)

• CRITICAL — reindex() and rebuildSecondaryIndexes() caused duplicate index entries: When a database from pre-0.0.24 versions opened in 0.0.23+, if the header was not marked "clean", open() automatically called rebuildSecondaryIndexes(). However, this method called rangeSearch() without deduplication. If rangeSearch() returned IDs multiple times due to B-Tree structural inconsistencies from old versions, each ID would be indexed multiple times. This corrupted secondary indexes, causing subsequent getAll() calls to return duplicate documents. Fixed by deduplicating rangeSearch() results using LinkedHashSet in both reindex([field]) and rebuildSecondaryIndexes().

• Test Results: All 94 tests pass. Databases created in 0.0.22 or earlier that were opened in 0.0.23-0.0.25 should now be cleaned up by running reindex() or closing/reopening in 0.0.26.

---

0.0.25

Bug Fixes (Out-of-Memory on Large Batch Inserts)

• CRITICAL — bulkLoad with large existing trees caused OOM: When inserting 5000+ documents into a database that already had data, the code extracted all existing entries into memory at once via _extractLeafEntries(), then merged and rebuilt. With 10,000+ total entries in RAM simultaneously, Dart's memory allocator was overwhelmed. Fixed by implementing a threshold strategy: trees with <500 entries use the extract-merge-rebuild approach (memory safe), while trees ≥500 entries fall back to individual inserts (no OOM, and now safe because the median-key bug from 0.0.24 is fixed). This trades some speed for memory safety on large existing datasets.

• Test Results: All 94 tests pass including the critical "large batch insert with write-behind does not OOM" test. Batch operations of 5000+ documents now work reliably.

---

0.0.24

Bug Fixes (B-Tree Duplicate IDs)

• CRITICAL — insertAll / commitBatch after existing data caused duplicate IDs: bulkLoad was replacing the B-Tree root with a freshly-built tree containing only the new entries, orphaning all previously-written pages on disk. On subsequent getAll() / rangeSearch() the old pages could still be reached via stale page references, returning IDs 3–5× duplicated. Fixed by extracting all existing leaf entries, O(N+M) merging them with the new batch, and rebuilding a single correct tree from the full combined set.

• CRITICAL — Median key data loss during leaf splits in individual inserts: When bulkLoad fell back to individual insert calls on a non-empty tree, B-Tree leaf splits moved the median key up to the internal node and discarded its document offset. Every 128th document was silently lost. Resolved by the merge-and-rebuild strategy above which eliminates leaf splits entirely for batch operations.

• MODERATE — rangeSearch could return duplicate IDs on structurally inconsistent trees: Added a seenIds Set<int> guard inside _rangeNode so each ID is emitted at most once regardless of tree state.

• MODERATE — getAllImpl did not deduplicate IDs from range scan: getAllImpl now passes IDs through a LinkedHashSet before fetching documents, providing a second safety net against duplicates reaching the caller.

• MINOR — WAL recovery was not idempotent: _recover() now compares the WAL entry bytes against what is already on disk and skips writes where the data already matches, preventing double-application of committed transactions after a crash-and-reopen cycle.

• Test Results: All 94 tests pass; 4 new regression tests added covering all duplicate-ID scenarios.

---

0.0.23

Bug Fixes (Query Cache & Reindex)

• CRITICAL — Query Cache Key Bug: Fixed query cache key generation that only included condition field types, not actual values. This caused different queries on the same field (e.g., equals('London') and equals('NoCity')) to share the same cache entry, returning stale results. Affected operations:

  • sumWhere(), avgWhere(), maxWhere(): returned wrong aggregated values
  • updateWhere(), deleteWhere(): returned wrong update/delete counts
  • findStream(): yielded stale cached documents
  • Fix: Cache key now includes condition values for all condition types (equals, range, contains, startsWith, isIn, isNull, fts)

• MODERATE — Reindex Cache Invalidation: Fixed reindex() not clearing query cache after rebuilding indexes, causing queries to return stale cached results. Added QueryBuilder.clearCache() call after index rebuild.

• MODERATE — Test Cross-Contamination: Fixed global static query cache causing test cross-contamination (one test's cached results affecting another test). Added QueryBuilder.clearCache() call at start of FastDB.open().

• Test Results: All 90 tests now pass (previously 11 were failing due to cache issues)

0.0.22

Performance Optimizations (Batch Reads & Cache)

• Default LRU cache size increased: cacheCapacity default raised from 256 (1 MB) to 2048 (8 MB) in FfastDb.init(). This dramatically improves performance on datasets with working sets larger than 1 MB, reducing disk I/O from repeated page cache evictions.

• New method: findByIdBatch(List<int> ids, {int concurrency = 50}): Parallelizes document reads up to a controlled concurrency limit. For queries returning many IDs, this is 10–100× faster than sequential findById calls. Automatically processes results in batches to prevent memory spikes on large result sets. Example:

  final ids = await db.query().where('status').equals('active').findIds();
  final docs = await db.findByIdBatch(ids);  // Parallel reads, 10–100x faster
  

• New method: stream<dynamic> stream(): Returns a lazy stream that yields documents one by one without loading the entire result set into memory. Ideal for exporting large datasets, pagination, or processing documents as they arrive. Example:

  await for (final doc in db.stream()) {
    print(doc);  // Process one doc at a time
  }
  

• getAll() optimized: Now uses findByIdBatch() internally instead of sequential reads, providing automatic 10–100× speedup on large databases with no code changes.

0.0.21

Bug Fixes (Index Corruption & Startup Reliability)

• CRITICAL — Bug #1: Fixed addSortedIndex and addBitmaskIndex overwriting already-loaded indexes with empty instances. Both methods now use putIfAbsent so that a persisted index loaded from disk by _loadIndexes() is preserved. The direct assignment caused every startup after the first to return empty query results and permanently corrupt the on-disk index.
• GRAVE — Bug #2: Fixed race condition in openDatabase where FfastDb.instance became accessible before secondary indexes were registered. Indexes are now passed into FfastDb.init() and registered before open() is called, so _loadIndexes() can match serialized blobs to the correct type and the singleton is never exposed in a half-initialized state.
• GRAVE — Bug #3: Fixed _loadIndexes() silently ignoring an index-type change between startups (e.g. HashIndex → SortedIndex). If the on-disk type differs from the pre-registered type the blob is now discarded; the index is rebuilt with the correct type via _rebuildSecondaryIndexes(), preventing silent O(n log n) degradation of range queries.
• MODERATE — Bug #4: Fixed a single corrupt document aborting the entire startup. _rebuildSecondaryIndexes() now wraps each findById call in a try/catch; corrupt documents are skipped and will be cleaned up on the next compact().
• GRAVE — Bug #5a: Fixed O(n) memory allocation on every startsWith query with HashIndex. _StartsWithCondition now calls the new HashIndex.filterKeys() method, which iterates buckets without sorting or copying the full index, reducing per-query allocation from O(n) to O(k) where k is the number of matching documents.
• GRAVE — Bug #5b: Changed the default autoCompactThreshold from 0 (never compact) to double.minPositive (compact as soon as any dead document exists). This prevents the WAL file from growing unboundedly and eliminates the associated RAM spike on startup caused by replaying months of historical writes.

0.0.20

Bug Fixes (Web Memory Crash)

• CRITICAL (Web/IndexedDB): Rewrote IndexedDbStorageStrategy with chunked incremental flush. Data is now stored as 64 KB chunks in IndexedDB; flush() only writes the chunks modified since the last flush. Previously every insert()/update()/delete() copied the entire database buffer (e.g. 50 MB) from Dart to JavaScript, causing peak memory of ~3× the DB size per flush and OOM crashes on large databases. Peak memory per flush is now O(64 KB) instead of O(DB size).
• Web/IndexedDB: Backward-compatible migration — databases stored in the old single-key format (<name>_buffer) are loaded transparently and migrated to the chunked format on the next flush().
• Web/IndexedDB: truncate() now cleans up orphan chunk keys in IndexedDB on the next flush, preventing stale data after compact().
• MemoryStorageStrategy: truncate() now reclaims the backing Uint8List when the used size shrinks by more than 512 KB (matching the existing fix in WebStorageStrategy and IndexedDbStorageStrategy). Previously a database that grew to 64 MB and was compacted to 5 MB still held the 64 MB buffer in RAM.
• Core: Eliminated redundant storage.flush() calls in _insertImpl() and _updateImpl() when dataStorage is null (single-file mode). On web this avoided creating an unnecessary second IndexedDB transaction per operation.
• Web: Reduced default cacheCapacity in openDatabase() from 256 to 64 pages (1 MB → 256 KB). On web the entire database is already in RAM, so a large LRU page cache is redundant overhead.

0.0.19

Bug Fixes (Memory)

• OOM fix - insertAll: Documents are now serialized and written to storage one at a time instead of accumulating all serialized Uint8List objects in RAM before writing. For large batches (e.g. 100K × 1KB docs) this eliminates ~100MB of peak heap usage.
• B-Tree node cache: Reduced _nodeCacheCapacity from 4096 to 512 deserialized nodes, cutting the in-memory node object overhead from ~16MB to ~2MB. Hot nodes remain fast via the underlying LRU page cache.
• BitmaskIndex: Default maxDocId reduced from 1,048,576 (128KB per bitset) to 65,536 (8KB per bitset). The index still grows automatically via _grow() when document IDs exceed the initial capacity, so behaviour is unchanged for large datasets.
• _BatchState enum: Removed the now-unused state-machine enum that was part of the old two-pass insertAll implementation.

0.0.18

Bug Fixes & Code Quality

• Web/LocalStorage: Fixed Uint8List not found compile error by adding missing dart:typed_data import (caused incomplete package analysis and 0/50 static analysis score on pub.dev).
• Static analysis: Resolved all lib/ warnings and infos: removed unused dart:js_interop import, fixed return null in void method, replaced LinkedHashMap() with collection literal, added library; directive to open_database.dart, made FieldCondition public (was _FieldCondition), fixed doc-comment angle brackets, and improved prefer_is_empty usage.

0.0.17

Bug Fixes (Web Memory)

• Web/IndexedDB: flush() now skips the IndexedDB put when no data has changed since the last flush (_dirty flag). Eliminates redundant writes that fired 2-3× per insert()/update()/delete() when needsExplicitFlush is true.
• Web/IndexedDB: flush() no longer creates an intermediate Dart sublist() copy of the buffer. A zero-copy typed-data view (buffer.asUint8List) is used instead, reducing the peak RAM during flush from 3× to 2× the database size.
• Web/IndexedDB & WebStorageStrategy: truncate() now releases the backing Uint8List when the used size shrinks by more than 512 KB (e.g. after compact()). Previously the oversized buffer was retained in RAM until the page reloaded.
• Web/LocalStorage: Added _dirty flag (same flush-deduplication as IndexedDB) and overrides for both write() and writeSync().
• Web/LocalStorage: flush() now catches QuotaExceededError (the ~5 MB localStorage limit) and throws a descriptive StateError that suggests switching to useIndexedDb: true, instead of silently losing data.

0.0.16

Bug Fixes

• WASM: The runtime failure in wasm is fixed

0.0.15

Bug Fixes

• Web: Fixed Function converted via 'toJS' contains invalid types compiler error in IndexedDbStorageStrategy by removing an invalid async keyword from a JS interop closure.

0.0.14

Critical Bug Fixes

• CRITICAL: Fixed openDatabase() unconditionally calling FfastDb.disposeInstance() at the start of every call. This caused "Bad state: Cannot perform operations on a closed database" errors when multiple code paths (e.g., a BLoC and a repository) called ffastdb.init() concurrently during app startup. The function now reuses the live instance if one is already open.
• CRITICAL (Web): Fixed IndexedDbStorageStrategy using the hardcoded key 'db_buffer' for all database instances. Opening two databases (e.g., 'users' and 'products') caused their data to collide in the same IndexedDB slot. Each database name now gets its own isolated key ('${name}_buffer').

New Features

• QueryBuilder.find() — executes a query and returns the full document list directly. No more manual findById loop. Use via db.query().where('field').equals('value').find().
• QueryBuilder.findFirst() — returns the first matching document or null, resolving only one document ID for efficiency.
• QueryBuilder.count() — returns the count of matching documents with an O(1) hot path for simple equality queries on indexed fields (reads the index bucket size directly).
• FastDB.isOpen getter — exposes whether the database instance is currently usable.

Improvements

• Improved error message for closed-database operations: now explains the three most common causes and how to recover, instead of the previous generic "Cannot perform operations...".
• EncryptedStorageStrategy doc comment updated with a clear security warning: it uses a Vigenère-style XOR cipher (obfuscation, not cryptographic-grade encryption). Guidance for using AES-256-GCM via encrypt / pointycastle is included.
• Barrel export (package:ffastdb/ffastdb.dart) now includes EncryptedStorageStrategy and the platform-appropriate storage strategy (IoStorageStrategy on native, IndexedDbStorageStrategy on web) — no more imports of internal src/ paths.

0.0.13

• solve wasm issues

0.0.12

• Fig minors issues

0.0.11 (unreleased)

Critical Bug fixes

• CRITICAL: Fixed database corruption on Android/iOS caused by using FileMode.append. On mobile platforms, FileMode.append ignores setPosition() calls and forces all writes to the end of the file, corrupting B-tree nodes that need to be updated at specific offsets. Now uses FileMode.write which correctly respects random-access writes.

API Changes

• Restored public FastDB() constructor for non-singleton use cases (benchmarks, multiple instances). For most applications, continue using FfastDb.init() with the singleton pattern.

0.0.10

• fix package compatibility

0.0.9

• add meta
• fix library versions

0.0.8

• Fix Garbage collector issue
• Fix Firebase problems

0.0.7

• fix unsupported type fallbacks

0.0.6

• add serializable

0.0.5

• fix firebase bugs

0.0.4

• Fix persistence bug

0.0.3

• Fixed web bug

0.0.2

Bug fixes

• Fixed corrupted documents being read silently from disk without checksum validation.
• Fixed database getting stuck when a batch insert fails halfway through.
• Fixed compact() not actually freeing disk space in single-file mode.
• Fixed index values greater than 2 billion being corrupted after a restart.
• Fixed memory growing unboundedly after many deletes or updates.
• Fixed nested transaction() calls silently corrupting rollback state — now throws a clear error.
• Fixed calling beginTransaction() twice discarding pending writes silently — now throws a clear error.
• Fixed watch() streams accumulating in memory after all listeners are gone.
• Fixed registering two adapters with the same typeId silently overwriting the first one — now throws an error.

New features

• DateTime is now supported natively — no more manual conversion needed.

Improvements

• Queries are noticeably faster: the query planner no longer runs each condition twice to estimate cost.
• startsWith() is now much faster on sorted indexes (uses a range scan instead of scanning everything).

0.0.1

• Pure Dart DB
• Type Adapters
• B-Tree primary index
• Multiplatform storage
• Index persistence
• Hash Index
• Sorted Index
• Bitmask Index
• CRUD Operations
• WAL crash recovery
• Transactions
• Schema migrations
• Fluent query builder
• Aggregations
• Reactive watchers
• Auto-compact
• First version