cocoon 0.1.2
cocoon: ^0.1.2 copied to clipboard
A typed document database for Flutter & Dart: Firestore-style collections, protobuf-packed docs and real indexes on disk. SQLite under the hood, zero SQL.
0.1.2 #
- Docs-only release: a new README (a short pitch and on-ramp that makes clear Cocoon is a NoSQL document store, with the deep topics on docs.page) and a new pub.dev description and topics.
- No API change and no storage change: 0.1.1 databases open as they are.
0.1.1 #
-
New: prefs, single models and preload.
db.prefsis a key-value store for strings, ints, bools, doubles and bytes, anddb.single(adapter, key:)keeps one generated model (the signed-in user, say) under one key. Both live in the same database file as your collections, with no collection to register and no index. Reads returnnullwhen the key is absent, and reading a key as the wrong kind (or a single through the wrong adapter) throws aPrefsTypeException.watchemits the current value first, then every committed change.await db.prefs.setString('theme', 'dark'); final me = db.single(userAdapter, key: 'currentUser'); await me.set(user); // Preload once (or open with `preloadPrefs: true`), then read // synchronously, for example in a router redirect. await db.prefs.preload(); final theme = db.prefs.getStringSync('theme'); // String? final signedIn = me.getSync() != null;tx.prefs,tx.singleandbatch.prefscommit atomically with the collection writes in the same transaction or batch. Prefs are device-local: they are in every backup (a restore brings the saved prefs back) and never in the change log. The preload map and the watches always live on the caller isolate; storage runs inline in caller and auto mode (never on the auto helper) and on the engine isolate in background mode. They are not encrypted, so keep tokens and passwords in the platform keystore. Existing databases pick up the one new internal table (_cocoon_prefs) on open; there is no storage-format change. Guide: Prefs & Single Models. -
Fixed: opening a database in auto mode no longer waits for the helper isolate. A slow helper could stall
Cocoon.openfor up to 60 s (its ready-handshake timeout) before degrading to inline — freezing an app that opens its database at startup. Open now returns as soon as the engine on the caller is committed, and every call runs inline until the helper attaches in the background (heavy work routes to it from then on; a helper that never starts is given up on after 30 s, logged once, without ever blocking the caller).close()while a helper is still spawning completes promptly and shuts the late helper down cleanly, anddb.isolateStatsnow reports the helper's state (starting,ready,failed,closed) plus the spawn duration in ms once known. -
Docs: the pub.dev page shows Kepo. The Cocoon mascot is now the package screenshot and sits at the top of the README.
0.1.0 #
First release on pub.dev. The newest changes come first; the full 0.1.0 feature set follows under "Everything in 0.1.0".
-
Closing an
autodatabase no longer leaves a-walbehind. The helper isolate now closes its read-only connection before it is killed, so SQLite's last-close cleanup runs and removes the-waland-shmsidecars. Before, the leftover-walcould replay its un-checkpointed pages over a database file restored into the same path. -
New: run the engine off the UI isolate — one option.
CocoonOptions(isolate: CocoonIsolate.background)spawns one worker isolate per database and runs the whole engine there (decoding pages, aggregates, group-bys, bulk-writer encodes, migrations, compaction bookkeeping), so that work no longer runs on the isolate that calledCocoon.open.CocoonIsolate.autokeeps small work and every transaction inline on the caller with zero round trips and moves the heavy reads and bulk encodes to a helper isolate automatically. Everything else — refs, queries, watches, transactions, batch, bulkWriter, backups — behaves the same in every mode, and the default (caller) is byte-for-byte today's behaviour. Measured: the worker round trip is near-free (single-get p50 0.32 ms → 0.34 ms) and reads decode off the caller (a 10k-page read 103 ms → 75 ms wall inauto), and the worst single caller stall in a mixed read/write workload drops from 102 ms to 27 ms inbackground, but a 100k bulk import's caller stall does not drop yet — the document copies that cross the boundary cost about what the encode they replace saved (~0.9 s total stall incalleragainst ~1.1–1.3 s); returning encoded bytes asTransferableTypedData(decoded lazily on access) is the identified next step. Full table and the recommendation: Isolates. -
Paging a collection group no longer skips documents that share an id. A document id is unique only within one parent, so a collection-group query's keyset tiebreak is now the full identity — the unbound path params (in pattern order), then the id, following the last sort key's direction. Before, a group query whose page ended on an id shared under another parent silently skipped every later row with that id: paging a 2,500-row collection group returned 1,000 rows, no error. Scoped queries (every param bound by the parent) keep their cursor bytes and fingerprints — only collection-group cursors from an older build changed, and those are rejected with a
CursorMismatchExceptioninstead of paging wrong (they were wrong). -
Paging ordered by a path param no longer throws on page 2. Cursor keys are built one per sort key, reading each key's own result column (field column, param column or id), instead of from a de-duplicated column list that dropped param columns — an order on a path param (as the DevTools query runner allows) no longer fails every page after the first with
InvalidCursorException: Cursor key count does not match the query sort order. -
Composite indexes stack as
@Indexlines above the class. Instead of growing the@Collection(indexes: [...])list, put one@Index([...])line per index directly above the model class, in index order (unique: trueincluded); the generator reads every@Indexline, in source order.indexes:keeps working — use it for a sharedconstlist — and the same index declared in any mix ofindexes:, a stacked@Indexand@Indexed()on the field is one index: the canonical form is the identity, so moving an index between the forms changes nothing incocoon.lock(byte-identical), with no schema version bump and no rebuild.@Indexon anything that is not a@Collectionclass is a build error naming the class. -
index addwrites the stacked form. For a composite (or any path-param index),dart run cocoon_generator:index add <Model> <fields>inserts one@Index([...])line above the class declaration, after the existing@Collection(...)and@Indexlines, instead of editing theindexes:list — so a model whose list is built from a sharedconstvariable now works like any other. One-field model-field indexes still become@Indexed()on the field. The new--uniqueflag makes the added index UNIQUE. Re-running an already-declared index is a no-op that names where it is declared, through any of the three forms. -
Turn a saved log into indexes:
index add --missing. Run the app in debug withscanAndWarn, save the output (flutter run,flutter logs,adb logcat, a test or Patrol run), thendart run cocoon_generator:index add --missing run.log(or-for stdin): it applies everyFix: dart run cocoon_generator:index add ...line the log names — any line prefix, deduped — one result line per index (Added .../Nothing to do: .../refused: ...) plus a summary, and exits 0 when every index ended up declared, 1 when any was refused.--dry-run(both forms) prints what would change and writes nothing. A development tool likemigrate, never a CI step;butterfly cocoon index addforwards both flags verbatim. -
Database calls inside a transaction fail fast with a readable error. Inside
runTransactionevery read and write must go through thetxobject. A call on the database itself —doc.set/get/delete,collection.add/get/count/getAll, a query'sget/count,db.getAll,batch().commit(), creating or using aBulkWriter, a nestedrunTransaction, orbackupTo— now throwsTransactionMisuseErrorimmediately, naming the operation and ending with a one-lineFix:hint, instead of waiting forever for the transaction's own write lock (writes) or silently reading committed data without the transaction's writes (reads). The transaction rolls back like any other error. Reads and writes on a different openCocoon(another database file) still work inside a body, and starting awatch()there is allowed (its first value reflects committed data; the transaction's writes arrive with its commit event). -
Single-field indexes go on the field:
@Indexed(). The annotation declares exactlyIndex(['field'])(orIndex(['-field'])withdescending: true,unique: truefor aUNIQUEindex) as if it were written in@Collection(indexes: [...]). The generator merges field-level and class-level indexes into one list — an index declared both ways is one index — and the physical index name comes from the canonical form, so moving a one-field index from the class to the field changes nothing: no schema version bump, no rebuild, no lock change. Composites (two or more parts) and path params stay on the class. -
A missing index now tells you the exact fix. The
MissingIndexExceptionmessage ends with aFix:line:dart run cocoon_generator:index add <Model> <fields>plus the declaration it would write. Run the command and it edits the model for you —@Indexed()on the field for a single model field, an appendedIndex([...])for composites and path params — thendart formats the file and prints the two follow-up commands (migrate, thenbuild_runner build -d; adding an index to an existing collection changes the lock version, somigratecomes first). It validates names with a "did you mean", is idempotent (an index already declared in exactly that order is a no-op naming where), and refuses annotation shapes it does not understand (a const variable, a spread, a non-literal) with the exact text to paste. A development tool likemigrate, never a CI step; also forwarded bybutterfly cocoon index addand exposed as an MCP tool. -
Debug-only full-scan fallback for missing indexes.
CocoonOptions(onMissingIndex: MissingIndexMode.scanAndWarn)runs queries no declared index serves as a full scan while you explore a schema: the missing index is printed once per distinct missing index per process (message with itsFix:line, on thedart:developerlog and stderr) and listed indb.missingIndexes(model, index parts, query shape, fix command, first seen). It is honoured only while asserts are enabled — in a release build it throws like the default (MissingIndexMode.throwError), so a forgotten.allowFullScan()can never silently degrade a shipped query. It never creates an index at runtime. -
The schema lock is a YAML
cocoon.locknow — Dart's own lock file (pubspec.lock) is YAML too, so reviews read it without braces-and-quotes noise next to the migration files.SchemaLock.parsereads both shapes and the lock's fingerprint is unchanged, so existing databases see the same hash and nothing re-migrates. Onedart run cocoon_generator:migrateconverts a project: it reads the legacycocoon.lock.json, writescocoon.lock, deletes the old file and reminds you to commit both changes; both files present is an error, never a guess. -
migrateis documented as a development tool, not a CI step. Run it on your machine after changing a model, answer its questions, then commitcocoon.lockandlib/cocoon_migrations/together. CI only builds: the build fails when the models and the committed lock disagree, so a model change that skippedmigratecannot pass CI — the fix is always "run migrate locally and commit", never "run migrate in CI", and--checkis read-only but redundant there. -
migratenow asks before a new required field silently reads its zero value. Adding a non-nullable field with no constructor default to an existing collection used to book every old document at the proto3 zero ('',0,false, the epoch) — a quiet data bug for money and dates. The migrate command now shows the exact value old documents would read and asks: write a backfill stub (the default — the engine then sets the field on every existing document at the next open, inside the usual crash-safe migration, the build failing while the stub isUnimplementedError), keep the zero knowingly (recorded in the lock, never asked again), or abort. Scripts use--backfill Model.field/--keep-zero Model.field;--yespicks the backfill, never a silent keep-zero, and--checkreports pending decisions. -
A storage-format upgrade no longer leaves wasted file size behind. After an open that upgraded an older file's layout, Cocoon now compacts the file once when more than a fifth of it sits on the freelist (at 1M: the upgrade transiently grew the file to ~519 MiB; the automatic
VACUUMreturns it to 304.9 MiB in ~1 s) — no phone carries the upgrade's dropped pages until its next backup. Disable withCocoonOptions.compactAfterUpgrade: false; a compaction that cannot run (a full disk) is reported through the newCocoonOptions.onCompactionFailedand never fails the open. Measured inbenchmarks/cocoon_rivals/STORAGE.md. -
db.getAll(refs): many documents by id in one trip. Loading a set of known ids — a split's participants, linked accounts, a sync batch — no longer costs one round trip per document.getAllanswers in refs order,nullwhere a document is missing, duplicates allowed, and mixes parents and subcollections of one model freely. Documents the hot cache holds come back without SQL; the misses are grouped per table and fetched with one statement per 500 scope keys, all inside one read transaction, so the whole call reads one consistent snapshot and populates the cache exactly likeget(). Measured on the 1M rivals dataset: 1,000 ids in 8.7 ms cold (was 75 ms one-by-one; the rivalsgetsworkload's shape) and 0.2 ms warm, at parity with drift's singleisInstatement (benchmarks/cocoon_rivals/GETALL.md). Inside a transaction,tx.getAll(refs)sees the transaction's own writes exactly liketx.get(ref);collection.getAll(ids)is the sugar when you already have the collection in hand.
Everything in 0.1.0 #
Cocoon is a typed, disk-backed, Firestore-style local database for Dart and Flutter, built on SQLite. Your model classes are the schema; documents live in collections and subcollections inside one SQLite file on disk; queries run only when an index serves them, exactly like Firestore; and the bounded hot-document cache keeps RAM flat no matter how large the file grows. This is the first published release. Everything below is in it.
-
Existing databases upgrade themselves on open — and every file got ~18% smaller at 1M transactions. The internal
pathcolumn (and its unique index) is gone from collection tables: a document is identified by its path params + id, and the full path you see in refs, snapshots and the change log is derived from it. The public API is unchanged. Files written by older Cocoon versions are rebuilt table by table inside the first open that touches them (crash-safe — a failure leaves the old file and the next open finishes; progress reports through the existingonMigrationProgress), a file written by a NEWER version refuses to open with a typedStorageFormatTooNewException, andrestoreBackupaccepts older-format backups (they migrate at the next open). Measured on a 1M file: 409 MB → 320 MB compacted (390 → 305 MiB), the one-time migration takes ~5-9 s and keeps every document and change-log row. -
Bound the change log (it exists for sync consumers; watches don't read it).
db.pruneChanges(upToSeq:)now returns how many rows it deleted; newdb.earliestChangeSeq()is the oldest seq still readable; and a consumer that reads from a pruned watermark gets a typedChangesPrunedExceptioninstead of a silent gap — resync fully, like Firestore's "resume token expired". Or let the engine do it:CocoonOptions(changeRetention: ChangeRetention(maxChanges: 50000, maxAge: Duration(days: 30)))keeps the newest N rows and/or everything newer than the duration, deleted in small batches after commits (never inside a user transaction, never stalling a write). The default keeps everything, as before.lastChangeSeq()still never goes backwards. -
bulkWriter(logChanges: false): importing a million documents no longer writes a million change-log rows (~52 MB of the file at 1M). In-process watches still fire per chunk; sync consumers must resync fully afterwards, because the log has a hole where the load ran. -
Batch
updatenow actually persists: anupdateof a document that already existed on disk was written with a shifted bind order and silently matched nothing — the new value only ever lived in the hot-document cache. Caught and fixed by the scope-key rewrite; reads from a fresh instance now see the update. -
Butterfly CLI shortcuts (optional):
butterfly generate model <Name> --cocoon [--collection <path>]scaffolds a starter@Collectionmodel, andbutterfly cocoon migrate [args...]/butterfly cocoon buildrun your project's owndart run cocoon_generator:migrateanddart run build_runner build -dfor you (args and exit codes pass through untouched, so the interactive rename/drop menus work). The CLI never generates, diffs or installs anything — your pinnedcocoon_generatorstays the only authority — and Cocoon works exactly as before without it. -
@Mirrored(): totals over fields you don't filter on. Annotating a field@Mirrored()copies it into its own typed column — exactly like an index field's column — without creating an index, sosum/average/minOf/maxOfandgroupByread it whilewhere/orderByon it still follow the index rule (MissingIndexExceptionsuggests theIndex([...]), or.allowFullScan()). The annotation is for fields you aggregate but never filter or sort on (an amount, a category): it no longer costs an index you don't use — measured ~4.2 bytes per row for the column vs ~27 for column + index at 1M transactions. A field that is both mirrored and indexed is just indexed. Adding the annotation to an existing collection adds and backfills the column on the next open (one pass, in the open transaction, crash-safe; ~12 s at 1M) and removing it drops the column;bulkWriter()fills mirrored columns too; the DevTools schema view labels them "mirrored (no index)". -
Aggregates: your totals no longer decode documents.
query.aggregate([count(), sum(Txn$.amountMinor), average(Txn$.rate)])computes in one SQLite statement over the mirrored columns — Firestore's aggregate API with Firestore's semantics (empty sum = 0, average always adouble?, nulls skipped; integer sums exact, overflowing past ±2^63 throws a typedAggregateOverflowExceptioninstead of wrapping), plus Cocoon'sminOf/maxOf. The filters follow the same index rule asget()(the sameMissingIndexExceptionwith the same suggestedIndex([...]), the same.allowFullScan(), the same wait behind a runningbulkWriter()), anorderBy+limitaggregates the first N documents in that order, andquery.count()is now exactlyaggregate([count()]). Aggregate fields must be mirrored into a column (anIndex([...])member or a@Mirrored()field) — anything else throws anUnmirroredFieldExceptionnaming the declaration that would mirror it. A Cocoon extension on top:query.groupBy([key(Txn$.currency), bucket(Txn$.bookedOn, width: 100), month(Txn$.bookedAt, utcOffset: Duration(hours: 8))]).aggregate([...]).get()returns oneAggregateGroupper key combination, ordered by the keys ascending, each carrying its typed keys plus the same aggregates andgetMin/getMax; keys can be a field, a path param (one group per parent on a collection group), a fixed-width numeric band, or a calendar year/month/day — calendar buckets need a fixedutcOffset(no IANA zones, no DST; the dartdoc explains what DST zones should do instead). Both kinds supportwatch()with the query watch's invalidation and dedup. At 1M transactions the groupBy path cut the rivals year report from 1,430 ms to 376 ms — drift's own SQLGROUP BYruns 358 ms, so the 3.6x gap is closed — and one account's monthly totals from 185 ms (9,874 documents read) to 35 ms (12 group rows) (benchmarks/cocoon_rivals/AGGREGATES.md). -
Backups and restores:
db.backupTo(path)writes a consistent, compacted copy of the whole database while the app keeps reading and writing — one read snapshot (SQLite'sVACUUM INTO), landed atomically as one file (0.8-1.0 s for a 388 MB, 1M-transaction database); an existing file refuses unlessoverwrite: true.Cocoon.inspectBackup(path)verifies a backup (Cocoon marker +PRAGMA integrity_check) and reads its metadata, with typed errors for "not a Cocoon backup", "damaged" and "written by a newer app".Cocoon.restoreBackup(backup:, path:, collections:)swaps a CLOSED database file for a verified backup: schema compatibility is checked before the current file is touched, the replaced database is kept as<path>.before-restore, and a failure at any point leaves the original database usable — then open as usual. The engine owns these two primitives and the file protocol; the app owns the policy (when, where, compression, encryption, files outside the database). Restores re-roll the database's newinstanceId, so sync consumers must compare it against the one saved next to their change-log watermark before trusting it (a restored database's change log is the backup's — a saved watermark can be ahead of the data). See the README's "Backup and restore". -
Bulk loads:
db.bulkWriter()— a Firestore-style BulkWriter for import-shaped work too big for one transaction. Queueset/update/deletesynchronously; the writer commits every 10,000 rows (tunablechunkSize) as one write transaction — atomic per chunk, never as a whole — andawait bulk.close()flushes the rest, rebuilds the deferred indexes and resolves when everything is durable. While a load runs it drops the touched tables' secondary indexes (when the table is empty or the load outgrows it) and rebuilds them at close; a query needing one of them waits until then,get()by id never waits, and a crash mid-load is recovered on the next open. Importing 1,000,000 transactions takes ~13 s atDurability.full(was 103 s throughdb.batch()per 1,000 rows on the same dataset — ~8x faster, within ~1.4x of raw SQLite's own deferred-index ceiling). Not atomic as a whole, atomic per chunk;db.batch()stays the tool when everything must commit or roll back as one. See the README's "Bulk loads" section. -
Bulk writes:
db.batch()queuesset/update/deletesynchronously andawait batch.commit()runs them in call order inside one write transaction — consecutive same-SQL operations batch into one round trip per ~2,000 documents, every change-log row lands in one batch, the cache is updated once and watches receive exactly one commit event. Semantics per operation are the standalone ones (updatefails the whole batch when the document is missing; any failure rolls everything back), and a failed batch leaves no cache or watch effects. This is the path for import-shaped work; it mirrors drift'sdb.batch. -
Writes are much faster at scale. Document writes inside
runTransactionnow buffer and flush as batched statements (before the commit, before any read or delete in the same transaction, every 2,000 rows), each collection's write SQL is built once instead of per document, and a write caches the document's bytes so the first read decodes them once. Measured on the rivals benchmark: the 100k import dropped from 7.9 s to ~2.9 s (was 2.9x drift's time, now on par), and the 1M import from ~119 s to ~37 s. NewCocoonOptionsknobs back this:pageCacheMiB(a 64 MiB SQLite page cache by default, was 2 MB) andtempStoreInMemory(temp B-trees in RAM), plus sqlite_async's dev-mode query timeline tracing is now off (it cost ~5 s of a 100k import and nothing in Cocoon reads it). -
A rivals benchmark,
benchmarks/cocoon_rivals, now measures Cocoon against drift, Isar, ObjectBox, Hive CE and sembast — plus Cocoon itself atDurability.normal— on the same finance-shaped data at up to 1,000,000 transactions, every rival's read results diffed against Cocoon's before any timing counts. At 1M txns Cocoon's indexed reads stay flat where the store-everything engines scan (latest-50 0.74 ms p50 vs 641 ms Hive and 769 ms sembast; the paged all-accounts month range 320 ms vs 2,476 ms for drift's offset paging and 99 s for sembast) and its memory share stays at the bare-harness floor where Hive and sembast hold ~2.2 GB; the losses stay in view: bulk import 1.3-15x slower than the six rivals, the year report 3.6x behind drift's SQL aggregation, restore 11x behind Hive, gets by id 6-20x behind the in-RAM engines, single writes atDurability.fullequal to drift's fsynced default (Durability.normal: 402 µs p50, 7.3x faster than drift's default). Tables and per-workload verdicts:benchmarks/cocoon_rivals/RESULTS.md. -
The Cocoon benchmark now measures memory without the harness's dataset in RAM: only
importbuilds the typed dataset, other operations take their ids and values from the database itself, and abaselinerun reports the bare-harness floor. Re-measured numbers (packages/cocoon_bench/RESULTS.md) show Cocoon's own footprint is flat — indistinguishable from the bare harness after open, +39 MB after reads at every size from 10k to 300k txns, cold open ~80 ms flat — while sembast grows to +598 MB over the floor at open and +704 MB after reads at 300k. New tuning measurements next to the defaults: single writes withDurability.normal(p50 2,032 → 483 us at 100k) and withkeepChanges: false(~11%), and gets by id withcacheSize2000 vs 20000, cold then warm in one process (warm 1,000-doc working set: 113 → 2 ms; RSS unchanged — the cache charges RAM only for documents actually read). The README and the docs site's Internals page carry short Benchmarks and Tuning sections from these numbers. -
Map<String, V>andMap<int, V>fields, for every supported non-Map value type (primitives,DateTime,Uint8List, enums,List<...>,@Embeddedclasses; the map itself nullable or not). Stored like protobuf's map type — one nested message per entry, key at field 1, value at field 2 — with entries written sorted by key, so equal maps always encode to equal bytes. Missing on decode: constructor default, elsenullwhen nullable, else an empty map. Maps cannot be indexed; naming one inIndex([...])is a build error. -
packages/cocoon_bench, an honest benchmark of Cocoon against sembast on finance-shaped data (20 accounts sharing 10k/100k/300k txns over five years, with indexes, embedded splits and string maps): bulk import, cold open, RSS and peak memory, latest-50 reads, monthly totals, gets by id and single-write latency, each in a fresh process. Numbers and the plain-words verdict — including where Cocoon loses — live inpackages/cocoon_bench/RESULTS.md. -
A Cocoon tab in Dart DevTools for running debug apps: browse collections and documents with a path breadcrumb, drill into subcollections, and read decoded documents (dates as UTC ISO-8601 strings, enums by name, binary fields as
<N bytes>placeholders, embedded classes and lists as nested JSON). -
A query runner that runs typed queries read-only and explains the plan in plain words: which index served the query ("Used index cx_txn_ab12cd34"), when it read every row ("Full scan: add
Index([...])"), and the exactIndex([...])to declare when the index rule refused the query. -
A live change feed of every committed write, one row per touched document, filterable by collection and pausable, streamed from the app as
cocoon:changesDevTools events. -
A schema view with the live tables, columns, indexes and row counts, plus the lock's pinned field numbers and reserved numbers when the app passes
schemaLock:toCocoon.open. -
Zero setup: the inspector registers its
ext.cocoon.*service extensions in debug and profile builds only; product builds are untouched. The extension UI is a separate package (cocoon_devtools, not an app dependency) whose prebuilt output ships inside the cocoon package underextension/devtools/. -
Docs: new "DevTools Inspector" page on the docs site and a README section.
-
Documentation: Cocoon now has its own tab on the Butterfly docs site (docs/libraries/cocoon/) — overview and why, models and code generation, collections and documents, queries and indexes, realtime, schema migrations and internals — plus installation steps in the site's Installation page.
doc/design.mdis now the short decision record + Status table linking to those pages. -
Schema migrations, end to end.
dart run cocoon_generator:migrateasks only what a human must answer (renames, drops), updates the committedcocoon.lockand, when stored data must change, writes a numbered migration file with one conversion stub per type change. Implement the stubs, commit both, and passschemaVersion: cocoonSchemaVersionandmigrations: cocoonMigrationstoCocoon.open: pending migrations run once, in order, inside the open transaction — documents rewritten in batches of 500 with a progress callback, one change-log row per rewritten document, migrated columns and indexes rebuilt from the rewritten bytes. A database written by a newer app refuses to open (SchemaTooNewException); a failing conversion rolls the whole open back. -
The migration building blocks are public:
CocoonMigration(one version step),FieldMigration(+ the type-checkedFieldMigration.typed<S, T>wrapper the generated code uses) andMigrationWireType, the exact wire encoding of both sides of a rewrite.rewriteFieldis the pure wire-level rewrite they run on: absent old fields are skipped without calling the conversion,nullconversions drop the value, and everything else is re-serialized byte-for-byte in canonical order. -
Code generation: annotate plain classes with
@Collection(and nested value classes with@Embedded) andcocoon_generatorwrites the adapter, typed query handles (Txn$.bookedOn) and the canonical codec. Field numbers come from the committedcocoon.lock, so reordering fields never matters, and the build stops with a clear message when the models and the lock disagree instead of silently writing data under the wrong numbers. -
Documents written before a field existed still load: the field takes the constructor default,
null, or its zero value. -
CollectionAdapterhas a const constructor, so adapters can beconst. -
Realtime watches.
docRef.watch()emits the current value on listen and again only when a commit changes that exact document — a rewrite that encodes to the same canonical bytes never emits, and a delete emitsnull.query.watch()emits pages, re-runs only for commits in the query's collection (and parent scope, for subcollections), coalesces bursts of commits into at most one more re-run, and skips pages identical to the previous one — so UIs rebuild only when something they show changed. Cancelling a subscription unregisters the watcher; closing the database completes every watch stream. -
query.watchFrom(cursor): a live window anchored at a fixed cursor. Rows inserted before the anchor never shift it — how live paginated lists keep their pages stable while new data arrives. -
A durable change log. Every commit appends one row per touched document (
Change: seq, collection, path, op, at) in the same transaction as the write, including one row per cascaded delete. Read it withdb.changesSince(seq), track the head withdb.lastChangeSeq(), and trim consumed rows withdb.pruneChanges(upToSeq: ...)— ready for a future sync outbox or audit trail. Opt out withCocoonOptions(keepChanges: false)when only in-process watches are needed; notifications are unaffected. -
The storage engine is here: open a Cocoon database on disk, write and read typed documents in collections and subcollections, delete a document with its whole subtree, and run transactions — all on SQLite with bounded RAM (
Cocoon.open,CocoonOptions,DocRef,CollectionRef,CollectionGroupRef,CocoonTransaction). -
Collections are wired to SQLite with a
CollectionAdapter<T>(what the code generator will emit later): typedFieldColumnhandles copied into real columns at write time,ParamColumnhandles for path params, andIndexDefindex declarations. Column types coverint,double,bool,String,DateTime(UTC microseconds) and enums. -
Queries are Firestore-strict: filters and sorts only run when an index serves them, and the failure names the
Index([...])to declare (MissingIndexException)..allowFullScan()opts out. Filter ops: equality, inequality, comparisons,whereIn(≤ 30),isNull/isNotNull. -
Keyset pagination over
QueryPage: opaque cursors (startAfter) that are rejected when reused on another query, page-size changes allowed,nextCursornull on the last page.count()on any query, under the same index rule. -
Reopening the database reconciles the schema by itself: creates missing tables and indexes, adds new field columns and backfills them from stored documents in batches of 500, drops removed columns and indexes, and does no DDL at all when the stored shape already matches.
-
Transactions read their own writes and roll back fully on error — the hot-document cache only updates after a successful commit.
-
Document ids are time-ordered UUIDv7 (
newDocId()): monotonic within a millisecond, lowercase and hyphenated. -
A bounded LRU cache (default 2000 documents,
cacheSize: 0disables) keeps hot reads out of SQLite; every write and delete keeps it consistent. OneCocooninstance must own its database file.
0.0.1 #
- Initial scaffold: collection annotations, typed collection/document paths, a canonical protobuf wire codec, opaque keyset cursors, and the schema lock diff that drives interactive migrations. No storage engine yet.
