serverpod_offline_sync 0.0.1

Shared package for the Serverpod Offline Sync project.

Serverpod Offline Sync

A plugin to build offline-first applications with Serverpod. Develop your app as a normal offline app and use this non-invasive plugin to automatically handle the sync process using CRDT (Conflict-free Replicated Data Type). Works out-of-the-box with Serverpod's existing APIs.

• Serverpod Offline Sync
  • Overview
  • Features
  • Getting started
    • 1. Add the package to the project
    • 2. Prepare the synced tables
    • 3. Configure the sync engine on the server
    • 4. Configure the sync engine on the client
    • 5. Wire the sync call on the client
  • Usage
    • Scopes and sharing
    • Data modeling limitations
  • How it works
  • Performance
  • Example
  • Additional information
  • License

---

Overview

Serverpod Offline Sync turns an ordinary Serverpod backend into an offline-first, collaborative system, without changing how you write your app. It is a delta-CRDT engine that runs the same code on the client and server databases, transparently and with minimal configuration.

It sits nearly alone in the space of offline-first solutions, because most others either use a non-relational database, require a new data API, or don't preserve foreign-key/unique invariants while merging data. This package is built on top of Serverpod, leveraging its ORM, server, and client to expose these capabilities with almost nothing new to learn: you build a normal offline Serverpod app (generated models, ordinary CRUD) and the plugin handles replication automatically.

The sync engine is a black box that "just works": changes are tracked atomically with each operation and you never intervene in conflict resolution. Conflicts that need a human decision bubble up as visible state for the end user to act on (e.g. a delete reverted by a concurrent constraint reappears, so the user can simply delete it again). Migrations work normally, and deleted rows never leave their tables — they are transparently hidden from the user.

The CRDT layer was inspired by the excellent work on Synql (Ignat et al., DAIS 2024). This package goes beyond Synql by covering the full set of foreign-key onDelete actions, extending unique-constraint handling to deeply relational schemas, and shipping a production-ready implementation.

Features

• Offline-first, no new API. Read and write generated models with ordinary Serverpod CRUD against a local database; the plugin replicates for you.
• Same code on both ends. Identical logic runs on the client's SQLite and the server's Postgres, so there are no asymmetric merge results to reconcile.
• Automatic conflict resolution. Field-level merges ordered by hybrid logical clocks, with a monotone causal-length tombstone governing row existence so add/delete/restore can never oscillate.
• Relational integrity preserved. All foreign-key onDelete actions (restrict / cascade / set null / set default) and unique constraints are honored after every merge, as if the merged data had existed in a single database at the time of each operation.
• Conflicts surface to the user. Resolutions that require a human decision become visible state instead of silent data loss.
• Scopes and sharing. Data is isolated per scope; each user has a personal scope and can be a member of any number of read-only or read-write shared scopes.
• One-shot and continuous sync. The same stream-based protocol powers both pull-to-refresh and near-real-time streaming.
• Minimal overhead. Metadata is normalized and never copies row data, keeping the storage and performance penalty small (see Performance).

Getting started

Using the package on a project takes just five steps. After this setup, you develop the app with normal repository calls against the wrapped session.

Note

This package is still in development and is not yet ready for production use. Although the package is functional and feature-complete, it will still pass through some refactors and breaking changes for a better integration on real Serverpod projects.

1. Add the package to the project

Add the packages to the server and client pubspec.yaml files.

# your_project_client/pubspec.yaml
dependencies:
  serverpod_offline_sync_client: 0.0.1

# your_project_server/pubspec.yaml
dependencies:
  serverpod_offline_sync_server: 0.0.1

Note

Version 0.0.1 requires Serverpod 4.0.0-beta.0.

After adding the dependencies, list the serverpod_offline_sync module on the generator.yaml file:

modules:
  serverpod_offline_sync:
    nickname: offline_sync

The next call to serverpod generate and serverpod create-migration will include the offline_sync module in the project. Note that the client tables will only be present on the client's migration if at least one model on the project is defined with database: all or database: client.

2. Prepare the synced tables

Each model you want to sync must be defined with database: all (so it exists on both the client and the server), have a UUID primary key, and declare a scopeId field. You mostly never set scopeId nor see its value on reads; it is maintained by the CRDT sync layer.

class: Person
table: person
database: all
fields:
  id: UuidValue?, defaultPersist=random_v7
  ### Owner scope of this row. Maintained by the CRDT sync layer.
  ### The user never sets this field nor sees its value on reads.
  scopeId: int?, relation(parent=crdt_scopes, onDelete=Cascade)
  name: String

3. Configure the sync engine on the server

On the server, the databaseInterceptor is what ensures that all session.db objects are database instances that tracks operations for sync.

final pod = Serverpod(
  args,
  Protocol(),
  Endpoints(),
  // Add the database interceptor.
  databaseInterceptor: crdtDatabaseInterceptor,
);

// Initialize the sync engine before pod.start() is called.
pod.initializeCrdtSync(
  syncTables: [Person.t, Book.t, Author.t],
);

Don't forget to also configure the authentication following the regular Serverpod auth setup. Authentication is required because the sync needs a logged in user to synchronize data.

4. Configure the sync engine on the client

On the client, the regular DatabaseSession must be replaced by a wrapped version that includes the sync engine.

final session = CrdtDatabaseSession.wraps(
  await client.createSession(databasePath, isDebugMode: kDebugMode),
  syncTables: [Person.t, Book.t, Author.t], // Must be the same as on the server.
  persistentUserId: persistentUserId,
);

// Initialize the sync engine.
await session.db.initialize();

Then, store the session instance in the service locator.

Note

The list of synced tables must be the same on the server and the client. Otherwise, all sync attempts will fail with a schema mismatch error.

5. Wire the sync call on the client

Be sure to authenticate the client before calling the sync methods. Otherwise, the sync operation will fail with a Unauthorized error.

// Push local changes and merge remote ones once (i.e. pull-to-refresh).
await client.crdt.syncOnce(session);

// Or stream changes both ways until cancelled for a near-real-time sync.
final syncSession = client.crdt.syncContinuously(session);

Usage

Once wired up, use normal generated-model CRUD against the session instance, like Person.db.insertRow(session, person), Person.db.find(session), etc. The sync layer tracks every operation atomically; you never touch conflict resolution.

Scopes and sharing

Data is isolated per scope. Each user has their own implicit personal scope and can be a member of any number of shared scopes, with read-only or read-write access.

• On the server, wrap database operations in session.db.transactionForUser to target the correct scope.
• On the client, set a persistentUserId to target the personal scope by default, or use transactionForUser to target a shared scope.

Shared scopes are created and managed on the server only through the session.crdt.scopes API.

// Create a shared scope granting read-write access to a single user.
final scopeId = await session.crdt.scopes.createFor(userUuid);

// Create a shared scope with multiple members.
final scopeId = await session.crdt.scopes.create(
  grants: {
    userUuid: CrdtScopeRole.readWrite,
    anotherUserUuid: CrdtScopeRole.readOnly,
  },
  // It is also possible to pass a transaction object to all operations.
  transaction: transaction,
)

// Add or update a member's role (readOnly or readWrite).
await session.crdt.scopes.grant(
  scope: scopeId,
  user: bobUuid,
  role: CrdtScopeRole.readWrite,
);

// Remove a member. No-op if they are not already a member.
await session.crdt.scopes.revoke(scope: scopeId, user: bobUuid);

If a streaming sync is ongoing, clients learn of new scopes and role changes on the next sync cycle and apply them immediately. There is no need to restart the sync session for changes to take effect.

The package does not expose a management endpoint, since this is a domain responsibility. If desired, create your own endpoints where you can enforce an authorization policy (invitations, team ownership, etc.) and call the session.crdt.scopes API accordingly.

Data modeling limitations

Because of the nature of merge conflicts, the package imposes some data-modeling limitations, enforced at runtime during initialization. Most are fundamental to the design and can never be lifted.

• Every synced table must:
  • Exist on both the client and the server with database: all.
  • Have a UUID primary key.
  • Declare the field scopeId: int?, relation(parent=crdt_scopes, onDelete=Cascade).
• Unique indexes must include scopeId together with the target columns.
• Global unique indexes are unsupported, except for FK-only indexes.
• Unique indexes are only supported with at least one String/UuidValue/nullable column.
• All 1:1 relations must have the foreign-key column nullable (optional relation).
• The only allowed non-synced-to-synced relation is scopeId -> crdt_scopes.id.
• All foreign-key relations must be deferrable (pending serverpod#5338).

Respecting these limitations, all other database invariants — including foreign-key actions — are preserved, with the exception of check constraints, which Serverpod does not support either.

How it works

• Architecture. The engine is a transparent layer between the application and its database. Because identical code runs on the client's SQLite and the server's Postgres, there are no asymmetric merge results to reconcile. Migrations are generated symmetrically for both ends. The engine respects the three mathematical properties a CRDT must hold to be consistent: idempotence, associativity, and commutativity.
• Metadata model. Three metadata tables track change history without copying row data: crdt_data_rows (insert + visibility), crdt_data_fields (update), and crdt_data_tombstone (delete/restore). Hybrid logical clocks order field-level values and break ties. A monotone causal-length tombstone governs row existence, so add/delete/restore advance generations forward and can never oscillate.
• Visibility as a pure function. Replicas never coordinate. Each merge applies remote facts monotonically, then derives the visible database — tombstone state, unique-conflict winners, and foreign-key repair — as a deterministic projection computed to a fixed point.
• Ownership & scopes. A scope is the unit of replication and ownership; a user is an authentication identity. Each row is globally identified by (table, rowId) and owned by exactly one scope, so merging is always one chain per row. Scopes can be purged for GDPR / account erasure via a database-enforced cascade.
• Sync protocol. A session handshakes once to validate that both ends share the same table schema. Per cycle, the server sends the authoritative scope set, and every scope the user can access is synced (transferred all at once in chunks, merged one at a time). The same stream-based implementation powers both one-shot and continuous sync.

For a deeper dive, see the design docs in docs/: row ownership, foreign-key invariants, unique-constraint invariants, and sync/non-sync relations.

Performance

Since all operations are tracked, a CRDT layer can weigh heavily on storage and speed. This package was built to keep that penalty minimal. The benchmark below uses 1,000 rows to measure the impact against a baseline of the same operations without the CRDT layer, both on SQLite (a fresh report is posted as a comment on every pull request):

📊 SELECT performance impact:
  Time: 12.62 ms --> 12.67 ms (48.50 μs)
  CRDT overhead: 0.38% slower

📊 INSERT performance impact:
  Delay per insert: 231.04 μs

📊 UPDATE performance impact:
  Delay per update: 1.88 ms

📊 DELETE performance impact:
  Delay per delete: 408.75 μs

💽 Storage impact (base footprint removed):
  Storage overhead range: 62.07% --> 300.00%

Despite the high percentage slowdown on insert/update/delete, the absolute impact is mostly in the microsecond range — barely noticeable in practice, and far below the network delay those operations would incur through an online endpoint. Storage tops out at ~3× the base size, an acceptable trade for offline-first speed and availability.

Example

The bundled desktop demo shows the model directly: two independent replicas of one account side by side, plus the server's merged truth. Drive each replica's sync independently and watch them converge. It visualizes hidden / tombstoned rows and every foreign-key onDelete action, with guided scenarios for the restrict / cascade / set-null / set-default / unique-conflict cases.

Run it by starting the test server — the app launches alongside it:

cd test/serverpod_offline_sync_test_server
serverpod start

See example/offline_sync_demo for details; the entire integration lives in ~60 lines in lib/src/offline_replica.dart.

Additional information

The package is tested extensively following Serverpod's testing principles. The vast majority of the tests live in the serverpod_offline_sync_test_server package, whose server is shared with the example/offline_sync_demo project.

License

This package is distributed under the BSD 3-Clause License. See the LICENSE file for more information.