Skip to main content

hydrated_mobx 3.0.0 copy "hydrated_mobx: ^3.0.0" to clipboard
hydrated_mobx: ^3.0.0 copied to clipboard

An extension to the MobX state management library which automatically persists and restores MobX states.

HydratedMobX #

Persist and restore MobX stores automatically. A store extends HydratedMobX, says how to turn its state into JSON and back, and survives app restarts.

This package started from code in hydrated_bloc by Felix Angelov (MIT). Thanks to its authors.

A counter keeps its value after the app is killed and relaunched

Write stats Namespaces Schema migration
Writes saved by persistDelay and change detection Session and account boxes migrate renames a field
Eviction Observer Storage inspector
maxEntries keeps the 5 latest One observer for every store Every key in every storage

The screens come from the example app, one per feature area.

Features #

  • Restores a store's state when it is created and saves every change.
  • Write coalescing: persistDelay turns a burst of changes into one write; flush() / flushAll() write the pending state on demand.
  • No redundant writes: a change that leaves the JSON as it was is not written, and neither is the state a store just hydrated.
  • Several storages: one Hive box per namespace (session, account, cache), each store bound to the one it belongs in.
  • Targeted clearing: clearAll(where:) deletes only the tokens you name, in one deleteAll (one Hive operation for a HydratedStorage), and stops their stores first.
  • Memory: encoding: HydratedEncoding.gzipJson keeps large state as bytes, not decoded trees, until a store hydrates it; FileStorage keeps per-id state on disk until it is used.
  • Eviction: maxAge and maxEntries keep per-id caches from growing forever.
  • Schema versions: version + migrate, which can also drop old data.
  • One error hook: HydratedMobX.observer sees every store's hydration and write failures, and every migration.
  • Encryption (AES-256), every platform Hive supports, an InMemoryStorage for tests, and importData for moving data in from another layer.

Benchmarks #

dart run benchmark/persistence_benchmark.dart, the same scenarios run against 1.2.0. Apple M4 Pro, Dart 3.13 JIT, Hive on the local SSD.

Scenario 1.2.0 2.0.0
1000 rapid changes to one store 1000 writes 1000 writes, or 1 with persistDelay: 100ms
Re-creating a hydrated store 100 times 100 writes 0 writes
One change to a 2000-row state, time blocking the caller 1.15 ms 0.97 ms, 0.41 ms with trustToJson
500 per-id cache keys kept forever 50 kept with maxEntries: 50

benchmark/memory_benchmark.dart measures what an open storage costs in memory. 200 note-sized states (900 transcript segments each, 28.7 MB of JSON), none of them hydrated, Dart 3.13 JIT on an M4 Pro, the median of three runs:

Storage Heap held On disk Open Read, per note
HydratedStorage, tree (default) 146 MB 34.9 MB 248 ms 0 ms
HydratedStorage, json 29 MB 28.7 MB 89 ms 0.52 ms
HydratedStorage, gzipJson 4.0 MB 4.0 MB 15 ms 0.57 ms
FileStorage, gzipJson 1.4 MB 4.0 MB 4 ms 0.60 ms, from disk

Heap figures vary by about a megabyte from run to run, so the FileStorage figure is within noise of nothing held.

Getting started #

dependencies:
  hydrated_mobx: ^3.0.0

Build the storage before runApp:

Future<void> main() async {
  WidgetsFlutterBinding.ensureInitialized();
  HydratedMobX.storage = await HydratedStorage.build(
    storageDirectory: kIsWeb
        ? HydratedStorageDirectory.web
        : HydratedStorageDirectory(
            (await getApplicationSupportDirectory()).path,
          ),
  );
  runApp(const App());
}

Upgrading from 1.x? Keep the directory you built the storage in before. The 1.x docs used getApplicationDocumentsDirectory(); a different directory opens an empty box, and every store starts from its defaults.

A hydrated store #

part 'counter_store.g.dart';

class CounterStore = _CounterStore with _$CounterStore;

abstract class _CounterStore extends HydratedMobX with Store {
  @observable
  int count = 0;

  @action
  void increment() => count++;

  @override
  String get storagePrefix => 'counter';

  @override
  Map<String, dynamic>? toJson() => {'count': count};

  @override
  void fromJson(Map<String, dynamic> json) => count = json['count'] as int;
}
  • storagePrefix is the store's fixed name on disk. It is required: a name taken from the class would change with a rename or an obfuscated build, and the stored state would be lost. Two store types must not share one.
  • The store hydrates in its constructor, before your constructor body runs, so toJson and fromJson may only use fields set by initializers.
  • toJson returning null skips the write.

Several instances of one store #

Give each instance its own key with storeId (or override id). The key is '$storagePrefix${storeId ?? id}'.

abstract class _ThreadStore extends HydratedMobX with Store {
  _ThreadStore(this.threadId) : super(storeId: threadId);

  final String threadId;

  @override
  String get storagePrefix => 'thread_';
  // ...
}

Writing less #

Coalescing with persistDelay #

Every change is written right away by default. With a persistDelay, changes made within that window are written once, when it ends:

_DraftStore() : super(persistDelay: const Duration(milliseconds: 300));

Set HydratedMobX.defaultPersistDelay to apply one to every store built without its own.

A coalesced change is lost if the process dies inside the window, so write the pending state when the app goes to the background:

AppLifecycleListener(onPause: () => HydratedMobX.flushAll());

store.flush() does the same for one store, and completes when the write has finished. dispose() starts a pending write instead of dropping it.

Unchanged state is not written #

Before writing, the store compares the new JSON with the JSON it last wrote or hydrated, and skips the write when they are equal. Opening a cached screen therefore costs a read and no write.

trustToJson for large states #

Before writing, the store walks the JSON your toJson returned: it calls toJson() on objects it meets and copies the result. When your toJson already returns plain JSON, skip that walk:

@override
bool get trustToJson => true;

@override
Map<String, dynamic>? toJson() => {
      'page': _page.toJson((post) => post.toJson()),
    };

Plain JSON means only null, bool, num, String, List and Map<String, ...>, built fresh on each call. Returning a collection the store keeps and later mutates breaks the comparison above.

With json_serializable, set explicit_to_json: true so nested objects come out as maps. A generic DTO's toJson takes a converter, which the walk cannot supply; call it yourself, as above, rather than round-tripping through jsonEncode/jsonDecode.

What fromJson receives #

fromJson gets a fresh, deep copy of the stored state: every nested map is a Map<String, dynamic> and every list a List<dynamic>, whatever Hive returned. Pass the parts straight to json_serializable factories; there is no need to round-trip through jsonEncode first.

The HydratedJson helpers read fields with safe defaults (readList, readObject, readStringList, readString, readInt, readDouble, readBool, writeList).

Errors #

Persistence runs in the background, so failures do not reach your code. Set an observer once to see them all:

class CrashReportingObserver extends HydratedObserver {
  @override
  void onStorageError(HydratedMobX store, Object error, StackTrace stack) =>
      FirebaseCrashlytics.instance.recordError(error, stack,
          reason: 'persist ${store.storageToken}');

  @override
  HydrationErrorBehavior onHydrationError(
    HydratedMobX store,
    Object error,
    StackTrace stack,
  ) {
    FirebaseCrashlytics.instance.recordError(error, stack);
    return HydrationErrorBehavior.overwrite;
  }
}

HydratedMobX.observer = CrashReportingObserver();
  • onStorageError covers failed writes and states that cannot be serialized.
  • onHydrationError decides what happens after a failed read: overwrite (default) lets new changes replace the stored state, retain keeps it and stops writing until the next successful hydrate().
  • onMigrate hears about every migration.

A store that passes its own onStorageError or onHydrationError handles its failures itself instead.

Using a storage after close() throws StorageClosedException, which reaches the same handlers.

Several storages #

Build one storage per namespace, each with its own boxName, and bind stores with storage:. Clearing one never touches another.

final session = await HydratedStorage.build(
  storageDirectory: directory,
  boxName: 'session',
);
final account = await HydratedStorage.build(
  storageDirectory: directory,
  boxName: 'account',
);

class OutboxStore extends _OutboxStore with _$OutboxStore {
  OutboxStore(Storage accountStorage) : super(storage: accountStorage);
}

// On sign-out:
await HydratedMobX.clearAll(storage: session);

The storage is a constructor parameter, so it works with dependency injection and with test doubles. Stores built without one use HydratedMobX.storage.

Clearing state #

Call Deletes Stops persisting
store.clearPersisted() that store's state that store (resume: true keeps it going)
HydratedMobX.clearAll(storage: s) everything in s every store bound to s
HydratedMobX.clearAll(where: (token) => ...) the matching tokens the matching stores
HydratedMobX.deleteToken('token') one token its store, if live
store.dispose() nothing that store, for good

Every call also deletes the version keys of what it deletes, and drops writes already scheduled, so cleared state cannot come back. clearAll(where:) deletes in a single deleteAll, so the keys you keep are never rewritten.

HydratedMobX.versionKeyOf, isVersionKey and tokenOfKey map between a store's token and the .__version key kept beside versioned state.

Call dispose() when a store's owner goes away (a widget's dispose, a DI scope's teardown). A persisting store is kept alive by its MobX reaction, and dispose() releases it.

Eviction #

Per-id stores (one per user, per thread, per item) add a key each and are never deleted on their own. Bound them when building the storage:

final cache = await HydratedStorage.build(
  storageDirectory: directory,
  boxName: 'cache',
  maxAge: const Duration(days: 30),
  maxEntries: 200,
  evictable: (key) => key.startsWith('profile_'),
);
  • maxAge deletes keys not hydrated or written for that long. A plain storage.read does not count, so an inspector screen keeps nothing alive.
  • maxEntries then keeps only the most recently used keys.
  • evictable limits both to the keys it accepts.

Eviction runs when the storage is built, before any store reads from it. storage.evict(...) runs it again later, and stops the live stores of the keys it deletes. Only a storage built with maxAge, maxEntries or trackUse: true records use times, in a small companion box; other storages pay nothing for eviction. A key's use time is rewritten at most once per useResolution (a minute by default). Keys written before 2.0 count as used at the first 2.0 launch, unless initialUseTime dates them from their stored value (see below). Keep the eviction settings the same across launches: keys written while use times were not tracked look as old as their last recorded use.

evict returns the keys it deleted, so state kept beside them under other keys can go too. keepLive: true spares the stores that are alive, like the item on screen; HydratedMobX.liveTokens(storage: cache) lists them. A key that may only go once some other state is safe, such as an outbox that has not synced, belongs in where:

final cache = await HydratedStorage.build(
  storageDirectory: directory,
  boxName: 'cache',
  trackUse: true,
  // Rank state saved before use times were kept by a date it carries,
  // instead of treating all of it as used at this launch.
  initialUseTime: (key, value) =>
      value is Map ? DateTime.tryParse('${value['openedAt']}') : null,
);

final evicted = await cache.evict(
  maxEntries: 30,
  keepLive: true,
  where: (key) => key.startsWith('note_') && !hasUnsyncedEdits(key),
);
const attachments = 'attachments_';
await HydratedMobX.clearAll(
  storage: cache,
  where: (token) =>
      token.startsWith(attachments) &&
      evicted.contains('note_${token.substring(attachments.length)}'),
);

The second step is best effort: if the app stops between the two calls, the attachments of the notes evicted then stay. To catch those too, match attachments against the notes still stored instead of against evicted (!cache.keys.contains('note_$id')). Run await HydratedMobX.flushAll(storage: cache) first: a new note whose first write is still held back by persistDelay has no stored key yet, and its attachments would go with the evicted ones.

compactionStrategy tunes when Hive rewrites the box file.

Memory #

Hive keeps every value of an open box in memory, decoded, for as long as the box is open, whether or not a store ever reads it. A decoded JSON tree takes about five times the memory of its text, so a box of large per-id states (one per document, say) can cost hundreds of MB at launch. encoding keeps large state as bytes instead and decodes it when a store hydrates:

final documents = await HydratedStorage.build(
  storageDirectory: directory,
  boxName: 'documents',
  encoding: HydratedEncoding.gzipJson,
  encodeAbove: 16 * 1024, // state up to this size stays as it is (the default)
);
  • tree (the default) keeps maps and lists as they are.
  • json keeps UTF-8 JSON, about a fifth of the memory.
  • gzipJson keeps gzipped JSON, a few percent of it, and a much smaller file. It is not available on the web.

Each read decodes again, so a store's hydration costs a fraction of a millisecond per 100 KB. Every value reads back whatever the setting, so it can change between launches. The first build after encoding or encodeAbove changes converts what is already in the box, back to tree too; later builds skip that walk. A live store also keeps the decoded state it last hydrated or wrote, to skip unchanged writes, so encoding saves the memory of state whose store does not exist, not of the one on screen. Do not let a version of hydrated_mobx before 3.0 open an encoded box: it cannot read the encoded values, and its stores would overwrite them.

One file per key #

Encoded state is small, but a box still holds all of it. For per-id state that only matters while its screen is open, FileStorage keeps each key in a file and holds nothing in memory until a store reads it:

import 'package:hydrated_mobx/file_storage.dart';

final notes = await FileStorage.build(
  storageDirectory: HydratedStorageDirectory('${support.path}/notes'),
  maxEntries: 50, // optional, as with HydratedStorage
);

// Before showing a note: read it ahead, so the store hydrates from memory.
await notes.preload('note_$id');
final store = NoteStore(id, storage: notes);
  • Opening lists the directory and reads no file. With maxAge or maxEntries it also evicts, which reads the modification time of every file evictable accepts: one stat per file, at each launch, run in a background isolate.
  • read stays synchronous. A key that was not preloaded or used lately is read from its file there and then, which blocks for that read (0.6 ms for a 146 KB note in the benchmark, on a Mac; measure on your devices).
  • The last cacheBytes (8 MB by default) of encoded values read or written stay in memory. A value larger than that is never kept, so preload skips it and the store reads it from its file.
  • A write goes to a temporary file that is renamed over the old one, so a crash keeps the old value or the new one. deleteAll and clear delete one file at a time, each state key before its version key: a crash midway leaves some keys, never state without its version.
  • Eviction (maxAge, maxEntries, evict) works as with HydratedStorage, using each file's modification time as its use time.
  • Files are not encrypted. Keep secrets in an encrypted HydratedStorage.
  • One storage at a time may have a directory open. A second build on it, from the same isolate or another process (an app extension sharing an app group, say), throws a StateError until the first is closed. Isolates of one process share its file lock, so a second isolate of the app is not caught: keep each directory to one isolate.
  • The storage holds that lock until it is closed. iOS terminates a suspended app that holds a file lock in an app-group container, so a FileStorage kept there must be closed when the app goes to the background and built again when it returns. Live stores stay bound to the closed storage, so flush them before closing and move them to the new one with rebindAll, then evict, as in Rebuilding a storage. A directory inside the app's own container is not affected.
  • File names are the key in lowercase base32, so keys that differ only in case stay apart on macOS and Windows. A state key can be up to 144 bytes of UTF-8, which leaves room for its version key.
  • It is not available on the web, so it has its own import.

HydratedMobX.transferTokens moves existing state over, once, at startup:

await HydratedMobX.transferTokens(
  from: HydratedMobX.storage, // the Hive box
  to: notes,
  where: (token) => token.startsWith('note_'),
  onError: (token, error, stackTrace) => log('kept $token: $error'),
);

Each token is written to the new storage before it is deleted from the old one, so an interrupted run loses nothing and the next launch finishes it. A token that fails to move stays where it was while the others move; without onError the first failure is rethrown once all were tried. Use times move along between storages that keep them, so a maxEntries on the new storage keeps the notes used last. Both sides must be different storages. Moving state out of an encrypted box into a FileStorage writes it unencrypted.

Schema versions #

Bump version when the shape of toJson changes, and upgrade older data in migrate. Return null to drop data too old to carry forward:

@override
int get version => 3;

@override
Map<String, dynamic>? migrate(int oldVersion, Map<String, dynamic> old) {
  if (oldVersion < 2) return null; // v1 cannot be read any more
  if (oldVersion < 3) old['count'] = old.remove('counter') ?? 0;
  return old;
}
  • migrate runs during hydration when the stored version is lower. Its result goes to fromJson and is written under the new version, which is recorded only after the state, so a crash re-runs the migration instead of skipping it.
  • Returning null deletes the stored state; fromJson is not called.
  • Data written before versioning existed counts as version 1.
  • Apply each step with stacked if (oldVersion < N) blocks. Dart switch cases do not fall through.

Importing data #

Seed a storage from another persistence layer before building the stores that read it. Keys are store tokens.

final prefs = await SharedPreferences.getInstance();
await HydratedMobX.importData({
  'counter': {'count': prefs.getInt('count') ?? 0},
});

Existing keys are kept unless overwrite: true. Pass version: when the data is already in a newer schema, so it is not migrated again; storage: targets a storage other than the global one.

Encryption #

final key = sha256.convert(utf8.encode(password)).bytes;
HydratedMobX.storage = await HydratedStorage.build(
  storageDirectory: directory,
  encryptionCipher: HydratedAesCipher(key),
);

Hive encrypts values, not keys: a store's token (storagePrefix plus its id) is readable in the box file. Keep secrets out of storeIds. FileStorage does not encrypt at all.

Testing #

setUp(() => HydratedMobX.storage = InMemoryStorage());

Or pass storage: InMemoryStorage({'counter': {'count': 3}}) to the store under test to start it from a known state.

Wrapping or writing a storage #

To wrap a storage (to count or log writes, say), extend ForwardingStorage and override what you need. It forwards everything to inner, suspends its own stores on clear, and passes eviction use times through. A wrapper that holds a write, delete or clear back before passing it on must also have read and keys show it from the moment it is called, as described below.

class LoggingStorage extends ForwardingStorage {
  LoggingStorage(super.inner);

  @override
  Future<void> write(String key, dynamic value) {
    debugPrint('write $key');
    return super.write(key, value);
  }
}

Storage calls and observer callbacks run untracked, so a storage or observer may read MobX observables without the persistence reaction depending on them.

To write one from scratch, implement Storage: read, write, delete, deleteAll, keys, clear and close. read is synchronous, because stores hydrate in their constructor. From clear, call HydratedMobX.suspendAll(storage: this) before wiping, so no store writes its state back. If write, deleteAll or clear can wait behind other work (a lock, a queue), have read and keys reflect each from the moment it is called; otherwise a store built in between hydrates state that is then replaced or deleted under it, and its next change overwrites the newer state. Implement UseTrackingStorage.markUsed to take part in eviction, and UseTimeStorage (usedAt, setUsedAt) so that moveTokens and transferTokens keep each key's use time when they move its state.

Rebuilding a storage #

To close a storage and open it again while stores are live, move them over:

await HydratedMobX.flushAll(storage: cache);
await cache.close();
final reopened = await HydratedStorage.build(
  storageDirectory: directory,
  boxName: 'cache',
  trackUse: true,
);
HydratedMobX.rebindAll(from: cache, to: reopened);
await reopened.evict(maxEntries: 200);

Evict after rebindAll, not while building: eviction stops the live stores of the keys it deletes, and only once they are bound to the new storage.

Performance notes #

  • Each change runs toJson and the comparison on the calling isolate; the Hive write itself is asynchronous. Use persistDelay for stores that change often, and trustToJson for large ones.
  • HydratedStorage keeps its box in memory, which is what makes read synchronous. Keep caches bounded with eviction rather than storing everything in one box.
  • With encoding, every write of a map or list encodes it to UTF-8 JSON to learn its size, and state up to encodeAbove then stays a tree. That is at most encodeAbove bytes of encoding per write (about 60 µs for 16 KB, AOT on an M4 Pro). Larger state is kept as those bytes, gzipped for gzipJson, and every read of it decodes them.
  • A FileStorage read of a value not in its cache is a synchronous file read and a decode on the calling isolate. preload does the file read ahead, asynchronously; the decode still happens at read.
  • hive_ce's IsolatedHive is not used: its reads are asynchronous, which a constructor-time hydration cannot wait for, and every write would still copy the whole state to the other isolate from the calling one.

Example app #

example/ has one screen per feature area:

Screen Shows
Counter A store that survives a restart; persistDelay and flush()
Keyed instances One store type, one key per storeId
Write stats Writes counted through a ForwardingStorage: 100 changes, 1 write
Namespaces Session and account boxes; sign-out clears one; clearAll(where:)
Schema migration Seeding old data with importData; migrate upgrading or dropping it
Eviction A per-item cache bounded by maxEntries and evictable
Observer & errors Hydration, serialization and closed-storage errors in one log
Storage inspector Every key in every storage, with delete and clear
Encryption An AES-256 box; the note is not in the file as plain text
trustToJson Time per write of a 2000-row state, with and without it
Large state Notes kept gzipped in a box, moved to a FileStorage with transferTokens, opened with preload

moveTokens and LegacyTokens are meant to run once, at app startup, before any store exists. The example app has no 1.x keys to move, so it does not call them; test/move_tokens_test.dart shows them at work.

cd example
flutter run

Migrating from 1.x #

1.x 2.0
storagePrefix defaults to runtimeType Required. Return the old class name (e.g. 'CounterStore') to keep reading 1.x data.
store.clear({resume}) store.clearPersisted({resume})
Map<String, dynamic> migrate(...) Map<String, dynamic>? migrate(...); null drops the data
defaultOnHydrationError, defaultOnStorageError HydratedObserver; per-store handlers are optional
Storage had 5 members Add deleteAll and keys; call HydratedMobX.suspendAll(storage: this) from clear
HydratedMobX.disposeAllForClear() HydratedMobX.suspendAll(storage: ...)
A closed storage ignored writes and read as empty Reads, writes and keys throw StorageClosedException; store failures go to the error handlers
HydratedStorage.hive (testing hook) Removed; each storage has its own Hive instance
.hydrated_bloc.json imported at build Removed; use importData
Every change and every construction wrote Unchanged state is not written

Stored data needs no conversion: 1.x boxes open as they are, under the default boxName, as long as the storage is built in the same directory as before.

Obfuscated builds #

1.x keyed each store by runtimeType. In an obfuscated build (Android release with --obfuscate) that is a short generated name, and in a web release build a minified one such as minified:Ab. Either changes whenever the code around the class changes. Returning '$CounterStore' from storagePrefix then rarely finds the old key. Move the state onto fixed prefixes once, at startup, before any store exists:

await HydratedMobX.moveTokens(LegacyTokens.match([
  LegacyStore(
    prefix: 'user',
    legacyPrefix: '$UserStore',
    fields: ['currentUser', 'settings'],
  ),
  LegacyStore(
    prefix: 'chat_',
    legacyPrefix: '$ChatStore',
    fields: ['messages'],
    idPattern: RegExp('[0-9a-f]{32}'), // one key per chat id
  ),
]));

LegacyTokens.match finds a key by this build's '$Type' when the name survived, and otherwise by the shape of its value: a key that looks like a class name (plus an id, for a keyed store) whose top-level JSON keys fit exactly one store. A key with no id moves a keyed store's state to its prefix alone, which is where that store reads it when built without a storeId. A name the obfuscator gave to another class never moves one listed store's state onto another, because the shape has to fit either way.

Only the listed stores' prefixes are left alone. A current store you did not list, whose state has the same fields as a listed one, could be moved onto it, so pass every other prefix your app writes as reservedPrefixes:

LegacyTokens.match(stores, reservedPrefixes: ['settings', 'draft_'])

With a variable-length idPattern such as RegExp(r'\d+'), the longest id the pattern accepts is taken: Qz123 moves to id 123. The pattern is anchored for you, so a leading ^ or trailing $ is ignored.

A 1.x type name longer than a prefix it starts with still moves: a store now under Counter finds its old CounterStore key. Keys whose name no listed store could have written, such as some.key, are never read.

moveTokens takes any (token, value) => newToken?, so it also handles a renamed storagePrefix. value is a function that reads the stored value, so keys rejected by name are never read:

await HydratedMobX.moveTokens(
  (token, value) =>
      token.startsWith('todo-') ? 'todo_${token.substring(5)}' : null,
);

It moves the version key along, never overwrites different state, skips tokens of live stores, and can run again after a crash. A key that fails to move stays where it is while the others move; pass onError to hear about it, or the first failure is rethrown at the end. It returns the moves it made.

More #

9
likes
160
points
1.63k
downloads
screenshot

Documentation

Documentation
API reference

Publisher

verified publishernarek-manukyan.dev

Weekly Downloads

An extension to the MobX state management library which automatically persists and restores MobX states.

Homepage
Repository (GitHub)
View/report issues

Topics

#mobx #state-management #persistence #storage #hydration

License

MIT (license)

Dependencies

hive_ce, meta, mobx, synchronized

More

Packages that depend on hydrated_mobx