QueueKit

pub package license

A task queue for Dart and Flutter. Queue up units of work as events, run them one at a time in order, retry the ones that fail, keep dependent events in order, and persist the whole queue so it survives an app restart.

Features

  • Sequential processing: events run one at a time, in the order they were added.
  • Start and pause: nothing runs until you say so, and you can pause at any time.
  • Typed results: listen to the results of all events, of one event type, or wait for a single event.
  • Retries: linear and exponential backoff out of the box, or bring your own strategy.
  • Dependent events: an event can depend on other pending events; it runs after them and waits with them through retries, so their order is preserved.
  • Queries: look up what is pending by id, type or any predicate.
  • Persistence: HydratedQueue saves the queue through callbacks you provide and restores it on the next launch, including retries in flight and dependencies.

Installation

dart pub add queuekit
# or, in a Flutter project
flutter pub add queuekit

QueueKit requires Dart 3.11 or later and has no Flutter dependency.

Quick start

import 'package:queuekit/queuekit.dart';

class SendMessage extends Event<String> {
  SendMessage(this.text) : id = 'send-${_created++}';

  static int _created = 0;

  // Ids must be unique among pending events.
  @override
  final String id;

  @override
  String get type => 'SendMessage';

  final String text;

  @override
  RetryConfig? retryConfig = LinearRetryConfig(maxRetries: 3, delay: const Duration(seconds: 2));

  @override
  Future<String> run() async {
    // Do the actual work, for example an HTTP call. Throw to fail the event.
    return 'sent: $text';
  }
}

void main() async {
  final startListenable = QueueStartListenable();
  final queue = Queue(startListenable);

  queue.listenWhere<SendMessage, String>(
    (result) => print('${result.event.text} -> ${result.result}'),
    onError: (event, error, stackTrace) => print('${event.text} failed: ${error.error}'),
  );

  queue.add(SendMessage('hello'));
  queue.add(SendMessage('world'));

  startListenable.start(); // Events start running here.

  final event = SendMessage('!');
  queue.add(event);
  print(await queue.waitFor<SendMessage, String>(event)); // sent: !
}

Events

An event is a unit of work. Extend Event<T> where T is the type of the result run produces.

abstract class Event<T extends Object?> {
  String get id;
  String get type;
  RetryConfig? get retryConfig;
  Future<T> run();
}
  • id identifies the event while it is pending. It must be unique among pending events: the retry queue, dependencies and findEvent are all keyed by it. Generate one per instance, for example with package:uuid, and persist it with the event if you use a HydratedQueue.
  • type names the kind of event. HydratedQueue uses it to pick the event's EventSerializer and writes it into the saved data, so it must stay stable across builds: a hand-written constant, never runtimeType.toString(), which obfuscated release builds change.
  • retryConfig decides what happens when run throws. Return null to never retry.
  • run does the work. Throwing fails the event; the returned value is the event's result.

The queue

final startListenable = QueueStartListenable();
final queue = Queue(startListenable);

A Queue processes events one at a time, in the order they were added, while it is started. QueueStartListenable controls that:

Member Effect
start() Starts processing. Events added while the queue is started run right away.
pause() Stops processing after the event that is currently running. Events can still be added.
isStarted Whether the queue is currently started.

A queue starts paused. Set up your listeners, add your events, then call start(); a listener added later misses the results that were already emitted. With a HydratedQueue, call hydrate() before start() so restored events run before new ones.

QueueStartListenable is a StateNotifier<bool>, so it can be driven by connectivity, authentication state or anything else your app knows about.

Adding events

queue.add(event);
queue.add(event, dependsOn: [otherEvent]); // See "Dependent events" below.

add appends the event to the end of the queue. Adding to a started queue that is idle starts it immediately.

Listening for results

queue.results is a broadcast Stream of QueueListenerResult, a record of (event: ..., result: ...). Every successful run emits one, and every failure is a stream error. The typed helpers below are usually more convenient than listening to it directly.

// Every event, untyped.
queue.listenAll((result) {
  print('${result.event.type} finished with ${result.result}');
});

// Only events of one type, with the event and result typed accordingly.
queue.listenWhere<SendMessage, String>((result) {
  final SendMessage event = result.event;
  final String text = result.result;
});

// A single event. Completes with its result once it has run, or with its EventFailedException
// if it fails permanently. A failure that is going to be retried does not complete it.
final String text = await queue.waitFor<SendMessage, String>(event);

waitFor matches the event by id on the result stream, so call it while the event is still pending, or before adding it. If the event has already finished, or is never added, the future never completes.

Errors

When run throws, the queue emits an EventFailedException on the stream. It carries the event, the original error, its stackTrace and willRetry, which tells whether the queue is going to retry the event. The onError callbacks of listenAll and listenWhere receive it as a typed EventFailedException, together with the failed event and the stack trace:

queue.listenWhere<SendMessage, String>(
  (result) {},
  onError: (SendMessage event, EventFailedException exception, StackTrace stackTrace) {
    final outcome = exception.willRetry ? 'will be retried' : 'gave up';
    print('${event.text} failed with ${exception.error}, $outcome');
  },
);

listenWhere only reports errors for events of the type it listens to. results.listen receives the EventFailedException like any stream error.

An error is emitted for every failed attempt, so an event that is retried twice before it succeeds produces two errors and then one result.

Retries

Give an event a RetryConfig and the queue retries it when run throws:

// Waits delay × retryCount: 1s, 2s, 3s, ...
LinearRetryConfig(maxRetries: 3, delay: const Duration(seconds: 1));

// Waits delay × multiplier ^ retryCount: 2s, 4s, 8s, ...
ExponentialBackoffRetryConfig(maxRetries: 3, delay: const Duration(seconds: 1), multiplier: 2);
  • When an event fails and has retries left, it moves to the retryQueue. Once the delay has passed it is added to the end of the queue again, so other events may run in between.
  • maxRetries is the number of retries after the first attempt. A negative value retries forever; retryCount then stays at 0, so the built-in configs wait minimumDurationForRetry(0), which is zero, between attempts. Use a custom RetryConfig for unlimited retries with a delay.
  • When an event fails with no retries left it is dropped after its last EventFailedException.
  • retryCount lives on the config instance, so share a config between events only if you want them to share a retry budget. canRetry tells whether a retry is left.

To write your own strategy, extend RetryConfig and implement minimumDurationForRetry and serializer. A HydratedQueue uses the serializer to persist the config with its event, so include retryCount; if you never persist events it may simply throw:

final class JitteredRetryConfig extends RetryConfig {
  JitteredRetryConfig({required super.maxRetries});

  @override
  Duration minimumDurationForRetry(int retryCount) {
    return Duration(milliseconds: 500 * retryCount + Random().nextInt(500));
  }

  @override
  JsonSerializer<JitteredRetryConfig> get serializer => JsonSerializer(
    fromJson: (json) => JitteredRetryConfig(maxRetries: json['maxRetries'] as int)
      ..retryCount = json['retryCount'] as int? ?? 0,
    toJson: (config) => {'maxRetries': config.maxRetries, 'retryCount': config.retryCount},
  );
}

Per-failure retry policy and injectable clock

Pass a retryPolicy to Queue or HydratedQueue to decide per failure whether to retry at a given timestamp (for example, honouring an HTTP Retry-After header), fail permanently, or pause the queue (for example, on 401 Unauthorized while refreshing credentials):

final queue = HydratedQueue.store(
  startListenable,
  eventSerializers: [UploadPhoto.serializer],
  store: store,
  retryPolicy: (event, error, attemptCount) {
    if (error is UnauthorizedException) return const RetryDecision.pause();
    if (error is RateLimitException) return RetryDecision.retryAt(error.retryAfter);
    if (error is BadRequestException) return const RetryDecision.fail();
    return null; // Fall back to event.retryConfig.
  },
);
  • RetryDecision.retryAt(nextAttemptAt) places the event in the retryQueue until nextAttemptAt.
  • RetryDecision.fail({bool? retainRecord}) fails the event permanently without further retries. In a store-backed HydratedQueue, omitting retainRecord (null) falls back to HydratedQueue.retainFailedEvents (which defaults to true), while passing true or false explicitly overrides it for that failure. When retained, the QueueRecord is kept in the QueueStore with QueueRecordState.failed instead of being deleted.
  • RetryDecision.pause() calls startListenable.pause() and leaves the event at the front of the currentQueue so it runs again once startListenable.start() is called.
  • Returning null falls back to event.retryConfig.

attemptCount is 1 on the first failure and goes up by 1 on every failure, including one that pauses the queue. A store-backed HydratedQueue persists it as QueueRecord.attemptCount.

Custom failure handling

To change how failures are handled, including how attempts are counted, subclass Queue or HydratedQueue and override handleEventError. It runs synchronously every time an event's run throws. The default implementation does three things:

  1. It adds 1 to the event's entry in the protected attemptCounts map.
  2. It calls resolveRetryDecision, which asks retryPolicy first and then event.retryConfig.
  3. It passes the decision to applyRetryDecision, which schedules the retry, pauses the queue, or fails the event and its dependents, then emits the EventFailedException.

applyRetryDecision also calls the persistence hooks, so a HydratedQueue subclass keeps saving whatever your override decides.

For example, a 401 that pauses the queue while the app refreshes its credentials was rejected, not lost. To keep that pause from using up the retry budget, store the new count only when the decision is not a pause:

final class OutboxQueue extends HydratedQueue {
  OutboxQueue.store(
    super.startListenable, {
    required super.eventSerializers,
    required super.store,
    super.retryPolicy,
  }) : super.store();

  @override
  void handleEventError(Event event, Object error, StackTrace stackTrace) {
    final attemptCount = (attemptCounts[event.id] ?? event.retryConfig?.retryCount ?? 0) + 1;
    final decision = resolveRetryDecision(event, error, attemptCount);
    if (decision is! PauseDecision) attemptCounts[event.id] = attemptCount;
    applyRetryDecision(event, decision, error, stackTrace);
  }
}

After the pause, the retryPolicy was still called with the would-be attemptCount. The persisted QueueRecord.attemptCount keeps its previous value, and if the event fails again after start(), that failure gets the same count it would have had without the pause.

Resolve the decision once per failure. resolveRetryDecision calls retryPolicy and can advance RetryConfig.retryCount, so an override that resolves a decision should apply it itself instead of also calling super.handleEventError.

Pass a QueueClock (such as FakeQueueClock in tests) to drive every retry timer and "is it due?" check deterministically:

final clock = FakeQueueClock(DateTime.utc(2026));
final queue = Queue(startListenable, clock: clock);

clock.advance(const Duration(seconds: 10)); // Fires due retry timers in order.

Querying pending events

Sometimes you need to know what is already queued, for example to avoid queueing the same work twice or to find an event to depend on.

// Everything that has not finished yet: the current queue in run order (including the running
// event), followed by the retry queue in the order its events were added to it.
final Iterable<Event> pending = queue.pendingEvents;

// By id.
final Event? event = queue.findEvent(uploadId);

// By type, optionally narrowed down with a predicate.
final uploads = queue.findEvents<UploadPhoto>();
final upload = queue.findEvents<UploadPhoto>(where: (e) => e.photoId == photoId).firstOrNull;

The lower-level currentQueue (a List<Event>) and retryQueue (a map from id to the event and the time of its next attempt) are also available as read-only views.

Dependent events

Retries move an event to the end of the queue, which breaks ordering between events that belong together. Say UploadPhoto must finish before SharePhoto: if the upload fails and is retried, the share would run first. Declare the dependency when adding the event and the queue keeps them in order:

final upload = UploadPhoto(photoId);
queue.add(upload);
queue.add(SharePhoto(photoId), dependsOn: [upload]);

The rules:

  • A dependent event runs after its dependencies, though not necessarily right after them: independent events may run in between. Dependencies must already be pending when the dependent is added, so they are ahead of it in the queue anyway; the declaration only changes anything when a dependency fails.
  • If a dependency fails and is going to be retried, every event that depends on it, directly or through other dependents, moves to the retry queue with it. When the dependency is retried they are all re-queued in their original order, right behind it. Events that do not depend on it keep running in the meantime.
  • Adding an event that depends on an event which is already waiting for a retry puts it straight into the retry queue, behind the dependency.
  • If a dependency fails permanently, because it has no RetryConfig or no retries left, its dependents do not run. Each fails with an EventFailedException whose error is a DependencyFailedException naming the dependency and the error it failed with. Further dependents down the chain fail the same way, each naming its direct dependency.
  • Events in dependsOn that are not pending (they already finished, or were never added) are ignored, so it is safe to depend on work that may already be done.
  • An event can depend on several events, and dependents can have dependents of their own.

Dependencies are usually discovered with a query:

void sharePhoto(String photoId) {
  final upload = queue.findEvents<UploadPhoto>(where: (e) => e.photoId == photoId).firstOrNull;
  queue.add(SharePhoto(photoId), dependsOn: [if (upload != null) upload]);
}

Handle a failed dependency like any other failure:

queue.listenWhere<SharePhoto, Object?>(
  (result) {},
  onError: (event, exception, stackTrace) {
    if (exception.error case DependencyFailedException(:final dependency, :final error)) {
      print('Not sharing ${event.photoId}: ${dependency.type} failed with $error');
    }
  },
);

HydratedQueue persists dependencies along with the events, so they survive a restart.

Persistence with HydratedQueue

HydratedQueue is a Queue that saves itself every time it changes and can restore itself on the next launch. You provide the storage; it can be anything that stores a string.

Each event class declares its type and an EventSerializer from the same constant, so the name is written once and toJson receives the event already typed:

class UploadPhoto extends Event<String> {
  UploadPhoto(this.photoId, {String? id})
    : id = id ?? '${DateTime.now().microsecondsSinceEpoch}-${_created++}';

  static int _created = 0;

  static const typeName = 'UploadPhoto';

  static final serializer = EventSerializer<UploadPhoto>(
    type: typeName,
    fromJson: (json) => UploadPhoto(json['photoId'] as String, id: json['id'] as String)
      ..retryConfig = linearRetryConfigSerializer.fromJson(
        json['retryConfig'] as Map<String, dynamic>,
      ),
    toJson: (event) => {
      'photoId': event.photoId,
      'id': event.id,
      'retryConfig': linearRetryConfigSerializer.toJson(event.retryConfig!),
    },
  );

  final String photoId;

  @override
  final String id;

  @override
  String get type => typeName;

  @override
  LinearRetryConfig? retryConfig = LinearRetryConfig(
    maxRetries: 3,
    delay: const Duration(seconds: 2),
  );

  @override
  Future<String> run() async => /* upload and return the remote URL */ '';
}

Register the serializers and hook the queue up to your storage:

final prefs = await SharedPreferences.getInstance();

final queue = HydratedQueue(
  startListenable,
  eventSerializers: [UploadPhoto.serializer, SharePhoto.serializer],
  saveCurrentQueue: (data) => prefs.setString('currentQueue', data),
  hydrateCurrentQueue: () => prefs.getString('currentQueue'),
  saveRetryQueue: (data) => prefs.setString('retryQueue', data),
  hydrateRetryQueue: () => prefs.getString('retryQueue'),
);

await queue.hydrate(); // Restore first...
startListenable.start(); // ...then start.
  • Adding an event whose type has no serializer, or whose serializer is for another class, throws an ArgumentError right away rather than silently not persisting it. Two serializers with the same type are rejected when the queue is created.
  • toJson must include everything needed to rebuild the event, including its id and retry configuration. The built-in retry configs ship with serializers (linearRetryConfigSerializer, exponentialBackoffRetryConfigSerializer); custom ones expose theirs through RetryConfig.serializer.
  • Saved events whose type has no serializer any more are dropped by hydrate.
  • The save callbacks may be synchronous or asynchronous. The hydrate callbacks return the last saved string, or null when nothing was saved yet.
  • hydrate restores the current queue, then the retry queue. Retries whose time has already come are placed on the current queue; the others wait for their turn. Dependencies are restored too. Call it once, before start().
  • The built-in retry config serializers include retryCount, so a restored event continues its retry budget and backoff where it left off. Do the same in a custom RetryConfig.serializer.
  • Changes made in the same synchronous run are saved together, and saves are written one at a time, in the order they happened, even when the storage is slow. add returns a future that completes once the change is saved and fails if a save callback throws.

Storage format

The current queue is saved as a JSON array, one object per event, in queue order:

[
  {"photoId": "p1", "id": "e1", "retryConfig": {"maxRetries": 3, "delay": 1000, "retryCount": 0}, "type": "UploadPhoto"},
  {"photoId": "p1", "id": "e2", "type": "SharePhoto", "dependsOn": ["e1"]}
]

Each object is the event's toJson output plus id, type and, when present, dependsOn. The whole object is passed back to fromJson.

The retry queue is saved as a JSON object keyed by event id, in the order the events entered the retry queue:

{
  "e1": {"type": "UploadPhoto", "event": {"photoId": "p1", "id": "e1", "retryConfig": {"maxRetries": 3, "delay": 1000, "retryCount": 1}}, "nextExecutionTime": "2026-10-15T10:00:00.000Z", "awaitingRetry": true},
  "e2": {"type": "SharePhoto", "event": {"photoId": "p1", "id": "e2"}, "nextExecutionTime": "2026-10-15T10:00:00.000Z", "awaitingRetry": false, "dependsOn": ["e1"]}
}

Here only the event object is passed back to fromJson, which is why toJson has to include the id itself. awaitingRetry distinguishes an event waiting for its own retry from one that is only waiting for a dependency.

Per-event persistence and transactional outbox (QueueStore)

For database-backed queues (such as SQLite through drift), use HydratedQueue.store with a QueueStore so each event is persisted as an individual QueueRecord rather than rewriting whole- queue snapshots:

final queue = HydratedQueue.store(
  startListenable,
  eventSerializers: [UploadPhoto.serializer, SharePhoto.serializer],
  store: store,
);
  • Each QueueRecord holds id, type, eventJson, state (pending, scheduled, or failed), dependsOn, attemptCount, nextAttemptAt, awaitingRetry, sequence and error.
  • Changes in the same synchronous run are applied in one store.apply(changes) call (QueueRecordInsert, QueueRecordUpdate, QueueRecordRemove). Override apply in a database store to run the batch inside a transaction so an event never transiently disappears between states.
  • Transactional outbox: build the record with queue.recordFor(event), write your local entity and the QueueRecord inside one database transaction, and call await queue.add(event, persisted: true) after the transaction commits. The event never becomes runnable before the commit, and add does not insert the record a second time. If the transaction rolls back, call queue.discardRecord(event.id):
final event = UploadPhoto(photoId);
final record = queue.recordFor(event);

try {
  await db.transaction(() async {
    await db.photos.insert(photo);
    await db.queueRecords.insert(record.toJson()); // Write the record in the same DB transaction.
  });
} catch (_) {
  queue.discardRecord(event.id);
  rethrow;
}

await queue.add(event, persisted: true);
  • Filtered restore: pass filter to hydrate to restore only one partition (for example, the signed-in account's mutations) in ascending sequence order while leaving other partitions' records untouched in the store:
await queue.hydrate(filter: (record) => record.eventJson['accountId'] == currentAccountId);
  • Terminal failures: when an event exhausts its RetryConfig or retryPolicy returns RetryDecision.fail(), its record is updated to QueueRecordState.failed in the store instead of being deleted (configurable via retainFailedEvents or RetryDecision.fail(retainRecord: false)). hydrate skips failed records on future launches.

Disposing

queue.dispose();

dispose cancels pending retries, closes the result stream, disposes the QueueStartListenable and clears the queue. Adding to or listening to the queue afterwards throws a StateError. A HydratedQueue does not save on dispose, so its persisted events are restored on the next launch.

API overview

Type Purpose
Event<T> A unit of work with an id, a type, an optional retryConfig and a run method.
Queue Runs events in order. add, pendingEvents, findEvent, findEvents, currentQueue, retryQueue, startListenable, clock, retryPolicy, results, listenAll, listenWhere, waitFor, dispose.
HydratedQueue A Queue that persists itself through a QueueStore or snapshot callbacks, prepares transactional records with recordFor / discardRecord, and restores with hydrate.
QueueStore Per-event persistence interface: load, insert, update, remove, maxSequence, apply.
InMemoryQueueStore An in-memory QueueStore with atomic apply batches.
QueueRecord A single persisted event record with state (QueueRecordState), dependsOn, attemptCount, nextAttemptAt and sequence.
QueueChange Sealed change set item (QueueRecordInsert, QueueRecordUpdate, QueueRecordRemove) passed to QueueStore.apply.
RetryPolicy / RetryDecision Per-failure hook returning RetryDecision.retryAt, RetryDecision.fail or RetryDecision.pause. Override Queue.handleEventError, resolveRetryDecision and applyRetryDecision to customize failure handling.
QueueClock / FakeQueueClock Injectable clock for retry timers and due-time checks.
QueueStartListenable Starts and pauses a queue: start, pause, isStarted.
RetryConfig Base class for retry strategies: maxRetries, retryCount, canRetry, retry, minimumDurationForRetry, minimumDurationForCurrentRetry, serializer.
LinearRetryConfig Retries after delay × retryCount.
ExponentialBackoffRetryConfig Retries after delay × multiplier ^ retryCount.
EventSerializer<T> Converts the events of one type to and from JSON for a HydratedQueue.
JsonSerializer<T> A fromJson/toJson pair, used for retry configs.
QueueListenerResult<T, U> The (event, result) record emitted for each successful run.
EventFailedException Emitted on results when an event fails; carries the event, error, stackTrace and willRetry.
DependencyFailedException The error of a dependent that could not run because its dependency failed.
RetryExceededException Thrown by RetryConfig.retry when called with no retries left.

Example

The example is a Flutter app that persists a HydratedQueue with shared_preferences.

License

MIT. See LICENSE.

Libraries

queuekit
A persistent task queue for Dart and Flutter. Runs events in order, retries failures, keeps dependent events in order and restores the queue across app restarts.