flutter_persistent_queue 3.0.0

persistent_queue #

Simple file-based non-volatile queue library for flutter.

Typical use-case scenario is for small to medium sized in-device buffers storing persistent yet temporary mission-critical data, until it can be efficiently and safely consumed / delivered permanently - such as for custom analytics and specialized logging applications.

Installation #

Add dependency to pubspec.yaml:

dependencies:
  ...
  flutter_persistent_queue: ^3.0.0

Run in your terminal:

flutter packages get

Example #

import 'package:flutter_persistent_queue/flutter_persistent_queue.dart';

// instantiate a new queue `pq`, with auto-flush set at 10 items
final pq = PersistentQueue('pq', flushAt: 10, onFlush: (list) async {
  print('auto-flush\n$list');
});

// push 25 Map<<String, dynamic>> items (await is optional)
for (int i = 0; i < 25; ++i) {
  pq.push(<String, dynamic>{'i': i});
}

// trigger a final manual flush for the remaining items
pq.flush((list) async => print('manual-flush\n$list'));

// check if all items have been flushed
assert(await pq.futureLength == 0);

// deallocate resources, if needed
await pq.destroy();

Testing #

For integration / performance tests:

git clone https://github.com/oakromulo/flutter_persistent_queue.git
cd ~/flutter_persistent_queue/test
flutter drive --target=test_driver/app.dart

Building documentation files #

rm -rf doc
dartdoc --exclude 'dart:async,dart:collection,dart:convert,dart:core,dart:developer,dart:io,dart:isolate,dart:math,dart:typed_data,dart:ui'

How it works #

Each JSON-encodable item when enqueued goes to its own indexed key in a single non-volatile file (one file per uniquely named queue) to an unspecified location on the permanent storage of a flutter-compatible device.

This particular design choice limits potential use cases requiring very long queues but otherwise provides good performance within a limited resource footprint, as it doesn't require serializing and deserializing contiguous or chunked dart:collections to the filesystem.

Every push() call triggers an asynchronous and isolated write to the permanent storage. If the caller awaits until its completion, it is notified via Completers and afterwards there's no possibility of data loss from the application perspective.

After user-defined flush() operations (either manual or auto-triggered) the queue is fully emptied if and only if the user-defined onFlush does not throw any errors.

All PersistentQueue methods are consumed sequentially (according to the strict chronological order of its calls) one at a time on a dedicated producer / consumer event loop.

Performance of persisted queues on standard simulated devices must be of at least 500 writes/second. This constraint is verified on integration tests.

All filesystem and JSON operations are abstracted by the minimalistic flutter_localstorage library.

License #

MIT

[3.0.0] - 2019-05-22

  • breaking change: OnFlush handlers must now return a boolean "ack" instead of void

[2.0.1+2] - 2019-05-21

  • minor style fix

[2.0.1+1] - 2019-05-21

  • rebuilt docs

[2.0.1] - 2019-05-21

  • alias feature added
  • cleaner example and tests
  • updated localstorage to v2.0.0
  • breaking change: length() method now returns Future<int> to avoid ambiguity and edgy race conditions

[1.0.2] - 2019-05-09

  • updated documentation

[1.0.1] - 2019-05-09

  • minor source code whitespace fix

[1.0.0] - 2019-05-09

  • new buffered implementation to enforce sequential behavior
  • integration tests added, including throughput performance benchmarking

[0.1.4] - 2019-02-27

[0.1.3] - 2019-02-27

  • silent flush/push error handling if user does not provide handlers
  • hard maxLength parameter added to enforce an absolute maximum queue length

[0.1.2] - 2019-02-26

  • fix error message and add stack trace to error handling

[0.1.1] - 2019-02-25

  • user-specified error handling behavior while flushing

[0.1.0] - 2019-02-21

  • minor source code readability improvements
  • no-reload feature added to setup method
  • improved example using FutureBuilder
  • no additional example dependencies

[0.0.1] - 2019-02-20

  • Initial release

example/lib/main.dart

// ignore_for_file: unawaited_futures, public_member_api_docs
import 'package:flutter/material.dart';
import 'package:flutter_persistent_queue/flutter_persistent_queue.dart';

Future<List<String>> example() async {
  final persistedValues = <String>{};

  const int flushAt = 12;

  int autoFlushCnt = 0, pushCnt = 0;

  List<String> strListFromMapList(List<Map<String, dynamic>> list) =>
      list.map<String>((v) => '${v['μs']}').toList();

  // instantiate queue and define implicit flush to fill [persistedValues]
  final pq = PersistentQueue('pq', flushAt: flushAt, onFlush: (list) async {
    persistedValues.addAll(strListFromMapList(list));

    autoFlushCnt += list.length;
    print('# of FLUSHED elements: ${list.length}');
  });

  // register # of persisted items from previous run
  final initCnt = await pq.length;

  // preview old elements before adding new ones, without dequeueing
  persistedValues.addAll(strListFromMapList(await pq.toList(growable: false)));

  // enqueue a random-ish amount of values, without awaiting
  for (int i = 0; i < 36; ++i) {
    final int microseconds = DateTime.now().microsecondsSinceEpoch;

    if (microseconds % 10 > 0) {
      continue;
    }

    pq.push(<String, dynamic>{'μs': '$microseconds'});
    ++pushCnt;
  }

  print('''\t
    items reloaded from previous run: $initCnt
    items left due to persist until next run: ${await pq.length}
    items read from the queue: ${persistedValues.length}
    items flushed from the queue: $autoFlushCnt
    items written to the queue: $pushCnt
    queue configured to flush at: $flushAt
  ''');

  await pq.destroy();
  print('the queue is gone');

  return persistedValues.toList(growable: false)..sort();
}

void main() => runApp(ExampleApp());

class ExampleApp extends StatelessWidget {
  @override
  Widget build(_) => FutureBuilder(future: example(), builder: app);
}

Widget app(BuildContext context, AsyncSnapshot<dynamic> snapshot) =>
    MaterialApp(home: home(snapshot));

Widget home(AsyncSnapshot<dynamic> snapshot) =>
    Scaffold(appBar: AppBar(title: Text('Example')), body: body(snapshot));

Widget body(AsyncSnapshot<dynamic> snapshot) {
  if (snapshot.data == null) {
    return Center(child: CircularProgressIndicator());
  }

  final children = (snapshot.data as List<String>)
      .map<Widget>((s) => ListTile(leading: Icon(Icons.grade), title: Text(s)))
      .toList();

  return ListView(children: children);
}

Use this package as a library

1. Depend on it

Add this to your package's pubspec.yaml file:


dependencies:
  flutter_persistent_queue: ^3.0.0

2. Install it

You can install packages from the command line:

with Flutter:


$ flutter pub get

Alternatively, your editor might support flutter pub get. Check the docs for your editor to learn more.

3. Import it

Now in your Dart code, you can use:


import 'package:flutter_persistent_queue/flutter_persistent_queue.dart';
  
Popularity:
Describes how popular the package is relative to other packages. [more]
46
Health:
Code health derived from static analysis. [more]
100
Maintenance:
Reflects how tidy and up-to-date the package is. [more]
100
Overall:
Weighted score of the above. [more]
73
Learn more about scoring.

We analyzed this package on Jul 17, 2019, and provided a score, details, and suggestions below. Analysis was completed with status completed using:

  • Dart: 2.4.0
  • pana: 0.12.19
  • Flutter: 1.7.8+hotfix.3

Platforms

Detected platforms: Flutter

References Flutter, and has no conflicting libraries.

Dependencies

Package Constraint Resolved Available
Direct dependencies
Dart SDK >=2.2.0 <3.0.0
flutter 0.0.0
localstorage ^2.0.0 2.0.0
Transitive dependencies
collection 1.14.11
meta 1.1.6 1.1.7
path_provider 1.1.2
sky_engine 0.0.99
typed_data 1.1.6
vector_math 2.0.8
Dev dependencies
flutter_driver
test any

Admin