squadron 6.0.4

Multithreading and worker thread pool for Dart / Flutter, to offload CPU-bound and heavy I/O tasks to Isolate or Web Worker threads.

Squadron - Multithreading and worker pools in Dart Offload CPU-bound and long running tasks and give your apps some air! Works everywhere: desktop, server, device, browser. Supports native, JavaScript & Web Assembly platforms.

View latest documentation on GitHub

Getting Started

1. Update your pubspec.yaml file to add dependencies to Squadron and squadron_builder + build_runner:

dependencies:
  squadron: ^6.0.0
  # ...

dev_dependencies:
  build_runner:
  squadron_builder: ^6.0.0
  # ...

2. Have dart download and install the dependencies:

dart pub get

Implementing a Service

Create a class containing the code you intend to run in a dedicated thread and make sure you provide squadron annotations:

• use SquadronService for the class;

• use SquadronMethod for the methods you want to expose.

Service methods must return a Future<T>, a FutureOr<T> or a Stream<T>.

// file hello_world.dart
import 'dart:async';

import 'package:squadron/squadron.dart';

import 'hello_world.activator.g.dart';
part 'hello_world.worker.g.dart';

@SquadronService(baseUrl: '~/workers', targetPlatform: TargetPlatform.vm | TargetPlatform.web)
// or @SquadronService(baseUrl: '~/workers', targetPlatform: TargetPlatform.all)
base class HelloWorld {
  @SquadronMethod()
  FutureOr<String> hello([String? name]) {
    name = name?.trim() ?? 'World';
    return 'Hello, $name!';
  }
}

Generating the Worker and WorkerPool code

Have squadron_builder generate the code with the following command line:

dart run build_runner build

This command will create the worker and worker pool from your service: HelloWorldWorker and HelloWorldWorkerPool.

Workers and worker pools generated by squadron_builder implement the same interface as the original service and proxy all method calls to an instance of the service running in its own thread.

Spawning a Worker

In your program, you can instantiate a Worker (or a WorkerPool if you need more threads) and use it just as you would use your original service.

Make sure you stop the workers and pools before exiting your program. Failure to do so will let your program run forever.

// file main.dart
import 'package:squadron/squadron.dart';

import 'hello_world.dart';

void main() async {
  final worker = HelloWorldWorker();
  try {
    // Squadron will start the worker for you so you don't have to call worker.start()
    final message = await worker.hello();
    print(message);
  } finally {
    // make sure the worker is stopped when the program terminates
    worker.stop();
  }
}

Building for the Web

You must compile your code to JavaScript or Web Assembly if your app is designed to run in a browser.

dart compile js ".\src\lib\hello_world.web.g.dart" -o "..\web\workers\hello_world.web.g.dart.js"
dart compile wasm ".\src\lib\hello_world.web.g.dart" -o "..\web\workers\hello_world.web.g.dart.wasm"

When compiling to only one of Javascript or Web Assembly, you must make sure your service @SquadronService() annotation only references the corresponding TargetPlatform.js or TargetPlatform.wasm.

You can also compile for both targets: at runtime, Squadron will use the workers matching your app's platform. In that case, make sure your service annotation targets platforms TargetPlatform.js | TargetPlatform.wasm or shortcut TargetPlatform.web.

Multithreading Constraints

There are a few constraints to multithreading in Dart:

• Dart threads do not share memory: values passed from one side to the other will typically be cloned. Depending on the implementation, this can impact performance.

• Service methods arguments and return values need to cross thread-boundaries: on Web platforms, the Dart runtime delegates this to the browser which is not aware of Dart's type-system. Extra-work is necessary to recover strongly-typed data on the receiving-end.

Data sent through Squadron are handled as dynamic types: to recover strong types and guarantee type-safety in your code, Squadron provides Converters to "convert" data on the receiving-end:

• native platforms use a DirectCastConverter that simply casts data;

• on Web platforms, objects sent to/from a Web Worker leave Dart's realm when they go through the browser's postMessage() function, losing their Dart type in the process. They must therefore re-enter Dart's type-system on the receiving end. Squadron provides a CastConverter (converting data as well as items in List/Set/Map objects) and a NumConverter (adding special handling for int/double values) depending on the underlying runtime (JavaScript or Web Assembly).

Native Platforms

On native platforms, it is generally safe to not bother about custom types and cloning. The Dart VM will take care of copying data when necessary, optimize data-transfer when possible (eg. Strings do not require copying), and object types are retained.

There are a few constraints on what type of data can be transferred, please refer to SendPort.send() documentation for more information.

On native platforms, Squadron uses a default DirectCastConverter that simply casts data on the receiving end.

Web Platforms

Web platforms have stronger constraints when it comes to transferable objects: for more information, please refer to Transferable objects documentation or the HTML specification for transferable objects. There may also be differences between browser flavors and versions.

On Web plaforms, Squadron uses a default CastConverter (JavaScript runtime) or NumConverter (Web Assembly runtime). One of the key differences between Dart, Web Assembly and JavaScript is number handling: JavaScript only really knows double numbers whereas Dart and Web Assembly support int and double as different types. As a result, on JavaScript platforms, Dart's int is actually a subtype of double and special care is required when transfering numbers: on Web Assembly platforms, the receiving-end may receive int values as double and require a conversion back to int.

More importantly, custom-types will require marshaling so they can be transferred across worker boundaries. Squadron is not too opinionated and there are various ways to achieve this: eg. using JSON (together with json_serializer for instance), by implementing marshal()/unmarshal() or toJson()/fromJson() methods in your data classes, or by using Squadron marshalers.

For instance, to transfer a Dart BigInt instance:

class BigIntMarshaler implements GenericMarshaler<BigInt> {
  const BigIntMarshaler();

  @override dynamic marshal(BigInt data) => data.toString();

  @override BigInt unmarshal(dynamic data) => BigInt.parse(data);
} 

Apply the marshaler by annotating BigInt parameters and return values:

@SquadronService(baseUrl: '~/workers', targetPlatform: TargetPlatform.web)
base class BigIntService {
  @SquadronMethod()
  @BigIntMarshaler()
  FutureOr<BigInt> add(@BigIntMarshaler() BigInt a, @BigIntMarshaler() BigInt b)
    => a + b;
}

squadron_builder will implement proper conversion in and out when generating the code for BigIntServiceWorker.

Optimizing Conversions

As stated in a previous paragraph, code designed to run only on native platforms should not worry about data conversion. Because Squadron native workers share the same code and execute in Isolates running in the same Dart VM, they never leave Dart's type-system. Data sent through Squadron is promoted from dynamic back to strong-types by simple cast operations.

On Web platforms, things are different because the data was handed over to the browser which down't know anything about Dart types:

• bool and String: casting is enough to re-enter Dart's type system (handled by CastConverter).

• int and double: integers may be received as floating points numbers; in JavaScript runtimes, intis a subtype of double and casting is enough (handled by CastConverter); in Web Assembly runtimes, integer values may be received as a double and require conversion back to int (handled by NumConverter).

• List and Map: these objects are received as List<dynamic> and Map<dynamic, dynamic> and item, key and value types are systematically lost. Type-casting is not enough and would always fail with a TypeError so additional processing is required from the converter.

• Set: these objects are received as List<dynamic>, converted to a List<T> using the converter, then transformed into a Set<T> by calling list.toSet().

To handle List and Map objects as efficiencly as possible, converters provided by Squadron optimize the process when the item type is a base type that can be handled by a simple cast. Eg. when a service method works with a List<String>, it is received/sent as a List<dynamic> and will be "promoted" back to List<String> by simply calling list.cast<String>(). For Map<K, V> objects where K and V are base types, the received Map<dynamic, dynamic> will be cast back to Map<K, V> with map.cast<K, V>(). In these scenarios, cast operations are deferred until an item is accessed. Dart's static type-safety checks guarantee the cast will succeed.

When lists and maps contain elements that cannot be cast, additional processing is required. For instance, a List<int> object sent to a Web Assembly worker will be received as a List<dynamic> containing double elements! Because int is not a subtype of double on Web Assembly runtimes, list.cast<int>() cannot be used.

Under such circumstances, list elements must be processed individually and converted back; eg. NumConverter handles this specific example as list.map(_toInt).toList() where _toInt is a function that returns the input value as an int after checking it is effectively an int or an integral double.

For large collections or complex structures (nested lists/maps), this process may impact performance because 1/ map() will iterate over all elements and 2/ toList() will create a fresh list to hold the converted elements.

It can be optimized in various ways:

• using Int32List and other typed data: under the hood, Squadron converters works with the underlying ByteBuffer which guarantees type-safety and hopefully comes with efficient cloning.

• alternatively, it is possible to assign the ambiant converter Squadron.converter with a specialized/optimized converter. Squadron provides two additional converters that could be useful:

  • InPlaceConverter: this implementation wraps around original List<dynamic> and Map<dynamic, dynamic> instances to avoid creating new list/map instances. dynamic items are immediately converted to their strong-type and stored in the original, dynamic list/map. Read operations will simply cast items when accessed.

  • LazyInPlaceConverter: this implementation also wraps around original List<dynamic> and Map<dynamic, dynamic> instances to avoid creating new instances. Conversion is deferred to when items are effectively accessed: the first read will convert the element and store it in the original instance; subsequent reads will find the element has already been converted and will simply cast and return it.

To activate one of these converters, the ambiant converter can be assigned like so:

Squadron.converter = LazyInPlaceConverter(Squadron.converter);

The original, default converter should be provided to guarantee proper handling of int.

Marshaling

Converters only take care of base types (strings, numbers, booleans, lists, maps and sets as well as Dart's typed data). The default behavior for other types (whether they're Dart types such as BigInt or Duration, or custom types that you or a third-party package implemented) is to simply cast the dynamic value to the specified type.

But this will only work on native Dart VM. On browser platforms, custom objects must be serialized when sent and deserialized when received. Squadron provides SquadronMarshaler<T, S> for you to implement your own marshaler:

• S marshal(T data): implement this method to serialize an instance of T to something that can be transfered, for instance a List;

• T unmarshal(S data): implement this method to deserialize from S and back to T.

unmarshal(marchal(obj)) should produce an instance of T that is functionaly equivalent to the original instance obj.

For instance, given the following class:

class Car {
  Car(this.color, this.price, this.engine);

  final String color;
  final double price;
  final Engine engine;
}

enum Engine { gaz, diesel, electric }

A marshaler could be implemented as:

class CarMarshaler implements SquadronMarshaler<Car, List> {
  const CarMarshaler();

  List marshal(Car data) =>
      [
        data.color, // color at index 0
        data.price, // price at index 1
        data.engine.index, // engine at index 2
      ];

  Car unmarshal(List data) =>
      Car(
        data[0], // index 0
        data[1], // index 1
        Engine.values.singleWhere((e) => e.index == data[2]), // index 2
      );
}

// for use as an annotation
const carMarshaler = CarMarshaler();

squadron_builder will use the marshaler based on annotations provided in your service implementation:

@SquadronService()
class CarService {
  @serviceMethod
  @carMarshaler
  FutureOr<Car?> buy(double cash, String color) { /* ... */ }

  @serviceMethod
  FutureOr<double> sell(@carMarshaler Car car) { /* ... */ }
}

Alternatively, if you own the target class, you can also simply annotate it:

@carMarshaler
class Car {
   // ...
}

@SquadronService()
class CarService {
  @serviceMethod
  FutureOr<Car> buy(double cash, String color) { /* ... */ }

  @serviceMethod
  FutureOr<double> sell(Car car) { /* ... */ }
}

If your application is designed to run both on native and Web platforms, it is possible to optimize for VM platforms by providing different marshalers depending on the platform and conditionally import the proper implementation.

///////////// file car_marshaler.web.dart /////////////
class _CarMarshaler implements SquadronMarshaler<Car, List> {
  const CarMarshaler();

  List marshal(Car data) => [ /* fields */ ];

  Car unmarshal(List data) => Car(/* arguments */);
}

// for use as an annotation
const carMarshaler = _CarMarshaler();


///////////// file car_marshaler.vm.dart /////////////

// for use as an annotation
// IdentityMarshalers do nothing :)
const carMarshaler = IdentityMarshaler<Car>();


///////////// file car.dart /////////////

import 'car_marshaler.vm.dart'
  if (dart.library.js_interop) 'car_marshaler.web.dart';

@carMarshaler
class Car {
  // ...
}

Thanks!

• Saad Ardati for his patience and feedback when implementing Squadron into his Flutter application.
• Martin Fink for the feedback on Squadron's first Stream implementation -- this has resulted in huge progress and a major improvement.
• Klemen Tusar for providing a sample Chopper JSON decoder leveraging Squadron.
• James O'Leary for sponsorship and contribution, very much appreciated.