InterpreterPool class

A round-robin pool of Interpreter instances with per-slot serialization locks.

Each slot has its own Future-chain lock so concurrent callers are serialized per-interpreter rather than globally, enabling parallel inference across multiple pool slots while preventing XNNPACK thread contention.

Usage:

final pool = InterpreterPool(poolSize: 3);
await pool.initialize(
  (options, _) async {
    final interp = await Interpreter.fromAsset('model.tflite', options: options);
    interp.resizeInputTensor(0, [1, 224, 224, 3]);
    interp.allocateTensors();
    return interp;
  },
  performanceConfig: PerformanceConfig(),
);

final result = await pool.withInterpreter((interp, iso) async {
  // run inference ...
});

await pool.dispose();

Constructors

InterpreterPool({int poolSize = 1})

Properties

hashCode → int
The hash code for this object.
no setterinherited
interpreters → List<Interpreter>
All interpreters in the pool, in creation order.
no setter
isInitialized → bool
no setter
poolSize → int
final
runtimeType → Type
A representation of the runtime type of the object.
no setterinherited

Methods

dispose() → Future<void>
Disposes all interpreters, isolates, and delegates.
initialize(InterpreterCreator factory, {PerformanceConfig? performanceConfig, bool useIsolateInterpreter = true}) → Future<void>
Initializes the pool by calling factory once per slot.
noSuchMethod(Invocation invocation) → dynamic
Invoked when a nonexistent method or property is accessed.
inherited
toString() → String
A string representation of this object.
inherited
withInterpreter<T>(Future<T> fn(Interpreter, IsolateInterpreter?)) → Future<T>
Runs fn with exclusive access to one interpreter, selected round-robin.

Operators

operator ==(Object other) → bool
The equality operator.
inherited