initializeFromBuffers method

Future<void> initializeFromBuffers({ required Uint8List bodyDetectorBytes, required Uint8List classifierBytes, required String speciesMappingJson, Uint8List? poseModelBytes, bool useIsolateInterpreter = true, bool useCompiledModel = false, bool compiledForceCpu = false, Set<Accelerator> accelerators = const {Accelerator.gpu, Accelerator.cpu}, Precision precision = Precision.fp32, })

Initializes the detector from pre-loaded model bytes.

Used for initialization within a background isolate where Flutter asset loading is not available. When useCompiledModel is true, every stage first attempts the LiteRT Next CompiledModel backend. The option is off by default, matching face, pose, hand, and object detection. accelerators and precision use the same public configuration shape as those packages.

Every compiled graph is checked with verifyCompiledModel before it is trusted. A numerically incorrect GPU partition is retried on CompiledModel CPU; if CPU also fails verification, only that stage falls back to the classic Interpreter. This is required because LiteRT can report success while returning incorrect outputs for some of these models. compiledForceCpu is retained for source compatibility and pins the requested CompiledModel backend to CPU.

Whether CompiledModel is faster is per-platform and per-model, so measure before shipping it on: its CPU accelerator beats the Interpreter's CPU/XNNPACK path on Apple Silicon macOS but is roughly 2x slower on iOS. See flutter_litert's test/benchmark/RESULTS.md.

Implementation

Future<void> initializeFromBuffers({
  required Uint8List bodyDetectorBytes,
  required Uint8List classifierBytes,
  required String speciesMappingJson,
  Uint8List? poseModelBytes,
  bool useIsolateInterpreter = true,
  bool useCompiledModel = false,
  bool compiledForceCpu = false,
  Set<Accelerator> accelerators = const {
    Accelerator.gpu,
    Accelerator.cpu,
  },
  Precision precision = Precision.fp32,
}) async {
  if (_isInitialized) {
    await dispose();
  }

  _bodyDetector = AnimalBodyDetector();
  if (useCompiledModel) {
    try {
      await _bodyDetector!.initCompiledFromBuffer(
        bodyDetectorBytes,
        forceCpu: compiledForceCpu,
        accelerators: accelerators,
        precision: precision,
      );
    } catch (error) {
      debugPrint(
        'Animal body detector CompiledModel rejected; using Interpreter: '
        '$error',
      );
      _bodyDetector!.dispose();
      _bodyDetector = AnimalBodyDetector();
      await _bodyDetector!.initializeFromBuffer(
        bodyDetectorBytes,
        performanceConfig,
        useIsolateInterpreter: useIsolateInterpreter,
      );
    }
  } else {
    await _bodyDetector!.initializeFromBuffer(
      bodyDetectorBytes,
      performanceConfig,
      useIsolateInterpreter: useIsolateInterpreter,
    );
  }

  _classifier = SpeciesClassifier();
  if (useCompiledModel) {
    try {
      await _classifier!.initCompiledFromBufferWithMapping(
        classifierBytes,
        speciesMappingJson,
        forceCpu: compiledForceCpu,
        accelerators: accelerators,
        precision: precision,
      );
    } catch (error) {
      debugPrint(
        'Animal species classifier CompiledModel rejected; using '
        'Interpreter: $error',
      );
      _classifier!.dispose();
      _classifier = SpeciesClassifier();
      await _classifier!.initializeFromBuffer(
        classifierBytes,
        speciesMappingJson,
        performanceConfig,
        useIsolateInterpreter: useIsolateInterpreter,
      );
    }
  } else {
    await _classifier!.initializeFromBuffer(
      classifierBytes,
      speciesMappingJson,
      performanceConfig,
      useIsolateInterpreter: useIsolateInterpreter,
    );
  }

  if (enablePose && poseModelBytes != null) {
    _poseEstimator = BodyPoseEstimator(model: poseModel);
    if (useCompiledModel) {
      try {
        await _poseEstimator!.initCompiledFromBuffer(
          poseModelBytes,
          forceCpu: compiledForceCpu,
          accelerators: accelerators,
          precision: precision,
        );
      } catch (error) {
        debugPrint(
          'Animal pose CompiledModel rejected; using Interpreter: $error',
        );
        _poseEstimator!.dispose();
        _poseEstimator = BodyPoseEstimator(model: poseModel);
        await _poseEstimator!.initializeFromBuffer(
          poseModelBytes,
          effectivePoseConfig,
          useIsolateInterpreter: useIsolateInterpreter,
        );
      }
    } else {
      await _poseEstimator!.initializeFromBuffer(
        poseModelBytes,
        effectivePoseConfig,
        useIsolateInterpreter: useIsolateInterpreter,
      );
    }
  }

  _isInitialized = true;
}