onnxruntime 1.4.1

Flutter plugin for OnnxRuntime provides an easy, flexible, and fast Dart API to integrate Onnx models in flutter apps across mobile and desktop platforms.

OnnxRuntime Plugin

Overview

Flutter plugin for OnnxRuntime via dart:ffi provides an easy, flexible, and fast Dart API to integrate Onnx models in flutter apps across mobile and desktop platforms.

Platform	Android	iOS	Linux	macOS	Windows
Compatibility	API level 21+	*	*	*	*
Architecture	arm32/arm64	*	*	*	*

*: Consistent with Flutter

Key Features

• Multi-platform Support for Android, iOS, Linux, macOS, Windows, and Web(Coming soon).
• Flexibility to use any Onnx Model.
• Acceleration using multi-threading.
• Similar structure as OnnxRuntime Java and C# API.
• Inference speed is not slower than native Android/iOS Apps built using the Java/Objective-C API.
• Run inference in different isolates to prevent jank in UI thread.

Getting Started

In your flutter project add the dependency:

dependencies:
  ...
  onnxruntime: x.y.z

copied to clipboard

Usage example

Import

import 'package:onnxruntime/onnxruntime.dart';

copied to clipboard

Initializing environment

OrtEnv.instance.init();

copied to clipboard

Creating the Session

final sessionOptions = OrtSessionOptions();
const assetFileName = 'assets/models/test.onnx';
final rawAssetFile = await rootBundle.load(assetFileName);
final bytes = rawAssetFile.buffer.asUint8List();
final session = OrtSession.fromBuffer(bytes, sessionOptions!);

copied to clipboard

Performing inference

final shape = [1, 2, 3];
final inputOrt = OrtValueTensor.createTensorWithDataList(data, shape);
final inputs = {'input': inputOrt};
final runOptions = OrtRunOptions();
final outputs = await _session?.runAsync(runOptions, inputs);
inputOrt.release();
runOptions.release();
outputs?.forEach((element) {
  element?.release();
});

copied to clipboard

Releasing environment

OrtEnv.instance.release();

copied to clipboard