stat_ort_plugin

On-device speech recognition (Vaani ASR) for Flutter via dart:ffi, powered by ONNX Runtime. It provides both file transcription and real-time streaming transcription, with Silero VAD based segmentation and optional speaker diarization.

Status: early release (0.0.4). The native pipeline works; the API may still change.

Why this package

  • On-device. Audio is never sent off the device and no network connection is required at transcription time.
  • Vaani ASR. Uses the Vaani models from IISc Bangalore, trained for Indian languages.
  • dart:ffi, not method channels. Audio frames are passed to the native pipeline through direct FFI calls, avoiding per-call platform-channel serialization on the streaming hot path.
  • File and streaming in one package. Streaming segments by voice activity (Silero VAD) and can label segments by speaker (diarization); the same pipeline backs both.
  • Models are supplied at runtime, not bundled. The published package stays small, and you control which model files and versions you ship.
  • No committed binaries. ONNX Runtime is resolved through the Android Gradle dependency and the iOS CocoaPod.

Features

  • 🎙️ Streaming transcription from a microphone PCM stream, segmented by voice activity.
  • 📁 File transcription for 16 kHz mono int16 WAV files.
  • 🧑‍🤝‍🧑 Optional speaker diarization (speaker-labelled segments).
  • ⚡ Native C pipeline; heavy work runs off the UI isolate.

Supported platforms

Platform Status ONNX Runtime source
Android ✅ com.microsoft.onnxruntime:onnxruntime-android (Gradle)
iOS ✅ onnxruntime-c (CocoaPods)

macOS / Windows / Linux are not currently supported.

Models

This package does not ship any models. You supply ONNX model files and a vocabulary at runtime:

Audio must be 16 kHz, mono, 16-bit PCM. See NOTICE.md for model licensing. The example app loads models from its assets and copies them to a temp directory; for a real app, download them on first launch rather than bundling hundreds of MB into your binary.

Installation

No extra setup is needed: ONNX Runtime is pulled in automatically by the Android Gradle dependency and the iOS CocoaPod.

Usage

File transcription

import 'package:stat_ort_plugin/stat_ort_plugin.dart';

final vaani = await Vaani.create(
  encoderPath, decoderPath, vocabPath, 4, // encoderThreads
  vadPath: vadPath,
  speakerPath: speakerPath,
);

final transcript = await vaani.transcribe(wavPath);
print(transcript);

vaani.dispose();

Streaming transcription

Feed raw 16 kHz mono int16 PCM as it arrives (e.g. from the record package). The plugin buffers internally into the 512-sample frames Silero VAD requires.

final stream = vaani.createStream();

await for (final Uint8List chunk in micPcmStream) {
  final segment = stream.pushChunk(chunk.buffer.asInt16List());
  if (segment != null) print(segment); // a finalised "[mm:ss - mm:ss] [Speaker N]: ..." line
}

// IMPORTANT: flush before closing to get the trailing segment.
final tail = stream.finish();
if (tail != null) print(tail);
stream.close();

A complete example (mic streaming + file transcription) is in example/.

Threading notes

  • Vaani.create and transcribe run the native work on a background isolate.
  • A single Vaani (pipeline) may back multiple VaaniStreams; VAD state is held per-stream, so independent streams don't interfere. Do not, however, drive the same pipeline's inference from multiple threads concurrently without your own serialization.
  • vaani_pipeline_init resolves the ONNX Runtime API once and is safe to call from multiple isolates.

Debug logging

The native code logs errors via LOGE always, and verbose progress via LOGD only when built with VAANI_DEBUG defined (CMake: -DVAANI_DEBUG=ON). Release builds carry no verbose-logging overhead.

License

MIT — see LICENSE. Third-party dependencies and models are listed in NOTICE.md; verify model licenses for your use case before shipping.

Libraries

stat_ort_plugin
On-device Vaani ASR for Flutter via dart:ffi.