flutter_wake_word

A Flutter plugin for custom wake word detection on iOS and Android.

Use it to create one or more wake word detector instances, start and stop keyword detection, receive detection events, configure VAD behavior, manage recording helpers, and use speaker verification where supported by the native platform.

Contact: [e-mail]

Installation

Add the package to pubspec.yaml:

dependencies:
  flutter_wake_word: ^0.0.39

Then run:

flutter pub get

Android Setup

The Android package includes a local native AAR. Add the plugin libs repository to your app's Android Gradle configuration if your app does not already resolve it automatically:

allprojects {
    repositories {
        google()
        mavenCentral()
        maven { url "${project(':flutter_wake_word').projectDir}/libs" }
        mavenLocal()
    }
}

Import

import 'package:flutter_wake_word/flutter_wake_word.dart';

Quick Start

final wakeWord = FlutterWakeWord();

await wakeWord.setKeywordDetectionLicense('YOUR_LICENSE');

await wakeWord.createInstance(
  'main',
  'hey_app.onnx',
  0.7,
  3,
);

await wakeWord.startKeywordDetection(0.7);

// Stop later.
await wakeWord.stopKeywordDetection();

For direct instance control:

final detector = createKeyWordFlutterPCInstance('main');

await detector.createInstance('hey_app.onnx', 0.7, 3);
await detector.setKeywordDetectionLicense('YOUR_LICENSE');

final subscription = detector.onKeywordDetectionEvent().listen((event) {
  final phrase = event['phrase'];
  final model = event['modelName'] ?? event['model'];
});

await detector.startKeywordDetection('main', 0.7);

Multi-Model Detection

final detector = createKeyWordFlutterPCInstance('multi');

await detector.createInstanceMulti(
  'multi',
  ['hey_app.onnx', 'help_now.onnx'],
  [0.7, 0.8],
  [3, 3],
  [1000, 1000],
);

detector.onKeywordDetectionEvent().listen((event) {
  final phrase = event['phrase'];
  final model = event['modelName'] ?? event['model'];
});

await detector.startKeywordDetection('multi', 0.7);

UseModel Helper

UseModel manages multiple wake word instances for you.

final model = UseModel();

await model.setKeywordDetectionLicense('YOUR_LICENSE');

await model.loadModel(
  [
    InstanceConfig(
      id: 'main',
      modelName: 'hey_app.onnx',
      threshold: 0.7,
      bufferCnt: 3,
    ),
  ],
  (event) {
    final phrase = event['phrase'];
  },
);

await model.startListening();
await model.stopListening();

Wake Word API

FlutterWakeWord

High-level API:

FlutterWakeWord.getPlatformVersion()
FlutterWakeWord.setKeywordDetectionLicense(String license)
FlutterWakeWord.createInstance(...)
FlutterWakeWord.createInstanceMulti(...)
FlutterWakeWord.startKeywordDetection(double threshold)
FlutterWakeWord.stopKeywordDetection()
FlutterWakeWord.getVoiceProps(String instanceId)
FlutterWakeWord.startSilentVADDetection(String instanceId)
FlutterWakeWord.stopSilentVADDetection(String instanceId)
FlutterWakeWord.startVADDetection(String instanceId)
FlutterWakeWord.stopVADDetection(String instanceId)
FlutterWakeWord.onVADDetectionEvent(String instanceId)
FlutterWakeWord.setVADParams(String instanceId, double threshold, int msWindow)
FlutterWakeWord.getVADParams(String instanceId)
FlutterWakeWord.setKeywordLicense(String instanceId, String license)
FlutterWakeWord.getRecordingWavArray(String instanceId)
FlutterWakeWord.getRecordingWav(String instanceId)
FlutterWakeWord.pauseDetection(String instanceId, {bool stopMic = false})
FlutterWakeWord.unPauseDetection(String instanceId)
FlutterWakeWord.removeAllRNBridgeListeners()
FlutterWakeWord.setWakewordAudioRoutingConfig(AudioRoutingConfig config)

KeyWordFlutterPC

Lower-level per-instance API:

createKeyWordFlutterPCInstance(String instanceId, {bool isSticky = false})

KeyWordFlutterPC.createInstance(...)
KeyWordFlutterPC.createInstanceMulti(...)
KeyWordFlutterPC.setKeywordDetectionLicense(String license)
KeyWordFlutterPC.setKeywordLicense(String license)
KeyWordFlutterPC.replaceKeywordDetectionModel(...)
KeyWordFlutterPC.startKeywordDetection(...)
KeyWordFlutterPC.stopKeywordDetection(...)
KeyWordFlutterPC.destroyInstance()
KeyWordFlutterPC.startForegroundService()
KeyWordFlutterPC.stopForegroundService()
KeyWordFlutterPC.onKeywordDetectionEvent()
KeyWordFlutterPC.getVoiceProps()
KeyWordFlutterPC.startSilentVADDetection()
KeyWordFlutterPC.stopSilentVADDetection()
KeyWordFlutterPC.startVADDetection()
KeyWordFlutterPC.stopVADDetection()
KeyWordFlutterPC.onVADDetectionEvent()
KeyWordFlutterPC.setVADParams(double threshold, int msWindow)
KeyWordFlutterPC.getVADParams()
KeyWordFlutterPC.getRecordingWavArray()
KeyWordFlutterPC.getRecordingWav()
KeyWordFlutterPC.pauseDetection({bool stopMic = false})
KeyWordFlutterPC.unPauseDetection()
KeyWordFlutterPC.removeListeners()
KeyWordFlutterPC.removeAllRNBridgeListeners()
KeyWordFlutterPC.setWakewordAudioRoutingConfig(AudioRoutingConfig config)

Top-level helpers:

pauseDetection(String instanceId, {bool stopMic = false})
unPauseDetection(String instanceId)
removeAllRNBridgeListeners()
setWakewordAudioRoutingConfig(AudioRoutingConfig config)

VAD

Voice activity detection APIs are available through FlutterWakeWord or KeyWordFlutterPC.

final detector = createKeyWordFlutterPCInstance('main');

await detector.startVADDetection();

final vadSub = detector.onVADDetectionEvent().listen((event) {
  // Handle VAD event payload.
});

await detector.setVADParams(0.5, 300);
final props = await detector.getVoiceProps();

await detector.stopVADDetection();

Platform support depends on the native library bundled for each platform.

Recording Helpers

final detector = createKeyWordFlutterPCInstance('main');

final wav = await detector.getRecordingWav();
final wavBytes = await detector.getRecordingWavArray();

getRecordingWav() returns the native recording payload or path. Availability depends on the native platform implementation.

Pause And Resume

await detector.pauseDetection(stopMic: true);
await detector.unPauseDetection();

Audio Routing

AudioRoutingConfig and RouteConfigEntry configure wake-word audio routing on iOS.

await setWakewordAudioRoutingConfig(
  AudioRoutingConfig(
    defaultRoute: RouteConfigEntry(
      category: WakewordAudioCategory.playAndRecord,
      mode: WakewordAudioMode.defaultMode,
      options: [
        WakewordAudioOption.mixWithOthers,
        WakewordAudioOption.defaultToSpeaker,
      ],
      preferredInput: WakewordPreferredInput.none,
    ),
    bluetoothHFP: RouteConfigEntry(
      category: WakewordAudioCategory.playAndRecord,
      mode: WakewordAudioMode.voiceChat,
      options: [
        WakewordAudioOption.allowBluetooth,
        WakewordAudioOption.mixWithOthers,
      ],
      preferredInput: WakewordPreferredInput.builtInMic,
    ),
  ),
);

Speaker Verification

Speaker verification mirrors the native speaker verification bridge where supported.

Verify WAV

final verifier = await createSpeakerVerificationInstance('speaker-engine');

await verifier.create(
  'speaker_model.dm',
  'enrollment.json',
  options: {
    'decisionThreshold': 0.35,
    'frameSize': 1280,
    'tailSeconds': 2.0,
    'maxTailSeconds': 3.0,
    'cmn': true,
    'expectedLayoutBDT': false,
  },
);

final result = await verifier.verifyWavStreaming(
  'test.wav',
  resetState: true,
);

await verifier.destroy();

Mic Controller

final controller = await createSpeakerVerificationMicController('speaker-mic');

await controller.create({
  'modelPath': 'speaker_model.dm',
  'options': {
    'decisionThreshold': 0.35,
    'frameSize': 1280,
    'tailSeconds': 2.0,
    'maxTailSeconds': 3.0,
    'cmn': true,
    'expectedLayoutBDT': false,
  },
});

final progressSub = onSpeakerVerificationOnboardingProgress((event) {
  // Handle onboarding progress.
});

final doneSub = onSpeakerVerificationOnboardingDone((event) {
  // Handle enrollment JSON/result.
});

final verifySub = onSpeakerVerificationVerifyResult((event) {
  // Handle verification result.
});

final errorSub = onSpeakerVerificationError((event) {
  // Handle native speaker verification errors.
});

await controller.beginOnboarding('user-1', 3);
await controller.getNextEmbeddingFromMic();
await controller.finalizeOnboardingNow();

await controller.setEnrollmentJson('{...}');
await controller.startVerifyFromMic();
await controller.startEndlessVerifyFromMic(
  hopSeconds: 0.5,
  stopOnMatch: false,
  resetState: true,
);

await controller.stop();
await controller.destroy();

await progressSub.cancel();
await doneSub.cancel();
await verifySub.cancel();
await errorSub.cancel();

Platform Notes

Native support differs by platform and by the bundled native library version.

iOS currently exposes the broader wake-word, VAD, audio routing, recording, and speaker verification APIs through the bundled xcframework.

Android supports the core wake-word APIs and the Android native methods exposed by the bundled AAR. APIs that are not available in the Android native library return Flutter's platform notImplemented response.

Example App

Example project:

https://github.com/frymanofer/Flutter_WakeWordDetection/

FAQ

What is a wake word?

A wake word is a keyword that activates your device, like "Hey Siri" or "OK Google".

How accurate is the platform?

The native DaVoice wake word engine is designed for high accuracy in varied environments.

Topics

Flutter wake word detection, custom wake words, keyword spotting, hotword detection, phrase spotting, voice activity detection, VAD, speaker verification, speaker identification, speaker recognition, voice authentication, enrollment, voice biometrics, Android wake word detection, iOS wake word detection, DaVoice.

Libraries

audio_routing_config

flutter_wake_word

flutter_wake_word_method_channel

flutter_wake_word_platform_interface

Index

instance_config

KeyWordFlutterPC

speaker_verification

use_model