FlutterPianoAudioDetection Plugin

pub package likes License: MIT style: effective dart

Flutter Piano Audio Detection implemented with Tensorflow Lite Model (Google Magenta)

  • x Android Implementation
  • x iOS/iPadOS Implementation

To keep this project alive, consider giving a star or a like. Contributors are also welcome.

Example Demo

ezgif com-gif-maker

Setting up a Flutter app with flutter_piano_audio_detection

1. Setting Tensorflow model file into your project

First, Add tensorflow lite file in your project. Copy the downloaded onsets_frames_wavinput.tflite.

Android : Copy the downloaded file YourApp/android/app/src/main/assets
iOS : Navigator -> Build Phases -> Copy Bundle Resourse

If you have experience installing other plugins, it should be very simple.

2. iOS Installation & Permissions

  1. Add the permissions below to your info.plist. This could be found in
    <string>Your Text</string>

2. Add the following to your Podfile file. Since the AudioModule library is sensitive to the iOS version, please apply the ios version in the Podfile to 12.1. and This plugin depends on [permission_handler flutter plugin](https://pub.dev/packages/permission_handler).
    platform :ios, '12.1' // or higher version.
    // ...
    post_install do |installer|
      installer.pods_project.targets.each do |target|
        target.build_configurations.each do |config|
          config.build_settings['GCC_PREPROCESSOR_DEFINITIONS'] ||= [
            ## dart: PermissionGroup.microphone

3. Android Installation & Permissions

  1. Add the permissions below to your AndroidManifest. This could be found in
<uses-permission android:name="android.permission.RECORD_AUDIO" />

  1. Edit the following below to your build.gradle. This could be found in YourApp/app/src/For example:
aaptOptions {
        noCompress 'tflite'

How to use this plugin

Please look at the example on how to implement these futures.

  1. Add line in pubspec.yaml
    flutter_piano_audio_detection: ${version}
  1. Usage in Flutter Code
  import 'package:flutter_piano_audio_detection/flutter_piano_audio_detection.dart';
  // ...
  class _YourAppState extends State<MyApp> {
    FlutterPianoAudioDetection fpad = new FlutterPianoAudioDetection();
    Stream<List<dynamic>>? result;
    List<String> notes = [];
    // ...
    void initState() {
    void start() {
      fpad.start(); // Start Engine 
      getResult();  // Event Subscription

    void stop() {
      fpad.stop();  // Stop Engine

    void getResult() {
      result = fpad.startAudioRecognition();
      result!.listen((event) {
        setState(() {
          notes = fpad.getNotes(event); // notes = [C3, D3]
    // ...