tflite_flutter 0.3.0

TensorFlow Lite Flutter plugin provides easy, flexible and fast Dart API to integrate TFLite models in flutter apps.

TensorFlow Lite Flutter Plugin

TensorFlow Lite plugin provides a dart API for accessing TensorFlow Lite interpreter and performing inference. It binds to TensorFlow Lite C API using dart:ffi.

(Most Important) Initial setup

Add dynamic libraries to your app

• Linux/Mac Users

  Place the script install.sh at the root of your project.

  Execute

  sh install.sh

  at the root of your project to automatically download and place binaries at appropriate folders.

  The binaries installed will not include support for GpuDelegateV2 and NnApiDelegate however InterpreterOptions().useNnApiForAndroid can still be used.

  Use install.sh -d instead if you wish to use these GpuDelegateV2 and NnApiDelegate.

• Windows users

  Place the script install.bat at the root of your project.

  Execute

  install.bat

  at the root of your project to automatically download and place binaries at appropriate folders.

  If you want to use delegate support then execute install.bat -d.

  These scripts install pre-built binaries based on latest stable tensorflow release.

Why do we need to do this?

tflite_flutter dynamically links to C APIs which are supplied in the form of libtensorflowlite_c.so on Android and TensorFlowLiteC.framework on iOS.

For Android, We need to manually download these binaries from release assets and place the libtensorflowlite_c.so files in the <root>/android/app/src/main/jniLibs/ directory for each arm, arm64, x86, x86_64 architecture as done here in the example app.

No setup needed for iOS as of now, TensorFlowLiteC.framework is embedded in the plugin itself.

How to build locally ?

The pre-built binaries are updated with each stable tensorflow release. However, you many want to use latest unstable tf releases or older tf versions, for that proceed to build locally, if you are unable to find the required version in release assets.

Make sure you have required version of bazel installed. (Check TF_MIN_BAZEL_VERSION, TF_MAX_BAZEL_VERSION in configure.py)

• Android

Configure your workspace for android builds as per these instructions.

For TensorFlow >= v2.2

    bazel build -c opt --cxxopt=--std=c++11 --config=android_arm //tensorflow/lite/c:tensorflowlite_c
    
    // similarily for arm64 use --config=android_arm64

For TensorFlow <= v2.1

    bazel build -c opt --cxxopt=--std=c++11 --config=android_arm //tensorflow/lite/experimental/c:libtensorflowlite_c.so
    
    // similarily for arm64 use --config=android_arm64

• iOS

Refer instructions on TensorFlow Lite website to build locally for iOS.

Note: You must use macOS for building iOS.

Import

import 'package:tflite_flutter/tflite_flutter.dart';

Usage instructions

Creating the Interpreter

Interpreter can be created in three ways:

• directly from asset (easiest)

  Place your_model.tflite in assets directory. Make sure to include assets in pubspec.yaml.

  final interpreter = await tfl.Interpreter.fromAsset('your_model.tflite');
  

• from buffer

    final buffer = await getBuffer('assets/your_model.tflite');
    final interpreter = Interpreter.fromBuffer(buffer);
  
  Future<Uint8List> getBuffer(String filePath) async {  
   final rawAssetFile = await rootBundle.load(filePath);  
   final rawBytes = rawAssetFile.buffer.asUint8List();  
   return rawBytes;  
  }
  

• from file

  final dataFile = await getFile('assets/your_model.tflite');
  final interpreter = Interpreter.fromFile(dataFile);
    
  Future<File> getFile(String fileName) async {
    final appDir = await getTemporaryDirectory();
    final appPath = appDir.path;
    final fileOnDevice = File('$appPath/$fileName');
    final rawAssetFile = await rootBundle.load(fileName);
    final rawBytes = rawAssetFile.buffer.asUint8List();
    await fileOnDevice.writeAsBytes(rawBytes, flush: true);
    return fileOnDevice;
  }
  

Performing inference

• For single input and output

  Use void run(Object input, Object output).

  // For ex: if input tensor shape [1,5] and type is float32
  var input = [[1.23, 6.54, 7.81. 3.21, 2.22]];
  	
  // if output tensor shape [1,2] and type is float32
  var output = List(1*2).reshape([1,2]);
  
  // inference
  interpreter.run(input, output);
  	
  // print the output
  print(output);
  

• For multiple inputs and outputs

  Use void runForMultipleInputs(List<Object> inputs, Map<int, Object> outputs).

  var input0 = [1.23];  
  var input1 = [2.43];  
  	
  // input: List<Object>
  var inputs = [input0, input1, input0, input1];  
  	
  var output0 = List<double>(1);  
  var output1 = List<double>(1);
  	
  // output: Map<int, Object>   
  var outputs = {0: output0, 1: output1};
  	
  // inference  
  interpreter.runForMultipleInputs(inputs, outputs);
  	
  // print outputs
  print(outputs)
  

Closing the interpreter

interpreter.close();

Improve performance using delegate support

Note: This feature is under testing and could be unstable with some builds and on some devices.

• NNAPI delegate for Android

  var interpreterOptions = InterpreterOptions()..useNnApiForAndroid = true;
  final interpreter = await Interpreter.fromAsset('your_model.tflite',
      options: interpreterOptions);
    
  

  or

  var interpreterOptions = InterpreterOptions()..addDelegate(NnApiDelegate());
  final interpreter = await Interpreter.fromAsset('your_model.tflite',
      options: interpreterOptions);
        
  

• GPU delegate for Android and iOS

  • Android GpuDelegateV2

  final gpuDelegateV2 = GpuDelegateV2(
          options: GpuDelegateOptionsV2(
        false,
        TfLiteGpuInferenceUsage.fastSingleAnswer,
        TfLiteGpuInferencePriority.minLatency,
        TfLiteGpuInferencePriority.auto,
        TfLiteGpuInferencePriority.auto,
      ));
  
  var interpreterOptions = InterpreterOptions()..addDelegate(gpuDelegateV2);
  final interpreter = await Interpreter.fromAsset('your_model.tflite',
      options: interpreterOptions);
  

  • iOS Metal Delegate (GpuDelegate)

  final gpuDelegate = GpuDelegate(
        options: GpuDelegateOptions(true, TFLGpuDelegateWaitType.active),
      );
  var interpreterOptions = InterpreterOptions()..addDelegate(gpuDelegate);
  final interpreter = await Interpreter.fromAsset('your_model.tflite',
      options: interpreterOptions);
  
  

Refer Tests to see more example code for each method.

Refer Text Classification Flutter Example App for demo.

Credits

• Tian LIN, Jared Duke, Andrew Selle, YoungSeok Yoon, Shuangfeng Li from the TensorFlow Lite Team for their invaluable guidance.
• Authors of dart-lang/tflite_native.