ML Object Detection & Tracking

The easy way to use ML Kit for object detection & tracking in Flutter.

With ML Kit's on-device object detection and tracking, we can detect and track objects in an image or live camera feed. Optionally, we can classify detected objects, either by using the built-in coarse classifier, or using custom image classification model.

universe

Getting Started

Add dependency to your flutter project:

$ flutter pub add learning_object_detection

or

dependencies:
  learning_object_detection: ^0.0.1

Then run flutter pub get.

Usage

import 'package:learning_object_detection/learning_object_detection.dart';

Input Image

As in other ML vision plugins, input is fed as an instance of InputImage, which is part of package learning_input_image.

You can use widget InputCameraView from learning_input_image as default implementation for processing image (or image stream) from camera / storage into InputImage format. But feel free to learn the inside of InputCameraView code if you want to create your own custom implementation.

Here is example of using InputCameraView to get InputImage for object detection & tracking.

import 'package:learning_input_image/learning_input_image.dart';

InputCameraView(
  title: 'Object Detection & Tracking',
  onImage: (InputImage image) {
    // now we can feed the input image into object detector
  },
)

Object Detection

After getting the InputImage, we can start detecting objetcs by calling method detect from an instance of ObjectDetector.

ObjectDetector detector = ObjectDetector();
List<DetectedObject> result = await detector.detect(image);

ObjectDetector is instantiated with default parameters as following.

ObjectDetector detector = ObjectDetector(
  isStream: false,
  enableClassification: true,
  enableMultipleObjects: true,
)

But we can override this by passing other values.

Parameter Value Default
isStream false / true false
enableClassification false / true true
enableMultipleObjects false, true true

Output

The result of face detection process is a list of DetectedObject, in each contains the following.

int? trackingId // Tracking ID of the object. The value is null when isStream is false.
Rect boundingBox // showing the rectangle of the detected object
List<DetectedLabel> labels // the list of possible labels for the detected object

Each object of DetectedLabel contains the following data.

int index // index of this label
String label // the label of the object
double confidence // the value representing the probability that the label is correct

Object Painting

To make it easy to paint from DetectedObject to the screen, we provide ObjectOverlay which you can pass to parameter overlay of InputCameraView. For more detail about how to use this painting, you can see at the working example code here.

ObjectOverlay(
  size: size,
  originalSize: originalSize,
  rotation: rotation,
  objects: objects,
)

Dispose

detector.dispose();

Example Project

You can learn more from example project here.

Libraries

learning_object_detection