learning_face_detection 0.0.3

The easy way to use ML Kit for face detection in Flutter.

ML Face Detection

The easy way to use ML Kit for face detection in Flutter.

With ML Kit's face detection, we can detect faces in an image, identify key facial features, and get the contours of detected faces. Note: it's only detecting faces, not recognizing people.

With face detection, we can get the information to perform tasks like embellishing selfies and portraits, or generating avatars from user's photo. Because it can perform face detection in real time, we can use it in applications like video chat, games, or TikTok that respond to user's expressions.

To get a starting grasp about the process of face detection, including landmarks, contours, and classification. Please read first Face Detection Concepts described in here.

Getting Started

Add dependency to your flutter project:

$ flutter pub add learning_face_detection

copied to clipboard

or

dependencies:
  learning_face_detection: ^0.0.2

copied to clipboard

Then run flutter pub get.

Usage

import 'package:learning_face_detection/learning_face_detection.dart';

copied to clipboard

Input Image

As in other ML vision plugins, input is fed as an instance of InputImage, which is part of package learning_input_image.

You can use widget InputCameraView from learning_input_image as default implementation for processing image (or image stream) from camera / storage into InputImage format. But feel free to learn the inside of InputCameraView code if you want to create your own custom implementation.

Here is example of using InputCameraView to get InputImage for face detection.

import 'package:learning_input_image/learning_input_image.dart';

InputCameraView(
  title: 'Face Detection',
  onImage: (InputImage image) {
    // now we can feed the input image into face detector
  },
)

copied to clipboard

Face Detection

After getting the InputImage, we can start detecting faces by calling method detect from an instance of FaceDetector.

FaceDetector detector = FaceDetector();
List<Face> result = await detector.detect(image);

copied to clipboard

FaceDetector is instantiated with default parameters as following.

FaceDetector detector = FaceDetector(
  mode: FaceDetectorMode.fast,
  detectLandmark: true,
  detectContour: true,
  enableClassification: false,
  enableTracking: false,
  minFaceSize: 0.15,
)

copied to clipboard

But we can override this by passing other values.

Parameter	Value	Default
mode	FaceDetectorMode.fast / FaceDetectorMode.accurate	FaceDetectorMode.fast
detectLandmark	false / true	false
detectContour	false / true	false
enableClassification	false / true	false
enableTracking	false, true	false
minFaceSize	Any value between 0.0 and 1.0	0.15

Output

The result of face detection process is a list of Face object, in each contains the following.

Rect boundingBox // showing the rectangle of the detected face

double headAngleY // Head is rotated to the right at headAngleY degrees

double headAngleZ // Head is tilted sideways at headAngleZ degrees

int? trackingId // Tracking ID

double? smilingProbability // the probability that the face is smiling

double? leftEyeOpenProbability // the probability that the left eye is open

double? rightEyeOpenProbability // the probability that the right eye is open

Map<FaceLandmarkType, FaceLandmark> landmarks // Map object representing the list of FaceLandmark

Map<FaceContourType, FaceContour> countours // Map object representing the list of FaceContour

copied to clipboard

The object of FaceLandmark contains two kinds of information: type and point.

FaceLandmarkType type
Offset point

copied to clipboard

Here is the list of FaceLandmarkType:

LEFT_EYE
RIGHT_EYE
LEFT_EAR
RIGHT_EAR
LEFT_CHEEK
RIGHT_CHEEK
NOSE_BASE
MOUTH_LEFT
MOUTH_RIGHT
MOUTH_BOTTOM

copied to clipboard

Each instance of FaceContour contains two information: type and points.

FaceContourType type
List<Offset> points

copied to clipboard

Here is the list of FaceContourType:

FACE
LEFT_EYE
RIGHT_EYE
LEFT_EYEBROW_TOP
LEFT_EYEBROW_BOTTOM
RIGHT_EYEBROW_TOP
RIGHT_EYEBROW_BOTTOM
NOSE_BRIDGE
NOSE_BOTTOM
LEFT_CHEEK
RIGHT_CHEEK
UPPER_LIP_TOP
UPPER_LIP_BOTTOM
LOWER_LIP_TOP
LOWER_LIP_BOTTOM

copied to clipboard

Face Painting

To make it easy to paint from Face object to the screen, we provide FaceOverlay which you can pass to parameter overlay of InputCameraView. For more detail about how to use this painting, you can see at the working example code here.

FaceOverlay(
  size: size,
  originalSize: originalSize,
  rotation: rotation,
  faces: faces,
  contourColor: Colors.white.withOpacity(0.8),
  landmarkColor: Colors.lightBlue.withOpacity(0.8),
)

copied to clipboard

Dispose

detector.dispose();

copied to clipboard

Example Project

You can learn more from example project here.