flutter_mediapipe 0.0.7 icon indicating copy to clipboard operation
flutter_mediapipe: ^0.0.7 copied to clipboard

The flutter mediapipe plugin project.

flutter_mediapipe #

Flutter plugin with mediapipe facemesh.

Devices #

Currently, it runs on below devices with "OK".

Android #

There are Mediapipe Manual Build for Android flutter plugin.

There are mobile_calculators list to run on Mobile.

Choose one. This plugin choose face_mesh.

  • face_detection
  • face_mesh
  • object_detection
  • hand_tracking

Docker #

git clone https://github.com/google/mediapipe.git
cd mediapipe
docker build --tag=mediapipe .
docker run -it --name mediapipe mediapipe:latest

When enter docker again:

docker start [your container ID] 
docker exec -i -t mediapipe bash

Run below commands on the container.

Setup #

apt update
apt install vim
apt install zip
bash ./setup_android_sdk_and_ndk.sh

Prepare #

mkdir mediapipe/examples/android/src/java/com/google/mediapipe/apps/flutter_mediapipe
vi mediapipe/examples/android/src/java/com/google/mediapipe/apps/flutter_mediapipe/BUILD 
  • "BUILD" file content.
load("//mediapipe/java/com/google/mediapipe:mediapipe_aar.bzl", "mediapipe_aar")

mediapipe_aar(
    name = "flutter_mediapipe",
    calculators = ["//mediapipe/graphs/face_mesh:mobile_calculators"],
)

Build #

  • jniLibs
bazel build -c opt --host_crosstool_top=@bazel_tools//tools/cpp:toolchain --fat_apk_cpu=arm64-v8a,armeabi-v7a --strip=ALWAYS //mediapipe/examples/android/src/java/com/google/mediapipe/apps/flutter_mediapipe:BUILD --linkopt="-s"
bazel build -c opt --host_crosstool_top=@bazel_tools//tools/cpp:toolchain --fat_apk_cpu=arm64-v8a,armeabi-v7a //mediapipe/examples/android/src/java/com/google/mediapipe/apps/flutter_mediapipe:flutter_mediapipe --linkopt="-s"
  • binary graph
bazel build -c opt --host_crosstool_top=@bazel_tools//tools/cpp:toolchain --fat_apk_cpu=arm64-v8a,armeabi-v7a --strip=ALWAYS //mediapipe/examples/android/src/java/com/google/mediapipe/apps/facemeshgpu:BUILD
bazel build -c opt --host_crosstool_top=@bazel_tools//tools/cpp:toolchain --fat_apk_cpu=arm64-v8a,armeabi-v7a //mediapipe/examples/android/src/java/com/google/mediapipe/apps/facemeshgpu:facemeshgpu

mkdir #

mkdir flutter_mediapipe/
mkdir flutter_mediapipe/android
mkdir flutter_mediapipe/android/libs
mkdir flutter_mediapipe/android/src
mkdir flutter_mediapipe/android/src/main
mkdir flutter_mediapipe/android/src/main/assets
mkdir flutter_mediapipe/android/src/main/jniLibs
mkdir flutter_mediapipe/protos

libs #

cp bazel-bin/mediapipe/examples/android/src/java/com/google/mediapipe/apps/flutter_mediapipe/libflutter_mediapipe_android_lib.jar flutter_mediapipe/android/libs

assets #

cp mediapipe/modules/face_detection/face_detection_front.tflite flutter_mediapipe/android/src/main/assets
cp mediapipe/modules/face_landmark/face_landmark.tflite flutter_mediapipe/android/src/main/assets
cp bazel-out/k8-opt/bin/mediapipe/graphs/face_mesh/face_mesh_mobile_gpu.binarypb flutter_mediapipe/android/src/main/assets

jniLibs #

mkdir work
cp bazel-bin/mediapipe/examples/android/src/java/com/google/mediapipe/apps/flutter_mediapipe/flutter_mediapipe.aar work/aar.zip
cd work/
unzip aar.zip
cd ..
cp -r work/jni/* flutter_mediapipe/android/src/main/jniLibs/

protos #

cp mediapipe/framework/formats/landmark.proto flutter_mediapipe/protos/

See regenerate.md

Zip #

zip -r flutter_mediapipe.zip flutter_mediapipe

And then, exit from the container.

Copy #

Change "3730fdc4d319" to your container ID.

docker cp 3730fdc4d319:/mediapipe/flutter_mediapipe.zip .
unzip flutter_mediapipe.zip

Then copy "flutter_mediapipe" to flutter plugin projects.

APK #

flutter build apk --split-per-abi

Refferences #

iOS #

Not implemented. Help me to develop.

Custormize mediapipe #

Graph #

When editing graphs or subgraphs, Build again.

  • graphs: binary graph
  • subgraphs: jniLibs
vi mediapipe/graphs/face_mesh/subgraphs/face_renderer_gpu.pbtxt

It is a sample to comment out of lines from 92 to 94 for hiding rectangles and landmarks.

# Draws annotations and overlays them on top of the input images.
node {
  calculator: "AnnotationOverlayCalculator"
  input_stream: "IMAGE_GPU:input_image"
#  input_stream: "detections_render_data"
#  input_stream: "VECTOR:0:multi_face_landmarks_render_data"
#  input_stream: "rects_render_data"
  output_stream: "IMAGE_GPU:output_image"
}

Calculator #

When editing c++ calculator source, Build again.

  • c++ source: jniLibs
vi mediapipe/graphs/face_mesh/calculators/face_landmarks_to_render_data_calculator.cc

It is a sample to comment out the lines(31 and 45) for hiding Left eyebrow connections.

constexpr int kNumFaceLandmarkConnections = 116; // (124 - (16/2))  c.f. l.93
// Pairs of landmark indices to be rendered with connections.
constexpr int kFaceLandmarkConnections[] = {
  :
    133,
    // Left eyebrow.
//    46, 53, 53, 52, 52, 65, 65, 55, 70, 63, 63, 105, 105, 66, 66, 107,
    // Right eye.
  :
  for (int i = 0; i < kNumFaceLandmarkConnections; ++i) {
    landmark_connections_.push_back(kFaceLandmarkConnections[i * 2]);
    landmark_connections_.push_back(kFaceLandmarkConnections[i * 2 + 1]);
  }
15
likes
100
pub points
63%
popularity

Publisher

unverified uploader

The flutter mediapipe plugin project.

Homepage

Documentation

API reference

License

Icon for licenses.BSD-3-Clause (LICENSE)

Dependencies

flutter, protobuf

More

Packages that depend on flutter_mediapipe