MediaPipe Tasks for Flutter

Tested with Flutter 3.47.9 Web CI Apache-2.0 license

Google's MediaPipe Tasks for Dart and Flutter on Android, iOS, macOS, Linux, Windows and the web: detect faces and objects, track face, hand and pose landmarks, recognize gestures, classify and embed images, text and audio, segment images, and more. Every task runs Google's official MediaPipe runtime for its platform, with Google's pinned models bundled at build time.

Try the live gallery

Which package do I need?

Package Use it for Install Guide
mediapipe_vision Face and object detection; face, hand, pose and holistic landmarks; gestures; image classification, embeddings and segmentation flutter pub add mediapipe_vision Vision
mediapipe_text Text classification, embeddings (EmbeddingGemma included) and language detection on every platform; proofreading and summarization on every platform except the web, where they are not yet available flutter pub add mediapipe_text Text
mediapipe_audio Audio classification, speech to text and text to speech, on every platform flutter pub add mediapipe_audio Audio
mediapipe_decision Yes-or-no, choice and score questions about a text (Decision Maker), on every platform flutter pub add mediapipe_decision Decision
mediapipe_retrieval Text, images and audio in one embedding space (Universal Embedder) and an on-device vector index over them (Semantic Retriever), on every platform flutter pub add mediapipe_retrieval Retrieval

Add only the families you use: each one bundles only Google's MediaPipe library for that family. Every family brings mediapipe_core, which holds the shared download code, model store and browser runtime settings (core guide), and exports its types, so you do not add or import core yourself unless your own code imports package:mediapipe_core/....

Quick start

The packages need Flutter 3.47 or newer; the latest stable release is recommended. Add a family from pub.dev, from your app's root:

flutter pub add mediapipe_vision

This adds mediapipe_vision: ^0.2.1 to your app's pubspec.yaml. List the models your app uses in the same pubspec, and declare the folder they are bundled in:

flutter:
  assets:
    - assets/mediapipe/

hooks:
  user_defines:
    mediapipe_vision:
      models: [face_landmarker]

Then bundle them from the app's root, and again whenever the list changes:

dart run mediapipe_core:bundle_models

It downloads each listed model once, checks it against its pinned SHA-256 and writes it into assets/mediapipe/, so the model ships inside your app: nothing is downloaded at run time, and the app works offline from its first launch. In CI, dart run mediapipe_core:bundle_models --check fails when the folder does not match the list. Each family's XxxModels.byName lists the names it accepts.

Now create a task with the bundled model:

import 'dart:typed_data';

import 'package:mediapipe_vision/mediapipe_vision.dart';

Future<int> countFaces(Uint8List rgba, int width, int height) async {
  final landmarker = await FaceLandmarker.create(
    FaceLandmarkerOptions(model: VisionModels.faceLandmarker),
  );
  try {
    final result = await landmarker.detect(
      VisionImage.fromPixels(
        pixels: rgba,
        width: width,
        height: height,
        format: VisionPixelFormat.rgba,
      ),
    );
    return result.faceLandmarks.length;
  } finally {
    await landmarker.dispose();
  }
}

A model that is not bundled makes create throw a RuntimeUnavailableException whose fix names the pubspec entry to add. To download models at run time instead, set ModelStore.allowDownloads = true before creating tasks; Android then needs the INTERNET permission, and a sandboxed macOS app the network client entitlement.

On macOS, build the app for Apple Silicon only (two lines in its AppInfo.xcconfig; without them, release builds fail). See platform setup. Your own models work too: pass modelPath or modelBytes instead of model.

Where it runs

Family Android iOS macOS arm64 Linux x64 Windows x64 Web
Vision ✓ ✓ ✓ ✓ ✓ ✓
Text (classify, embed, detect language) ✓ ✓ ✓ ✓ ✓ ✓
Text (EmbeddingGemma) ✓ ✓ ✓ ✓ ✓ ✓
Text (proofread, summarize) ✓ ✓ ✓ ✓ ✓
Audio ✓ ✓ ✓ ✓ ✓ ✓
Audio (transcribe, speak) ✓ ✓ ✓ ✓ ✓ ✓
Decision ✓ ✓ ✓ ✓ ✓ ✓
Retrieval ✓ ✓ ✓ ✓ ✓ ✓

Every text task runs on every platform listed except Text Proofreader and Text Summarizer, which are not yet available on the web, matching the MediaPipe Tasks Web SDK.

CPU works everywhere a family is listed, except that browsers run Decision and Retrieval on WebGPU alone, which needs a hardware adapter. GPU (Metal, OpenGL ES, WebGL, WebGPU or Android GPU) depends on the task; Macs run the vision tasks on the CPU for now, until Google confirms its Metal results since 1.1.1. On iPhones, Metal is offered for nine vision tasks and Audio Transcriber, tested on an iPhone 15 Pro (iOS 27) with Google's 1.2.2 library; iPads were not tested. Every cell is tested in CI on hosted runners, the Android emulator, the iOS Simulator and Chromium, Firefox and WebKit. The vision status table lists each task, delegate, known upstream limit and physical-device result. Ask at run time with the queryXxxCapabilities() functions: they report the supported delegates and why any other is unavailable.

More

Origin and license

Source is licensed under Apache-2.0, with upstream notices retained.

Libraries

mediapipe_core
What every MediaPipe task family shares, identical on Android, iOS, macOS, Linux, Windows and the web: the options base, the delegate, the value types results are made of, the exceptions, the model store and the capability types. Each family's library exports it, so apps need not import this package directly.
native_assets
Build-time helpers for downloading and bundling pinned MediaPipe libraries. Import this library from hooks, not from application code.
native_structs
Google's MediaPipe C structs that more than one family binds, from the release core pins for every family (family_runtimes.dart), and the helpers that fill and copy them. For the family packages' FFI bindings, whose ABI tests check every size and field offset against Google's wheel; not for applications.
platform_interface
What the family packages and platform plugins need from core and apps do not: model resolution, the runtime tables behind the capability queries, the checks every family shares, and the host details of Google's C API.
task_worker
One worker isolate per task, for the family packages' native runtimes: requests from the calling isolate run there one at a time, in call order, on a WorkerTask that is made, used and closed there. Not for applications, and not for browsers, which have no isolates.
web_task_bridge
The browser plumbing the family plugins share: one worker per task, created from the family's worker script, with requests and disposal in order; the page's scripts; and the WebGPU adapter. Web only, and not for applications.