MediaPipe Tasks for Flutter
Google's MediaPipe Tasks for Dart and Flutter on Android, iOS, macOS, Linux, Windows and the web: detect faces and objects, track face, hand and pose landmarks, recognize gestures, classify and embed images, text and audio, segment images, and more. Every task runs Google's official MediaPipe runtime for its platform, with Google's pinned models bundled at build time.
Which package do I need?
| Package | Use it for | Install | Guide |
|---|---|---|---|
mediapipe_vision |
Face and object detection; face, hand, pose and holistic landmarks; gestures; image classification, embeddings and segmentation | flutter pub add mediapipe_vision |
Vision |
mediapipe_text |
Text classification, embeddings (EmbeddingGemma included) and language detection on every platform; proofreading and summarization on every platform except the web, where they are not yet available | flutter pub add mediapipe_text |
Text |
mediapipe_audio |
Audio classification, speech to text and text to speech, on every platform | flutter pub add mediapipe_audio |
Audio |
mediapipe_decision |
Yes-or-no, choice and score questions about a text (Decision Maker), on every platform | flutter pub add mediapipe_decision |
Decision |
mediapipe_retrieval |
Text, images and audio in one embedding space (Universal Embedder) and an on-device vector index over them (Semantic Retriever), on every platform | flutter pub add mediapipe_retrieval |
Retrieval |
Add only the families you use: each one bundles only Google's MediaPipe
library for that family. Every family brings mediapipe_core, which holds the
shared download code, model store and browser runtime settings
(core guide), and exports its types, so
you do not add or import core yourself unless your own code imports
package:mediapipe_core/....
Quick start
The packages need Flutter 3.44 or newer; the latest stable release is recommended. iOS and macOS builds with Xcode 27 need Flutter 3.47 or later. Add a family from pub.dev, from your app's root:
flutter pub add mediapipe_vision
This adds mediapipe_vision: ^0.3.0 to your app's pubspec.yaml. List the
models your app uses in the same pubspec, and declare the folder they are
bundled in:
flutter:
assets:
- assets/mediapipe/
hooks:
user_defines:
mediapipe_vision:
models: [face_landmarker]
Then bundle them from the app's root, and again whenever the list changes:
dart run mediapipe_core:bundle_models
It downloads each listed model once, checks it against its pinned SHA-256 and
writes it into assets/mediapipe/, so the model ships inside your app:
nothing is downloaded at run time, and the app works offline from its first
launch. In CI, dart run mediapipe_core:bundle_models --check fails when the
folder does not match the list. Each family's XxxModels.byName lists the
names it accepts.
Now create a task with the bundled model:
import 'dart:typed_data';
import 'package:mediapipe_vision/mediapipe_vision.dart';
Future<int> countFaces(Uint8List rgba, int width, int height) async {
final landmarker = await FaceLandmarker.create(
FaceLandmarkerOptions(model: VisionModels.faceLandmarker),
);
try {
final result = await landmarker.detect(
VisionImage.fromPixels(
pixels: rgba,
width: width,
height: height,
format: VisionPixelFormat.rgba,
),
);
return result.faceLandmarks.length;
} finally {
await landmarker.dispose();
}
}
A model that is not bundled makes create throw a
RuntimeUnavailableException whose fix names the pubspec entry to add. To
download models at run time instead, set ModelStore.allowDownloads = true
before creating tasks; Android then needs the INTERNET permission, and a
sandboxed macOS app the network client entitlement.
On macOS, build the app for Apple Silicon only (two lines in its
AppInfo.xcconfig; without them, release builds fail). See
platform setup. Your own models work too: pass
modelPath or modelBytes instead of model.
Where it runs
| Family | Android | iOS | macOS arm64 | Linux x64 | Windows x64 | Web |
|---|---|---|---|---|---|---|
| Vision | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Text (classify, embed, detect language) | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Text (EmbeddingGemma) | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Text (proofread, summarize) | ✓ | ✓ | ✓ | ✓ | ✓ | |
| Audio | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Audio (transcribe, speak) | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Decision | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| Retrieval | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
Every text task runs on every platform listed except Text Proofreader and Text Summarizer, which are not yet available on the web, matching the MediaPipe Tasks Web SDK.
CPU works everywhere a family is listed, except that browsers run Decision
and Retrieval on WebGPU alone, which needs a hardware adapter. GPU (Metal,
OpenGL ES, WebGL, WebGPU or Android GPU) depends on the task; Macs run the
vision tasks on the CPU for now, until Google confirms its Metal results since
1.1.1. On iPhones, Metal is offered for nine vision tasks and Audio
Transcriber, tested on an iPhone 15 Pro (iOS 27) with Google's 1.2.2 library;
iPads were not tested. Every cell is tested in CI on hosted runners, the
Android emulator, the iOS Simulator and Chromium, Firefox and WebKit. The vision status table
lists each task, delegate, known upstream limit and physical-device result.
Ask at run time with the queryXxxCapabilities() functions: they report the
supported delegates and why any other is unavailable.
More
- Samples for every task: vision, text, audio, decision and retrieval.
- Object detection guide, with a sample app for every platform, following Google's Android guide and sample.
- Video and live stream samples for every camera-capable vision task.
- Vision options and extra outputs: regions of interest, RGB input, masks, matrices and every capability query.
- Platform setup: permissions, entitlements, minimum OS versions, offline builds and browser hosting.
- Privacy and licenses: what is downloaded, from where, and under which license.
- Migration from Google's
mediapipe_textandmediapipe_core0.0.1 previews. - Gallery: the full demo app, also live on GitHub Pages.
- Contributing, including how to add a task.
Origin and license
Source is licensed under Apache-2.0, with upstream notices retained.
Libraries
- mediapipe_core
- What every MediaPipe task family shares, identical on Android, iOS, macOS, Linux, Windows and the web: the options base, the delegate, the value types results are made of, the exceptions, the model store and the capability types. Each family's library exports it, so apps need not import this package directly.
- native_assets
- Build-time helpers for downloading and bundling pinned MediaPipe libraries. Import this library from hooks, not from application code.
- native_structs
- Google's MediaPipe C structs that more than one family binds, from the
release core pins for every family (
family_runtimes.dart), and the helpers that fill and copy them. For the family packages' FFI bindings, whose ABI tests check every size and field offset against Google's wheel; not for applications. - platform_interface
- What the family packages and platform plugins need from core and apps do not: model resolution, the runtime tables behind the capability queries, the checks every family shares, and the host details of Google's C API.
- task_worker
- One worker isolate per task, for the family packages' native runtimes: requests from the calling isolate run there one at a time, in call order, on a WorkerTask that is made, used and closed there. Not for applications, and not for browsers, which have no isolates.
- web_task_bridge
- The browser plumbing the family plugins share: one worker per task, created from the family's worker script, with requests and disposal in order; the page's scripts; and the WebGPU adapter. Web only, and not for applications.