vision_ai_pose

On-device body pose detection (33-point skeleton) for Flutter, with pure-Dart fitness/posture analytics and skeleton overlays. Built on the vision_ai engine — 100% on-device, no cloud.

What it does

33-point body skeleton per frame via the MediaPipe Pose Landmarker (LIVE_STREAM, single person by default).
Normalized landmarks (image coords, plus per-point visibility/presence) for overlays, and worldLandmarks (metres, origin at the hip midpoint) for joint-angle / limb metrics.
Bring your own model — swap the bundled full variant for lite / heavy or a fully custom .task.

Install

dependencies:
  vision_ai_pose: ^0.1.0

This package bundles the pose model and re-exports vision_ai, so importing package:vision_ai_pose/vision_ai_pose.dart is all you need.

Android release builds: MediaPipe requires R8 shrinking disabled. See the vision_ai README.

Quick start

import 'package:vision_ai_pose/vision_ai_pose.dart';

final vision = await PoseVision.create(
  pose: const PoseConfig(), // numPoses: 1 by default
);
final textureId = await vision.start();

// Render the camera preview
Texture(textureId: textureId);

vision.results.listen((result) {
  final pose = result.primaryPose;
  if (pose == null) return;
  // 33 landmarks in MediaPipe order; see PoseLandmarkType.
  final leftWrist = pose.landmarks[PoseLandmarkType.leftWrist];
  print('left wrist: ${leftWrist.x}, ${leftWrist.y} (vis ${leftWrist.visibility})');
});

await vision.stop();
await vision.dispose();

The model

PoseModels.ensureLoaded() copies the bundled pose_landmarker_full.task into app storage on first launch and returns its path; PoseVision.create calls it for you.

Variant	Size	Use
lite	~5.5 MB	low-end devices, highest FPS
full (bundled)	~9 MB	balanced — the live fitness/posture default
heavy	~29 MB	offline/clinical precision on high-end devices

Attaching a custom model

You can swap the bundled full model for the lite/heavy variant or any custom-trained MediaPipe Pose Landmarker .task:

final vision = await PoseVision.create(
  customModelPath: '/data/.../pose_landmarker_lite.task', // absolute file path
);

When customModelPath is set, the bundled asset is not copied — your path goes straight to the native engine.

The contract:

It must be a MediaPipe Pose Landmarker .task bundle (33-landmark topology). Download the official variants from storage.googleapis.com/mediapipe-models/pose_landmarker/<variant>/float16/latest/.
customModelPath is an absolute filesystem path, not a Flutter asset key. The engine reads the file directly (setModelAssetBuffer on Android, modelAssetPath on iOS), so the .task must exist on disk first.

Getting a file path — three ways:

Bundle it as an app asset, then copy to storage (most common). Declare the .task under flutter: assets: in your app's pubspec.yaml, then copy it to the app-support directory once and pass that path:

import 'dart:io';
import 'package:flutter/services.dart' show rootBundle;
import 'package:path_provider/path_provider.dart';

Future<String> _copyAsset(String assetKey, String fileName) async {
  final dir = await getApplicationSupportDirectory();
  final dest = File('${dir.path}/$fileName');
  if (!await dest.exists()) {
    final data = await rootBundle.load(assetKey);
    await dest.writeAsBytes(data.buffer.asUint8List(), flush: true);
  }
  return dest.path;
}

final path = await _copyAsset('assets/pose_landmarker_heavy.task',
    'pose_landmarker_heavy.task');
final vision = await PoseVision.create(customModelPath: path);

Download at runtime to the app-support directory (keeps your app binary small) and pass the downloaded file's path.
Any existing absolute path — e.g. a model you ship via your own update mechanism.

No core or native change is needed; the engine loads whatever .task the path points at, so the variant/model is entirely the caller's choice.

Landmarks & topology

PoseLandmarkType gives the 33 indices (nose = 0 … rightFootIndex = 32), and poseConnections is the canonical bone list for drawing the skeleton. Both PoseResult.landmarks and PoseResult.worldLandmarks follow this order.

Analytics (pure Dart, zero model weight)

All analytics consume a PoseResult and run as plain Dart math — no extra models.

// Joint angles (degrees at the middle point), 2D or 3D.
final angles = PoseAngles.of(pose);          // leftKnee, rightElbow, … (null if unseen)
final elbow = jointAngle(a, b, c, use3D: true);

// Rep counter — down→up oscillation with hysteresis (pick any joint).
final reps = RepCounter(
  angleSelector: (p) => PoseAngles.of(p).leftKnee,
  downAngle: 90,  // flexed
  upAngle: 160,   // extended
);
final state = reps.update(pose);             // state.count, state.phase

// Posture: upright / leaning / crouching / lyingDown / unknown.
final posture = const PostureClassifier().classify(pose);

// Fall: torso goes near-horizontal shortly after a rapid hip drop (latched).
final fall = FallDetector();
final fallen = fall.update(pose, timestampMs);

// Gate analytics on landmark confidence.
if (isPoseReliable(pose)) reps.update(pose);

Stateful detectors (RepCounter, FallDetector) expose reset(); switch subjects with it.

Overlays

PoseCameraView stacks the skeleton over the camera Texture; PoseSkeletonPainter draws poseConnections as bones plus landmark dots (visibility-aware), and PoseOverlayStyle tunes colors/width/minVisibility. RepCounterOverlay binds a RepCounter to a live count/phase badge.

PoseCameraView(
  results: vision.results,
  textureId: textureId,
  mirrored: usingFrontCamera,
)

Performance note

Pose is heavier than hand/face. Running pose alongside other engines stacks models on the single native analysis thread and costs FPS — prefer pose as its own mode. Inference itself runs async (LIVE_STREAM) off the analysis thread.

License

Apache 2.0 — see LICENSE. Pose detection uses the MediaPipe Pose Landmarker model (Apache 2.0).