mediapipe_vision library

Classes

BoundingBox: A box in pixels of the input image, as Google's C API reports it, not clipped to the image.
BrowserOverlay: A canvas the browser worker draws a task's landmarks or boxes into.
CategoryMask: The winning category index per pixel, in row-major input-image coordinates, owned by the result and valid after the task is disposed.
Classifications: The categories one classifier head produced, best first.
ClassifierOptions: Classification filters for a model head, as Gesture Recognizer nests them for its canned and custom gestures.
ConfidenceMask: Per-pixel confidences in row-major input-image coordinates, from 0 to 1, owned by the result and valid after the task is disposed.
Detection: One detected object or face: its box, categories and, for models that report them, keypoints.
DownloadAsset: A reviewed download. Every location must serve the same pinned bytes.
DownloadFailure: One unsuccessful source.
Embedding: One embedding head's vector: floats, or bytes when quantized.
FaceDetector: Google's Face Detector: boxes, scores and six keypoints per face.
FaceDetectorOptions: Options for the official Face Detector.
FaceDetectorResult: The faces in an image: each a box, a score and six keypoints (right eye, left eye, nose tip, mouth, right tragion, left tragion).
FaceLandmarker: Google's Face Landmarker: 478 landmarks per face, with optional blendshapes and transformation matrixes.
FaceLandmarkerOptions: Options for the official Face Landmarker.
FaceLandmarkerResult: The 478 landmarks of each face, with the optional blendshapes and transformation matrixes.
FaceLandmarksConnections: Google's FaceLandmarksConnections: the facial drawing edges, in the original index order.
GestureRecognizer: Google's Gesture Recognizer: the gestures and landmarks of each hand.
GestureRecognizerOptions: Options for the official Gesture Recognizer.
GestureRecognizerResult: The gestures and landmarks of each hand.
HandLandmarker: Google's Hand Landmarker: 21 landmarks per hand, in image and world space, with handedness.
HandLandmarkerOptions: Options for the official Hand Landmarker.
HandLandmarkerResult: The 21 landmarks of each hand, in image and world space, with handedness.
HandLandmarksConnections: Google's HandLandmarksConnections: the hand drawing edges, in the original index order.
HandTrackingOptions: Detection and tracking settings shared by the hand tasks.
HolisticLandmarker: Google's Holistic Landmarker: face, pose and hand landmarks of one person.
HolisticLandmarkerOptions: Options for the official Holistic Landmarker.
HolisticLandmarkerResult: The face, pose and hand landmarks of one person.
ImageClassifier: Google's Image Classifier: the categories of every model head.
ImageClassifierOptions: Options for the official Image Classifier.
ImageClassifierResult: The categories of every classifier head.
ImageEmbedder: Google's Image Embedder: one vector per model head.
ImageEmbedderOptions: Options for the official Image Embedder.
ImageEmbedderResult: The vectors of every embedder head.
ImageSegmenter: Google's Image Segmenter: confidence masks per category, a category mask, or both.
ImageSegmenterOptions: Options for the official Image Segmenter.
ImageSegmenterResult: The masks the Image Segmenter was asked for.
InteractiveSegmenter: Google's stateful MagicTouch Interactive Segmenter: an image, then the mask of the object a stroke history selects.
InteractiveSegmenterOptions: Options for Google's stateful MagicTouch Interactive Segmenter, which takes an image and then stroke histories.
Landmark: A landmark in world space, in meters from the subject's center.
Matrix: A matrix in column-major order, as Google's C API lays it out.
MediaPipeCategory: One classification: Google's Category, under the package's name because Flutter's foundation library already exports a Category.
MediaPipeWebRuntime: The browser runtime's location, shared by the vision, text and audio packages. Exists on every platform; only browsers read it.
ModelSource: Where a verified model is: a file path on native platforms, bytes in browsers. What ModelStore returns and a task reads.
ModelStore: Verified models shared by every task family: the app's bundled copies, cached in the application's persistent support directory, and downloads when asked for. The same API in browsers, where models live in Cache Storage and a ModelSource holds bytes instead of a path.
NormalizedKeypoint: A point normalized to the input image: 0 to 1 across its width and height.
NormalizedLandmark: A landmark in image space: x and y relative to the image's width and height, z scaled like x with smaller values closer to the camera.
ObjectDetector: Google's Object Detector: a box and categories per object.
ObjectDetectorOptions: Options for the official Object Detector.
ObjectDetectorResult: The objects in an image: each a box and its categories.
PoseLandmarker: Google's Pose Landmarker: 33 landmarks per pose, in image and world space, with optional foreground masks.
PoseLandmarkerOptions: Options for the official Pose Landmarker.
PoseLandmarkerResult: The 33 landmarks of each pose, in image and world space, with optional foreground masks.
PoseLandmarksConnections: Google's PoseLandmarksConnections: the pose drawing edges, in the original index order.
Stroke: One stroke of the history a segmentation request carries: its brush mode and its points, normalized to the input image.
TaskCapabilities: Declared package support for one task on a platform: which Delegates run there and why the others do not. This is not an inference self-test.
TaskOptions: Model and delegate shared by every task's options, in every family.
TaskPlatform: The current process platform, including its ABI (not the host CPU's ABI).
VisionImage: An input image. Pixel buffers are copied; the caller never owns native memory and may reuse its buffer at once.
VisionModels: Verified official models accepted by vision task options.
VisionRegionOfInterest: A region of an image in normalized coordinates, for the tasks that accept one (Image Classifier and Image Embedder).
VisionResult: What every vision task's result reports about its input: the decoded size and, in video and live stream modes, the frame's timestamp.
VisionTask: What every vision task has, so an app can hold any task through one type.
VisionTaskOptions: What every vision task's options add to TaskOptions: the running mode.

Enums

BrowserEngine: A browser's engine, for a task whose browser runtime fails in one of them.
BrushMode: How a stroke affects the selection, as Google's API names them.
Delegate: The processor a task runs on, chosen when the task is created and fixed until it is disposed. One enum serves vision, text and audio.
RunningMode: How a task is fed, chosen when it is created.
VisionPixelFormat: Channel order of 8-bit input pixels.

Functions

faceDetectorCapabilitiesForPlatform(TaskPlatform platform) → TaskCapabilities: Face Detector runs where Face Landmarker does, except on the Linux and Android GPU (_faceDetectorGpuGap).
faceLandmarkerCapabilitiesForPlatform(TaskPlatform platform) → TaskCapabilities: Face Landmarker: CPU on Google's vision library on Android, iOS, macOS, Linux and Windows, and the registered web adapter; GPU everywhere but Windows, macOS (_macosGpuGap) and the iOS Simulator.
gestureRecognizerCapabilitiesForPlatform(TaskPlatform platform) → TaskCapabilities: Gesture Recognizer runs where Face Landmarker does.
handLandmarkerCapabilitiesForPlatform(TaskPlatform platform) → TaskCapabilities: Hand Landmarker runs where Face Landmarker does.
holisticLandmarkerCapabilitiesForPlatform(TaskPlatform platform) → TaskCapabilities: Holistic Landmarker runs on the CPU where Face Landmarker does; its GPU runs in browsers only, since Google's libraries cannot open its face blendshapes model on a desktop, iPhone or Android GPU (UP-026).
imageClassifierCapabilitiesForPlatform(TaskPlatform platform) → TaskCapabilities: Image Classifier runs where Face Landmarker does.
imageEmbedderCapabilitiesForPlatform(TaskPlatform platform) → TaskCapabilities: Image Embedder runs where Face Landmarker does, except on the Linux GPU.
imageSegmenterCapabilitiesForPlatform(TaskPlatform platform) → TaskCapabilities: Image Segmenter, declared as imageClassifierCapabilitiesForPlatform, except on Android, where only the CPU is offered (_androidSegmenterGpuGap).
interactiveSegmenterCapabilitiesForPlatform(TaskPlatform platform) → TaskCapabilities: Evaluate support for an explicit process platform snapshot: CPU on Google's vision library on Android, iOS, macOS and Linux, and the web adapter. Google's Windows library exports the stateful API too, but a stroke takes about 25 seconds on it (upstream-issues.md UP-048), so Windows stays off.
objectDetectorCapabilitiesForPlatform(TaskPlatform platform) → TaskCapabilities: Object Detector runs where Face Landmarker does.
poseLandmarkerCapabilitiesForPlatform(TaskPlatform platform) → TaskCapabilities: Pose Landmarker runs where Face Landmarker does.
queryFaceDetectorCapabilities() → Future<TaskCapabilities>: Query Face Detector support on this process platform without a model.
queryFaceLandmarkerCapabilities() → Future<TaskCapabilities>: Query official FaceLandmarker platform support without creating a task.
queryGestureRecognizerCapabilities() → Future<TaskCapabilities>: Query Gesture Recognizer support on this process platform without a model.
queryHandLandmarkerCapabilities() → Future<TaskCapabilities>: Query Hand Landmarker support on this process platform without a model.
queryHolisticLandmarkerCapabilities() → Future<TaskCapabilities>: Query Holistic Landmarker support on this process platform without a model.
queryImageClassifierCapabilities() → Future<TaskCapabilities>: Query Image Classifier support on this process platform without a model.
queryImageEmbedderCapabilities() → Future<TaskCapabilities>: Query Image Embedder support on this process platform without a model.
queryImageSegmenterCapabilities() → Future<TaskCapabilities>: Query Image Segmenter support on this process platform without a model.
queryInteractiveSegmenterCapabilities() → Future<TaskCapabilities>: Describe the package's stateful MagicTouch support on this process platform.
queryObjectDetectorCapabilities() → Future<TaskCapabilities>: Describe the package's Object Detector support on this process platform.
queryPoseLandmarkerCapabilities() → Future<TaskCapabilities>: Query Pose Landmarker support on this process platform without a model.

Typedefs

RuntimeTargets = Map<String, String?>: Minimum product version for one process target, e.g. macos/arm64: 14.0.

Exceptions / Errors

MediaPipeException: A failure from MediaPipe or its supporting runtime.
ModelDownloadException: A model could not be downloaded from any configured source.
RuntimeUnavailableException: The platform cannot load a task runtime required by this operation.
TaskException: A failure Google's runtime reported while creating or running a task, on any platform and in any family.

mediapipe_vision library

Classes

Enums

Functions

Typedefs

Exceptions / Errors

mediapipe_vision package

mediapipe_vision library