HandDetector class

On-device hand detection and landmark estimation using TensorFlow Lite.

Implements a two-stage pipeline based on MediaPipe:

  1. Palm detection using SSD-based detector with rotation rectangle output
  2. Hand landmark model to extract 21 keypoints per detected hand

All inference runs in a background isolate, keeping the UI thread free.

Usage

// One-step construction
final detector = await HandDetector.create();

// Or two-step, if you need to configure between construction and init
final detector = HandDetector();
await detector.initialize();

final hands = await detector.detect(imageBytes);
await detector.dispose();

Constructors

HandDetector()
Creates a hand detector instance.

Properties

activeAccelerator → String?
Active inference backend label. On native this is always null (the engine is selected via useCompiledModel / PerformanceConfig, not a LiteRT.js accelerator); the web implementation reports 'webgpu' / 'wasm'. Kept for cross-platform API parity so the same code compiles on every platform.
no setter
hashCode → int
The hash code for this object.
no setterinherited
isInitialized → bool
Returns true if the detector has been initialized and is ready to use.
no setter
isReady → bool
Returns true if the detector has been initialized and is ready to use.
no setter
runtimeType → Type
A representation of the runtime type of the object.
no setterinherited

Methods

detect(Uint8List imageBytes) → Future<List<Hand>>
Detects hands in an image from raw bytes.
detectFromCameraFrame(CameraFrame frame, {int? maxDim}) → Future<List<Hand>>
Detects hands directly from a CameraFrame produced by prepareCameraFrame.
detectFromCameraImage(Object cameraImage, {CameraFrameRotation? rotation, bool? isBgra, int? maxDim}) → Future<List<Hand>>
One-call wrapper for live camera streams: takes a CameraImage-shaped object directly (any object exposing width, height, and planes with bytes / bytesPerRow / bytesPerPixel) and runs YUV packing, colour conversion, rotation, and downscale in the detection isolate, all off the UI thread.
detectFromFilepath(String path) → Future<List<Hand>>
Detects hands in an image file at path.
detectFromMat(Mat image) → Future<List<Hand>>
Detects hands in a pre-decoded cv.Mat image.
detectFromMatBytes(Uint8List bytes, {required int width, required int height, int matType = 16}) → Future<List<Hand>>
Detects hands from raw pixel bytes without constructing a cv.Mat first.
detectOnMat(Mat image) → Future<List<Hand>>
Detects hands in an OpenCV Mat image.
detectOnMatBytes(Uint8List bytes, {required int width, required int height, int matType = 16}) → Future<List<Hand>>
Detects hands from raw pixel bytes without constructing a cv.Mat first.
dispose() → Future<void>
Releases all resources used by the detector.
initialize({HandMode mode = HandMode.boxesAndLandmarks, HandLandmarkModel landmarkModel = HandLandmarkModel.full, double detectorConf = 0.45, double palmNmsIou = 0.45, double palmRoiScale = 2.6, int maxDetections = 10, double minLandmarkScore = 0.5, bool enableTracking = false, TrackingConfig trackingConfig = const TrackingConfig(), int interpreterPoolSize = 1, PerformanceConfig performanceConfig = const PerformanceConfig(), bool enableGestures = false, double gestureMinConfidence = 0.5, bool useCompiledModel = false, String liteRtAccelerator = 'auto', Set<Accelerator> accelerators = const {Accelerator.gpu, Accelerator.cpu}, Precision precision = Precision.fp16}) → Future<void>
Initializes the hand detector by loading TensorFlow Lite models.
initializeFromBuffers({required Uint8List palmDetectionBytes, required Uint8List handLandmarkBytes, Uint8List? gestureEmbedderBytes, Uint8List? gestureClassifierBytes, HandMode mode = HandMode.boxesAndLandmarks, HandLandmarkModel landmarkModel = HandLandmarkModel.full, double detectorConf = 0.45, double palmNmsIou = 0.45, double palmRoiScale = 2.6, int maxDetections = 10, double minLandmarkScore = 0.5, bool enableTracking = false, TrackingConfig trackingConfig = const TrackingConfig(), int interpreterPoolSize = 1, PerformanceConfig performanceConfig = const PerformanceConfig(), bool enableGestures = false, double gestureMinConfidence = 0.5, bool useCompiledModel = false, Set<Accelerator> accelerators = const {Accelerator.gpu, Accelerator.cpu}, Precision precision = Precision.fp16}) → Future<void>
Initializes the hand detector from pre-loaded model bytes.
noSuchMethod(Invocation invocation) → dynamic
Invoked when a nonexistent method or property is accessed.
inherited
resetTracking() → Future<void>
Clears the MediaPipe-style cross-frame tracking state (see initialize's enableTracking).
toString() → String
A string representation of this object.
inherited

Operators

operator ==(Object other) → bool
The equality operator.
inherited

Static Methods

create({HandMode mode = HandMode.boxesAndLandmarks, HandLandmarkModel landmarkModel = HandLandmarkModel.full, double detectorConf = 0.45, double palmNmsIou = 0.45, double palmRoiScale = 2.6, int maxDetections = 10, double minLandmarkScore = 0.5, bool enableTracking = false, TrackingConfig trackingConfig = const TrackingConfig(), int interpreterPoolSize = 1, PerformanceConfig performanceConfig = const PerformanceConfig(), bool enableGestures = false, double gestureMinConfidence = 0.5, bool useCompiledModel = false, String liteRtAccelerator = 'auto', Set<Accelerator> accelerators = const {Accelerator.gpu, Accelerator.cpu}, Precision precision = Precision.fp16}) → Future<HandDetector>
Creates and initializes a hand detector in one step.
modelVersionFor({HandMode mode = HandMode.boxesAndLandmarks, HandLandmarkModel landmarkModel = HandLandmarkModel.full, bool enableGestures = false}) → String
Builds a version key for a specific hand detector configuration.

Constants

modelVersion → const String
Version key for the default hand detection pipeline.