fdemon_driver

In-process driver for debug-mode Flutter apps. It registers ext.fdemon.driver.* Dart VM service extensions so fdemon can find widgets and synthesize taps, scrolls, long-presses, and text entry — the same in-process mechanism integration_test uses.

Setup

import 'package:fdemon_driver/fdemon_driver.dart';

void main() {
  enableFdemonDriver(); // No-op in release builds.
  runApp(const MyApp());
}

That's it. When the app runs in debug mode, fdemon connects to the app's Dart VM service and drives it through the registered extensions.

Protocol

Every extension takes a single args parameter holding a JSON-encoded object (individual string parameters are accepted as a fallback) and returns JSON. Failures are returned as {"error": "...", ...} payloads — for example with a candidates list when a target matches zero or multiple widgets — never as RPC faults.

find match entries have the shape {"description","type","key"?,"text"?,"rect":{"x","y","w","h"},"hitTestable"} with rect in global logical pixels.

Settling (tap / longPress)

Tapping a widget inside a scrollable that is still fling-decelerating is standard Flutter behavior gone wrong for automation: the pointer-down is consumed as "stop scrolling" instead of activating the item, and find can report mid-fling rects (including hitTestable: false for moving targets).

tap and longPress therefore settle widget targets by default (settle, default true; coordinate x+y gestures never settle): the handler re-resolves the widget fresh (it may rebuild while scrolling) and samples its global rect every ~80 ms until two consecutive samples agree within 0.5 px on all four edges, then computes the gesture point. If the rect has not stabilized after settleTimeoutMs (default 3000) of sampling, the gesture proceeds anyway at the last observed position — a target inside a looping animation must not fail the action — and the response carries "settled": false plus "motion": {"dx","dy","samples"} (the center movement observed over the sampling window). A successful settle adds "settled": true; pass "settle": false to act immediately with no settled field.

Semantics labels

semanticsAction matches labels exactly first; when nothing matches exactly it falls back to a whitespace-normalized, case-insensitive substring match, so labels merged from several widgets (for example by MergeSemantics, which joins the parts with newlines) are still targetable by any one part. No-match errors include a candidates list of the labeled semantics nodes currently on screen so the caller can pick a valid label.

Security note

The driver is debug-mode only: enableFdemonDriver() runs entirely inside an assert, so it compiles to nothing in release builds. Even so, treat the Dart VM service as a privileged surface — anyone who can reach the VM service port can drive the app, read memory, and execute code. Keep the VM service bound to localhost (the Flutter default) and never expose it on untrusted networks.