marionette_mcp 0.7.0

MCP server that lets AI agents (Claude, Copilot, Cursor) inspect and interact with running Flutter apps — tap, scroll, type, and screenshot.

0.7.0

• Behaviour change: get_interactive_elements now reports only primitive-valued properties (bool, num, String, Enum) for every caller. The ButtonStyle, TextStyle, InputDecoration and colour blobs that previously dominated the payload are gone from every level of output, which shrank by about a quarter on the example app. A TextField's decoration.labelText/hintText is now surfaced as label so keyless fields keep a human-readable handle
• Behaviour change: get_interactive_elements now compacts its payload by default (MarionetteConfiguration.compaction defaults to CompactionMode.compact). On top of the primitive filter it drops rendering details and text font metrics, drops a Text element's duplicated data, rounds bounds to whole logical pixels, and reports visible only when an element is not visible — about 65% smaller than before on the example app. Nothing it drops is read by tap, enter_text, scroll_to or swipe, which match on key, identifier, text, type or coordinates. Set compaction: CompactionMode.none to get the full property dump back
• Add a compaction parameter to get_interactive_elements (--compaction=<none|compact> in the CLI) that overrides MarionetteConfiguration.compaction in both directions, so an agent can ask for the full payload on an app that keeps the default. CompactionMode deliberately has only none and compact: a third level is planned, but shipping a value that behaves exactly like compact would mislead, and enum values can be added later
• Add isInteractiveElement and shouldStopTraversalAtElement configuration callbacks that receive an Element, and deprecate the Type-based isInteractiveWidget and shouldStopTraversal (#125)
• Fix get_interactive_elements not recognizing generic built-in widgets (DropdownButton<T>, DropdownButtonFormField<T>, Radio<T>, RadioListTile<T>, PopupMenuButton<T>) and subclasses of built-in widgets, such as ElevatedButton.icon
• Fix press_back_button on a page-based Navigator (as produced by go_router) that refuses the pop while a route is still animating in — it now waits for the route transition to finish before popping (other animations, such as a loading spinner, don't hold it up) instead of tripping navigator.dart's internal route-lifecycle assertion, which used to fail every navigation for the rest of the session (#113)
• Fix register/unregister rejecting instance names with dots or colons, such as device IDs like 192.168.1.1:5555 — only /, \, and control characters are disallowed now, and names are encoded into portable, Windows-safe file names on disk
• Add marionette-flutter-drive-app, an Agent Skill shipped inside the marionette_flutter package, installable via dart run skills@ get (Dart's package skills convention) or npx skills add leancodepl/marionette_mcp — see the new "Agent Skill" section in the README
• Add opt-in session reports: with MarionetteConfiguration(enableSessionReports: true), connect/CLI commands log to a per-run .marionette/sessions/ directory (steps.md, screenshots, and an agent-written report.md) — see Session Reports
• Add set_device_config tool and set-device-config CLI command for overriding text scale, bold text, and platform brightness in a running app — opt in by wrapping your root widget in the new MarionetteDeviceConfig
• Add ancestor_keys to tap, secondary_tap, double_tap, long_press, swipe, pinch_zoom, scroll_to and enter_text, and a repeatable --ancestor-keys to the matching CLI commands, to scope a match to one subtree when the same key repeats across identical subtrees (grid cells, repeated cards, embedded app instances) — keys nest outermost first, only descendants of the last one match, and a key with no element fails the call by name instead of widening the search; scroll_to keeps scrolling first, so a scope inside a lazily built list works
• Add ancestor_keys to get_interactive_elements, and --ancestor-keys to get-interactive-elements, to list only one subtree — the same chain can then be passed to the matcher tools to act inside it
• Add matching by Semantics identifier to the matcher tools (--identifier in the CLI), and report each element's identifier in get_interactive_elements (#90)
• Fix extension errors being reported as the generic Server error instead of the detail the extension returned — this silently swallowed every validation message and setup instruction, including the get_logs log-collector onboarding help
• Fix press_key with enter not submitting the focused text field: it now performs the field's text input action, so onSubmitted runs, while a multiline field still gets a new line (#117)
• Fix MarionetteBinding.ensureInitialized() silently hanging the app when another binding, such as the one SentryFlutter.init installs, was created first — it now throws an error that says to initialize MarionetteBinding first (#96)
• Fix connect hanging forever when the app's Dart Development Service stops answering stream subscriptions: it now waits at most 500 ms for the Service stream and continues, and hot_reload/hot_restart report failure in that state (#127)
• Fix scroll_to on layered UIs: scroll the list the user can reach and stop at the reachable copy of the target, instead of dragging a screen covered by a bottom sheet, dialog, or pushed route
• Fix scroll_to missing a target that only comes into view on its final drag, at either end of a list
• Fix scroll_to being unable to reach targets in long lists: it now scrolls by a share of the visible viewport instead of a fixed 64px, so lists past roughly 170 rows are reachable and shorter ones take far fewer gestures
• Fix scroll_to on screens with more than one scrollable: when the target is not built yet it now tries the two best-ranked reachable scrollables in turn rather than committing to a single guess, so a chip row, tab strip or nav rail no longer swallows the gesture
• Fix scroll_to reporting an attempt count instead of the cause when no scrollable could move at all: it now fails with no Scrollable could be scrolled to reveal it
• Fix every pointer gesture (tap, double_tap, long_press, secondary_tap, swipe, pinch_zoom, scroll_to) silently doing nothing, and every element being reported visible: false, in an app whose content is not in the implicit view — as with the desktop windowing API: gestures are now dispatched to, and visibility measured against, the view the element is actually in

0.6.0

• Promote schema-bearing custom extensions to first-class MCP tools via the new ExtensionInputSchema/ExtensionParam DSL
• Add press_key tool for dispatching hardware key events with modifiers
• Add hot_restart tool for fully restarting the app from main()
• Add secondary_tap tool for right mouse button clicks on desktop
• Add swipe CLI command for element- and coordinate-based swipe/drag gestures
• Surface Semantics(label:) and Semantics(value:) in get_interactive_elements
• Exit the stdio MCP server on stdin EOF to prevent orphaned processes
• Bump mcp_dart to ^2.1.0 and drop the bespoke Copilot-compat stdio transport
• Add a Dockerfile for building marionette_mcp as a container image

0.5.0

• Breaking: Pass Element instead of Widget to the extractText configuration callback, giving access to the full element context. Migration: change (widget) to (element) and use element.widget to access the widget.
• Add gesture support across all layers: swipe/drag, long_press, double_tap, pinch_zoom, and press_back_button.
• Add video recording with TCP/WS transport and Android auto-fallback.
• Improve screencast stability with broken-pipe race handling, concurrent stop safety, and orphaned session cleanup.
• Improve runtime robustness around input validation, resource cleanup, and non-monotonic timestamp handling.
• Strengthen release CI with explicit minimum-version compatibility checks and Flutter version sourced from .fvmrc.

0.4.0

• Add marionette_cli package with multi-instance CLI support
• Add call_custom_extension tool for arbitrary VM service extensions
• Add custom extension registry with registerMarionetteExtension
• Add focused selector to enter_text
• Add version compatibility check on connect
• Decouple logging from marionette_flutter into marionette_logger and marionette_logging
• Improve ScrollSimulator with reverse scrolling direction and configurable max scroll attempts
• Fix tap matching widgets behind modal route barriers
• Fix onChanged not being called on enter_text
• Fix pointer device collision with macOS mouse cursor
• Fix extracted text not being exposed in interactive elements
• Ensure correct status code range in MarionetteExtensionError

0.3.0

• Add screenshot resizing configuration
• Fix SIGTERM handling on Windows
• Add GitHub Copilot support
• Require Dart 3.10 for the MCP server
• Fix hot_reload tool response handling

0.2.4

• Add CI workflow with analyze, format, and version checks
• Add version generation script with validation and robustness
• Replace example symlinks with actual directories in both packages
• Improve project structure and dependencies

0.2.3

• Add example symlinks for improved pub.dev package scoring
• Fix documentation lints for angle brackets in code references

0.2.2

• Add compatibility for Copilot initialize call
• Make configuration parameter optional in MarionetteBinding.ensureInitialized()

0.2.1

• Update README

0.2.0

• Support tapping by widget type
• Support tapping by coordinates
• Return more info in get_interactive_elements
• Add hot reload tool, and return logs only since last hot reload
• get_interactive_elements returns only hittable elements now
• The tools have better naming to conform to the MCP implementations
• scroll_to does not assume that the widget is in the tree
• Text is a stopwidget now

0.1.0

• Initial version of Marionette MCP
• Support for getting the interactive widget tree
• Support for entering text
• Support for tapping and scrolling
• Support for getting logs from logging package