flutter-skill

flutter-skill

Give your AI eyes and hands inside any app.
E2E testing bridge for Claude, Cursor, Windsurf — across 8 platforms.

pub.dev npm CI Stars License Agent Skill

Quick Start • Platforms • Features • Install • Docs

🎬 AI tests a full TikTok clone — 28 actions, zero test code

https://github.com/user-attachments/assets/d4617c73-043f-424c-9a9a-1a61d4c2d3c6

You: "Test the login flow — enter test@example.com and password123, tap Login, verify Dashboard"

AI Agent:
  1. screenshot()        → sees the login screen
  2. enter_text("email") → types the email  
  3. enter_text("pass")  → types the password
  4. tap("Login")         → taps the button
  5. wait_for_element("Dashboard") → confirms navigation
  ✅ Login flow verified!

No test code. No selectors. Just tell the AI what to test.

Quick Start

1. Install

npm install -g flutter-skill        # npm (recommended)
# or: brew install ai-dashboad/flutter-skill/flutter-skill
# or: dart pub global activate flutter_skill
# or: npx skills add ai-dashboad/flutter-skill  # Agent Skill (Claude Code, Cursor, Copilot, Cline, etc.)

2. Add to your MCP config (Claude Code / Cursor / Windsurf)

{
  "mcpServers": {
    "flutter-skill": {
      "command": "flutter-skill",
      "args": ["server"]
    }
  }
}

3. Add to your app (2 lines)

import 'package:flutter_skill/flutter_skill.dart';

void main() {
  if (kDebugMode) FlutterSkillBinding.ensureInitialized();
  runApp(MyApp());
}

4. Test — just talk to your AI:

"Launch my app, tap Sign Up, fill the form, and verify the success screen"

That's it. Zero configuration.

📦 More install methods (Windows, Docker, IDE extensions)
Method Command Platform
npm npm install -g flutter-skill All
Homebrew brew install ai-dashboad/flutter-skill/flutter-skill macOS/Linux
Scoop scoop install flutter-skill Windows
One-click curl -fsSL .../install.sh | bash macOS/Linux
Windows iwr .../install.ps1 -useb | iex Windows
Docker docker pull ghcr.io/ai-dashboad/flutter-skill All
pub.dev dart pub global activate flutter_skill All
VSCode Extensions → "Flutter Skill" All
IntelliJ Plugins → "Flutter Skill" All
Agent Skill npx skills add ai-dashboad/flutter-skill All (28+ agents)
🔧 Zero-config onboarding (auto-detect & patch your app) 
cd your-app/
flutter-skill init    # Detects platform, patches entry point, configures MCP
flutter-skill demo    # Launches built-in demo app to try it out

init auto-detects Flutter, iOS, Android, React Native, or Web projects and patches them automatically.

Platforms

flutter-skill works across 8 platforms with a unified bridge protocol:

Platform SDK Tests Status
Electron sdks/electron 75/75 ✅ Stable
KMP Desktop sdks/kmp 75/75 ✅ Stable
Tauri (Rust) sdks/tauri 75/75 ✅ Stable
.NET MAUI sdks/dotnet-maui 75/75 ✅ Stable
React Native sdks/react-native 75/75 ✅ Stable

| Android (Kotlin) | sdks/android | 74/75 ✅ | Stable | | Flutter iOS | flutter_skill (pub.dev) | 60/65 ✅ | Stable | | Flutter Web | flutter_skill (pub.dev) | 63/65 ✅ | Stable |

582/590 tests passing across all platforms (98.6% pass rate)

Each platform is tested against a complex social media app (tabs, feeds, forms, modals, 50+ elements) with 75 E2E tests covering: inspect, tap, enter_text, get_text, find_element, wait_for_element, scroll, swipe, screenshot, go_back, press_key, inspect_interactive, eval, and error handling.

Each SDK README has platform-specific setup instructions. The same CLI and MCP tools work for all platforms.

Platform setup examples

Web — add one script tag:

<script src="flutter-skill.js"></script>
<script>FlutterSkill.start({ port: 50000 });</script>

React Native — npm install:

npm install flutter-skill

import FlutterSkill from 'flutter-skill';
FlutterSkill.start();

iOS (Swift/SwiftUI) — Swift Package Manager:

import FlutterSkill
FlutterSkillBridge.shared.start()

Text("Hello").flutterSkillId("greeting")
Button("Submit") { submit() }.flutterSkillButton("submitBtn")

Android (Kotlin) — Gradle:

implementation("com.flutterskill:flutter-skill:0.7.5")
FlutterSkillBridge.start(this)

Electron / Tauri / KMP / .NET — see each SDK's README for details.

What Can It Do?

40+ MCP tools organized in 4 categories:

👀 See the Screen

  • screenshot — full app screenshot
  • inspect — list all interactive elements
  • get_widget_tree — full widget hierarchy
  • find_by_type — find by widget type
  • get_text_content — extract all text

👆 Interact Like a User

  • tap / double_tap / long_press
  • enter_text — type into fields
  • swipe / scroll_to / drag
  • go_back — navigate back
  • Native: native_tap, native_swipe

✅ Verify & Assert

  • assert_text / assert_visible
  • wait_for_element / wait_for_gone
  • get_checkbox_state / get_slider_value
  • assert_element_count

🚀 Launch & Control

  • launch_app — launch with flavors/defines
  • scan_and_connect — find running apps
  • hot_reload / hot_restart
  • list_sessions / switch_session
Full tool reference (40+ tools)

Launch & Connect: launch_app, scan_and_connect, hot_reload, hot_restart, list_sessions, switch_session, close_session

Screen: screenshot, screenshot_region, screenshot_element, native_screenshot, inspect, get_widget_tree, find_by_type, get_text_content

Interaction: tap, double_tap, long_press, enter_text, swipe, scroll_to, drag, go_back, native_tap, native_input_text, native_swipe

Assertions: assert_text, assert_visible, assert_not_visible, assert_element_count, wait_for_element, wait_for_gone, get_checkbox_state, get_slider_value, get_text_value

Debug: get_logs, get_errors, get_performance, get_memory_stats

Example Workflows

"Test login with test@example.com / password123, verify it reaches the dashboard"

"Submit the registration form empty and check that all validation errors appear"

"Navigate through all tabs, screenshot each one, verify back button works"

"Take screenshots of home, profile, and settings — compare with last time"

The AI agent figures out the steps. No test code needed.

Native Platform Support

Flutter Skill sees through native dialogs that Flutter can't — permission popups, photo pickers, share sheets:

Capability iOS Simulator Android Emulator
Screenshot xcrun simctl adb screencap
Tap macOS Accessibility adb input tap
Text input Pasteboard + Cmd+V adb input text
Swipe Accessibility scroll adb input swipe

No external tools needed — uses built-in OS capabilities.

Test Results

Every platform is tested against a complex social media app (4-tab navigation, 50+ scrollable items, forms, modals, search) with a comprehensive 75-test E2E suite.

75 tests per platform — full breakdown
Category Tests What's tested
Initialize 1 Protocol handshake
Inspect 5 Element tree, bounds, types, state changes
Inspect Interactive 5 Semantic refs, actions, bounds, cross-page
Tap 6 By key, text, coordinates, ref, invalid key/ref
Enter Text 6 Basic, empty, emoji/unicode 🌍世界, overwrite, long (500+ chars), error handling
Get Text 4 Counter, input readback, missing key, button label
Find Element 5 By key, missing, by text, partial text, bounds
Wait For Element 4 By key, by text, timeout, fast resolution
Press Key 10 Enter, Tab, Escape, Backspace, arrows, Ctrl+A, Delete, Home/End, invalid key
Scroll 6 Up, down, left, right, large distance, zero
Swipe 4 Up, down, left, right
Screenshot 2 Base64 encoding, image format (PNG/JPEG)
Navigation 8 Detail page, inspect after nav, interactive elements, go_back, round-trip
Logs 3 Get, clear, verify empty
Eval 2 Expression evaluation, return values
Error Handling 3 Unknown method, missing params, malformed request
Total 75

Test apps per platform:

Platform Test App Elements
Electron Social media app (HTML/JS) 60+ items, dark theme
Android Native Kotlin app with Fragments 50+ items, Material Design
KMP Compose Desktop app 50+ items, tab navigation
Tauri Web frontend (Rust backend) 55+ feed cards
.NET MAUI Simulated bridge with 6 screens 114 elements
React Native Real RN app on iOS simulator Self-registering components
Flutter iOS TikTok clone (10 feature modules) Full social media app
Flutter Web TikTok clone via bridge proxy Full social media app

Troubleshooting

Problem Fix
"Not connected" flutter-skill scan_and_connect to auto-find apps
"Unknown method" flutter pub add flutter_skill then restart (not hot reload)
No VM Service URI Add --vm-service-port=50000 to launch args
Claude Code priority Run flutter_skill setup for priority rules

📖 Full docs: Usage Guide · Architecture · Troubleshooting · Flutter 3.x Fix

📦 Install

🔌 IDE Extensions

📖 Docs

If flutter-skill saves you time, ⭐ star it on GitHub!

GitHub Sponsors · Buy Me a Coffee

MIT License

Libraries

flutter_skill
flutter_skill_semantic_refs
flutter_skill_web_stub