flutter_cockpit_test 4.0.49

Cockpit-powered Flutter integration tests with native evidence and AI-first selectors.

flutter_cockpit_test

Write normal Flutter integration tests with Cockpit's real locator, control, evidence, and diagnostics engine.

English · 简体中文

flutter_cockpit_test is a development-only test facade. It keeps Flutter's official integration_test runner and adds the parts that flutter_test cannot provide by itself: Cockpit's source-friendly Element selectors, real hit-tested actions, lazy-list reveal, compact snapshots, native screenshots, recording, viewport control, and explicit host/system actions.

Install

Add it to the development shell or test-only package, never to production application code:

flutter pub add --dev flutter_cockpit_test

The package is intended for a non-published cockpit/ shell that already uses flutter_cockpit. It does not depend on the Cockpit CLI, daemon, MCP server, or any secret store.

Quick start

import 'package:flutter/gestures.dart';
import 'package:flutter/material.dart';
import 'package:flutter_cockpit_test/flutter_cockpit_test.dart';

void main() {
  cockpitTestWidgets(
    'creates a task',
    app: buildDevelopmentApp,
    body: (cockpit) async {
      await cockpit.tap('New task');
      await cockpit.type('Buy milk', into: 'Task title');
      await cockpit.tap('Save');
      await cockpit.expectText('Task created', 'Task created');
    },
  );
}

Widget buildDevelopmentApp() {
  return const MaterialApp(home: TaskEditorScreen());
}

The helper wraps a plain Flutter widget in FlutterCockpitApp. If the builder already returns FlutterCockpitApp, it is mounted as-is and its existing Cockpit root is reused. This makes migration from an existing development shell incremental.

Selectors use the same syntax as cockpit dev:

await cockpit.tap('#save');
await cockpit.hover('More options');
await cockpit.tap(null, at: const Offset(400, 300),
    device: PointerDeviceKind.mouse, buttons: kSecondaryButton);
await cockpit.tap('Dialog >> FilledButton["Continue"]');
await cockpit.type('hello', into: '@message');
await cockpit.scroll('Settings >> Text["Advanced"]', align: 'center');
await cockpit.wheel(
  target: '#list',
  delta: const Offset(0, 120),
  steps: 2,
);

Plain text is exact. Use #id, @key, widget type, ancestor chains, and multiple conditions when source context gives you a stronger locator. No business Key or Semantics changes are required for Cockpit's Element plane.

The facade covers the complete Flutter interaction loop directly: pointer gestures (tap, hover, longPress, doubleTap, drag, fling, swipe, pinch, rotate, panZoom, multiTouch, wheel), text and keyboard input (type, clear, copy, paste, focus, setTextEditingValue, selectText, keyDown, keyUp, hotkey, press), controls and navigation (increase, decrease, showOnScreen, scroll, waitFor, waitForUi, waitForRoute, back, dismiss, dismissKeyboard), assertions and evidence (expectVisible, expectText, screenshot, snapshot, watch, execute). Each command advances Flutter's test clock through the same commit and reveal logic used by the live bridge, so route pushes and async UI updates do not need hand-written sleeps. Use cockpit.flutter when a test intentionally needs a Flutter-only matcher or custom pump.

All gestures are real hit-tested pointer events. Coordinate input is available as at when a target is not discoverable; device and buttons cover mouse, stylus, and touch-sensitive behavior without changing the app. wheel sends real PointerScrollEvent signals to Scrollable, custom Listener(onPointerSignal: ...), and trackpad-aware widgets. Its delta is applied per event; use steps, interval, device, or at only when the scenario needs them.

Every facade command has a 10-second default timeout. Override one known-slow call with timeout; the value must be positive and no longer than one hour:

await cockpit.waitForRoute('/reports', timeout: const Duration(seconds: 30));
await cockpit.tap('Refresh', timeout: const Duration(seconds: 5));

CockpitTestOptions.commandTimeout changes the default for all in-app commands. Native capture, recording, viewport, and capability calls use a separate two-minute default through nativeTimeout, and each native method also accepts its own timeout. A timed-out recording start requests cancellation before the timeout is reported.

Native and host capabilities

Flutter's test binding controls Flutter widgets. Cockpit's native facade covers the app-window capabilities exposed by the installed plugin:

final available = await cockpit.native.queryCaptureAvailability();
if (available) {
  final capture = await cockpit.native.captureScreenshot(
    name: 'task-created',
    timeout: const Duration(seconds: 30),
  );
  // capture.screenshot.artifact.relativePath identifies the evidence artifact.
}

final recording = await cockpit.native.queryRecordingCapabilities();
if (recording.supportsNativeRecording) {
  await cockpit.native.startRecording(
    name: 'task-flow',
    timeout: const Duration(minutes: 2),
  );
  // exercise the flow
  final result = await cockpit.native.stopRecording(
    timeout: const Duration(seconds: 30),
  );
  // result.artifact or result.sourceFilePath identifies the recording.
}

final resized = await cockpit.native.resizeViewport(width: 800, height: 600);

OS dialogs, app links, accessibility controls, and other host actions belong to Cockpit's system plane. They are intentionally explicit and supplied by the test host:

await cockpit.host.action(
  'openUri',
  parameters: {'uri': 'myapp://tasks/42'},
);

Configure CockpitTestOptions.hostCommand with a host adapter that forwards the command to Cockpit's public control API. Without that callback, host actions fail immediately with a useful configuration error; no external side effect is guessed or hidden.

Flutter APIs remain available

CockpitTester.flutter is the original WidgetTester. Use it for custom matchers, golden assertions, pump control, or APIs that are intentionally outside Cockpit's command surface. CockpitTester.execute accepts a complete CockpitCommand when a test needs a lower-level operation.

Every executed command is recorded into the in-app Cockpit session and a compact cockpit entry is merged into integration_test's reportData. Large snapshots and binary evidence are kept as artifacts; they are not dumped into test output.

Performance profiling

Profile an interaction with the same test clock and frame pipeline used by the app. Cockpit records raw vsync and raster-finish wall-time timestamps together with engine FrameTiming values (build, raster, vsync, total span, raster-cache usage, jank budget, and p50/p90/p99/worst values). On native Flutter targets it also captures the official integration-test VM timeline and GC events plus bounded process RSS samples; web reports the timeline and memory as unavailable instead of fabricating data:

final report = await cockpit.profile(
  () async {
    await cockpit.tap('#open-list');
    await cockpit.scroll('#list');
  },
  name: 'open-list',
  streams: const <String>['Dart', 'GC', 'Embedder'],
);
expect(report.summary.jankCount, 0);

Native captures also sample process RSS every 100ms by default and retain the start/end/min/max/average/peak/delta summary plus the bounded sample timeline. Set memory: false only when the extra process metric is irrelevant; use sampleEvery to trade sampling overhead for temporal resolution. Unsupported targets leave memory unavailable rather than reporting zero.

The complete bounded report is stored under cockpit.performance.open-list in IntegrationTestWidgetsFlutterBinding.reportData; the normal Cockpit result contains only the compact summary. dropped counts are explicit when a configured retention bound is reached; aggregates then describe the retained sample only. Empty phases omit duration aggregates rather than reporting a fabricated zero, and fps is omitted when the original engine timestamps cannot establish a strictly increasing cadence. The phase budget is derived from the target display refresh rate when Flutter exposes it, otherwise the report records the exact rounded 60Hz fallback interval (16,667µs). The report also records debug, profile, or release; debug timings are diagnostic and must not be used as release performance evidence. Never treat a missing or unavailable metric as zero.

Each cockpitTestWidgets run also records cold-start milestones in the compact cockpit.startup entry and in the HTML report: app build/mount, first pumped frame, and initial-ready time. The clock begins immediately before the app builder, so the values are honest Dart-harness measurements. Native process launch time is not inferred when the host cannot provide it.

Host-side integration_test_driver.dart files should import package:flutter_cockpit_test/flutter_cockpit_test_report.dart; this pure-Dart entrypoint exports the report models and HTML renderer without loading dart:ui.

Open a complete offline HTML report

CockpitTester.exportPerformanceHtml() writes one self-contained file for the captures completed by the current test. It includes a report switcher, frame pacing and budget chart, VM timeline lanes, phase percentiles, cache/GC pressure, searchable event arguments, paged frame/event tables, and the exact raw JSON payload. It works without a server or external assets:

final htmlPath = await cockpit.exportPerformanceHtml(
  title: 'Task flow performance',
  // path: 'build/reports/task-flow.html', // optional
);
// Pass htmlPath to a human or CI artifact collector.

The default path is a unique file under build/cockpit/performance/. For a custom host, CockpitPerformanceHtml.render(report) or CockpitPerformanceHtml.renderMany(reports) returns the HTML string without touching the file system. JSON remains the canonical machine-readable output; the HTML is the human-facing view.

Run

Run with Flutter's normal integration-test commands:

flutter test integration_test/task_flow_test.dart -d <device>

For Cockpit-managed development sessions, the same test can run from the development shell and its steps remain visible in the session timeline and artifacts. Case/Suite documents remain available for AI-generated, black-box, matrix, and cross-platform journeys; this package is the ergonomic Dart layer for Flutter source projects.