fluttertest_ai 0.3.1

A local-first CLI that analyzes Flutter apps and generates conservative, runnable tests without an API key.

Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

0.3.1 - 2026-10-07

Fixed

• Capped the test dev-dependency to >=1.25.8 <1.31.2 so pub get cannot resolve test_api 0.7.13+, which requires Dart SDK ^3.11.0 (language version 3.11). This restores a working dart test toolchain on Dart SDKs capped at language version 3.10.

0.3.0 - 2026-10-06

Added

• StateManagementAdapter gained five new members so an adapter can drive real test-environment setup instead of only wrapping a widget harness: appWidgetFor({required bool cupertino}) (the root app widget name to pump inside), requiresBindingInit (whether setUpAll needs TestWidgetsFlutterBinding.ensureInitialized()), setUpStatements(...)/ tearDownStatements() (raw Dart statements for the generated setUp/ tearDown), and registrationFor(RequiredDependency) (how to register one dependency). All five have safe no-op/default-MaterialApp defaults, so every existing adapter still compiles unchanged. GetxAdapter is the first real implementation: it now pumps GetMaterialApp/ GetCupertinoApp, requires binding init, sets Get.testMode = true and calls Get.reset() in setUp/tearDown, and registers a Fake<TypeName>() for every RequiredDependency with mustBeRegistered == true (alphabetical, de-duplicated by type name). This targets the three largest failure classes seen against a real GetX project: "X not found" DI lookups, contextless-navigation null checks, and uninitialized-binding errors. GetxAdapter no longer extends GenericAdapter. The generated test harness that consumes these five members (appWidgetFor/setUpStatements/tearDownStatements, and the Fake<TypeName> class declarations registrationFor depends on) is TestHarnessBuilder, added below and wired into the generation pipeline in this same release.

• Added TestHarnessBuilder (lib/src/generation/test_harness_builder.dart), which builds the source for a single shared, user-editable test/fluttertest_ai_harness.dart file instead of repeating environment setup in every generated test. For a given ProjectAnalysis and StateManagementAdapter it emits: a Fake<TypeName> class (using Fake from package:flutter_test, so no mockito/mocktail dependency is ever required in the target project) for every must-be-registered dependency with a resolvable import — types with no resolvable import are skipped and reported as a note rather than emitting a broken reference; setUpTestEnvironment()/tearDownTestEnvironment(), which call TestWidgetsFlutterBinding.ensureInitialized() when the adapter requires it, then a capped (10), best-effort, try/catch-guarded call to any structurally-matching static zero-arg initialize/init/ ensureInitialized method found on a class whose name suggests global/app setup (theme, typography, tokens, config, locator, DI), then the adapter's own setup/teardown statements; and buildTestApp(Widget child), which wraps child in Scaffold inside the adapter's app widget (GetMaterialApp for GetX, etc.), picking Cupertino only when strictly more analyzed files use it than Material. Output is deterministic for a given input. Wired into the generation pipeline later in this release — see below.

• DartSourceAnalyzer now detects dependency-injection requirements from unresolved AST syntax: Get.find<T>(), Get.put(...)/Get.lazyPut(...), Get.isRegistered<T>(), extending GetView<T>/GetWidget<T>, context.read<T>()/ref.read<T>(), context.watch<T>()/ref.watch<T>(), Provider.of<T>(context), and BlocProvider.of<T>(context). Each finding is exposed as a RequiredDependency (type name, DependencyKind, and a best-effort resolved importUri) on the new DartFileInfo. requiredDependencies. This detection feeds both TestHarnessBuilder and UnitTestGenerator, added later in this release. Added to close the single largest cause of generated-test failures observed against a real GetX project: calls like Get.find<AuthService>() throw at runtime when nothing registered AuthService first, and the generator previously had no way to know a widget/controller needed a dependency registered before construction.

• TestHarnessBuilder is now wired into the generation pipeline: DartTestWriter.write emits test/fluttertest_ai_harness.dart once per run, whenever the plan contains at least one widget test, obeying the same handwritten-preservation and --force rules as every other generated file. WidgetTestGenerator now imports the harness with a relative path computed from the test file's own location (correct at any nesting depth under test/), calls setUp(setUpTestEnvironment) / tearDown(tearDownTestEnvironment), and pumps via buildTestApp(Widget()) instead of an inline MaterialApp(home: ...) / adapter.wrapHarness(...) expression. The adapter's own extraImports (e.g. package:get/get.dart) are no longer repeated in each generated widget test file, since the harness is now the only place that needs them. This closes the two largest GetX failure classes observed in the field (missing DI registration, missing GetMaterialApp ancestor) by construction, for every generated widget test.

Fixed

• TestHarnessBuilder copied a RequiredDependency.importUri straight from the declaring source file's own import list, which may be relative ('../../data/services/auth_service.dart', 'profile_controller.dart'). A relative import is only meaningful relative to the file that wrote it; copied as-is into test/fluttertest_ai_harness.dart, it resolved to a path that does not exist, so the harness failed to compile and every generated test that imported it failed to load. The harness builder now resolves each dependency's import by finding its declaring class among the analyzed project files and computing a package: import from that file's own location (reusing templates.packageImport), falling back to the dependency's own best-effort import only when it is already a package: URI, and otherwise skipping the dependency with a note. A relative import can no longer reach the harness; DartSourceAnalyzer itself also no longer records a relative import as importUri at all, for the same reason. UnitTestGenerator's own harness-wireable check was updated to use the same project-wide resolution (previously it checked only the single file's own imports), so a dependency declared elsewhere in the project is not incorrectly treated as unresolvable.

• TestHarnessBuilder always registered a Fake<TypeName> for a must-be-registered GetX dependency, even when the real type was a plain, publicly-constructible class declared in the project. Fake throws UnimplementedError on every member access, so a widget depending on that fake (e.g. through GetView.controller) failed with a different error instead of actually being testable. The harness builder now registers the real type (Get.put<T>(T(), permanent: true)) whenever its declaring class is in this project, publicly constructible, and not abstract, and falls back to Fake only for external-package types, abstract types, or types with required constructor arguments. StateManagementAdapter.setUpStatements/registrationFor gained an optional DependencyRegistrationHint (instantiation expression + controller flag) so an adapter can make this choice; GetxAdapter also now registers non-controller dependencies before controller (GetxController/GetxService) dependencies, since a controller's own field initializer may call Get.find<SomeService>() at construction time and would otherwise fail if the service were not registered yet.

• fix could hang for an effectively unbounded amount of time on a large target project (observed: ~1500 generated test files, 50-100 MB of flutter test output). Four separate causes, all fixed together:

  • GeneratedTestRepairer.repair called failureOutput.contains(...) four times per candidate file, inside the per-file loop. With a multi-MB failureOutput and hundreds of files this was effectively O(files * output length). The four checks are now computed once, before the loop, as hoisted boolean flags (RepairResult also gains a scanned count for the new per-attempt summary below).
  • FlutterTestRunner.run used Process.run, which buffers all child output and prints nothing until the process exits — on a run that takes tens of minutes, the terminal looked frozen. It now uses Process.start and streams stdout/stderr line-by-line through an optional onProgress(String line) callback (both CLI commands pass print), draining both streams concurrently via Future.wait so neither the ~64 KB OS pipe buffer nor the child process can block.
  • Nothing bounded the run, so a wedged flutter test (pending timers, tickers, contextless navigation — all plausible at this scale) hung forever. run and fix both gained a --timeout <seconds> option (default 1800 = 30 minutes), implemented in FlutterTestRunner.run by racing process.exitCode against the timeout, escalating from SIGTERM to SIGKILL after a grace period, and returning a TestRunResult with exit code 124 (never a false success) instead of throwing.
  • Retained process output is now capped to a 1 MB head + 1 MB tail (with an elision marker noting how many characters/lines were dropped) instead of holding, sanitizing, and re-encoding a 50-100 MB string several times over. flutter test is invoked with the directory form (flutter test test) instead of one argument per file once there are more than 200 generated files, or whenever every discovered test file is already a generated one, to avoid exceeding ARG_MAX. Both the runner and the repairer now check only the first 4 KB of a file for the generated-ownership marker (it always lives in the file header) instead of reading the whole file, and walk test/ with followLinks: false.
  • fix now prints Attempt N of M: scanned <n> file(s), repaired <n> file(s) before each repair pass's existing ✓/– lines, so a long run shows visible progress instead of silence between attempts.

• TestHarnessBuilder excludes any RequiredDependency whose typeName starts with _ from both the emitted Fake<TypeName> class and the container-registration list: a library-private type cannot be implemented (extends Fake implements _Foo) from the harness file, which lives in a different library, so this previously emitted non-compiling output. The exclusion is reported as a note.

• The unit-test generator addressed three failure classes measured from a real GetX project run (527 + 37 + 140 of 3013 observed failures):

  • ClassInfo.isFlutterState is now true for a class extending State, ConsumerState, or HookState (matched on the superclass, never the class's own name, so an unrelated domain class merely named ...State is unaffected). LogicAnalyzer.pureMethodClasses/lifecycleClasses exclude these — initState/dispose can never work outside the element tree that creates a State — and LogicAnalyzer.flutterStateSkippedClasses reports the skip, surfaced by TestPlanBuilder into the plan's skipped list.
  • A class whose file requires an unregistered dependency-injection type (Get.find<T>(), context.read<T>(), ...) is no longer constructed bare. UnitTestGenerator now wires the shared harness's setUpTestEnvironment/tearDownTestEnvironment (imported with a relative path computed by the new templates.harnessImport, shared with the widget generator) when every must-register dependency has a resolvable, public fake to register, and otherwise skips the controller/state-container instance tests with a note naming the dependency — registering only some of them would just trade one failure for another. Separately, onInit/onReady/onClose on a GetxController/GetxService/Bloc/Cubit are never turned into a bare Controller().onInit() smoke test, since those hooks only fire meaningfully under a framework-driven lifecycle; a plain synchronous method on the same controller remains testable.
  • A callable name-shaped like a JSON decoder (*FromJson, *FromJsonString, decode*) taking a single String parameter is no longer called with 'sample' or '' — both are guaranteed-invalid JSON and jsonDecode throws on either. ValueFixtures now recognises this shape and supplies a confidently-valid minimal literal ('{}', or '[]' when the return type is list-shaped) for the typical-input case, and reuses the same tolerant accept-either-outcome assertion _fromJsonTest already uses (0% failure rate) for the empty-input case.

0.2.1

Fixed

• Widget-test generation asserted that every static string found inside a Text(...) was visible immediately after the first pump(), with no check for whether that Text was conditionally rendered. Any text behind a collection-if, an if/else, a ternary, a ??, a loop, or a switch produced a find.text(...) assertion that failed the instant the gating condition started out false (e.g. if (_submitted) Text('Signed in') when _submitted starts false) — a pattern common to conditionally-rendered error/empty/loading/success states in real apps. DartSourceAnalyzer now walks the AST ancestor chain of every candidate node up to its enclosing function/method body, and only feeds visibleTexts, keys, buttonTypes, textFieldTypes/hasTextField, hasForm, and loadingIndicators when the node is unconditionally built. The same conditional-render bug affected keys (findsOneWidget is even stricter than the text case's findsWidgets), buttons, text fields, forms, and loading indicators, so all six signals are fixed together rather than only the text case.

0.2.0

Corrective release. 0.1.0 generated no unit tests at all, and only a single "renders without crashing" widget test per screen. Both generation paths are fixed and verified end to end against real Flutter applications.

Fixed

• State-management detection now matches structured, tokenised evidence (exact dependency names, exact package: import package-name segments, exact ClassInfo.superclassName matches, exact creationNames widget constructors) instead of substring-matching a lowercased blob of all imports/class names concatenated together. The previous approach let the GetX token get match package:flutter/widgets.dart (get is a substring of "widgets"), misclassifying any Flutter project as GetX. Confidence now scales with the number and diversity of independent evidence kinds (dependency, import, class inheritance, widget usage, behavior), and precedence between competing frameworks is resolved deterministically by comparing confidence rather than by hardcoded map iteration order.
• setState is no longer inferred merely from "a widget class exists somewhere in the project" — it now requires real evidence (DartFileInfo.usesSetState, and/or a class extending State). Projects with only StatelessWidgets and no setState() calls now correctly resolve to unknown/genericAdapter.
• Widget test generator now consumes the already-computed source analysis (visible text, keys, button/text-field/form/loading signals) instead of re-parsing source and emitting only a render smoke test. Generated tests now assert on static UI text, stable keys, button/text-field presence, loading indicators, and (when unambiguous) input/tap/form-submit interactions, using only stable selectors (keys, text, types).
• Widget test harness selection now delegates to the adapter registry (StateManagementAdapters.forFramework) instead of a hardcoded Riverpod check, so all nine state-management adapters are used for widget-test harnesses, not just Riverpod and the generic fallback.
• Widget tests now use the generatedRegionBegin/generatedRegionEnd markers, so --force correctly refreshes only the generated region.
• Widgets that read inherited state unsafely for their detected framework (WidgetAnalyzer.requiresUnsafeState) are now actually skipped with an explanatory note, matching the behaviour already documented in report limitations text.
• Fixed a source-analysis gap where widget/form/button/loading-indicator detection only matched const/new-prefixed constructor calls; bare calls (e.g. ElevatedButton(onPressed: _increment, ...), which cannot be const when an argument like an instance-method tear-off isn't a compile-time constant) were invisible to analysis. This affected nearly all real-world widget code, since most non-trivial widget instantiations aren't const.
• Unit-test generation was fixed so it reliably produces a compiling test file for every eligible target (typed top-level functions, static and instance methods, getters, fromJson factories, and state-container lifecycle smoke tests), synthesizing typical, empty, and null-input argument lists only where a safe value is known, and deduplicating argument lists that would otherwise produce byte-identical test bodies.
• Generated unit-test files now import package:flutter_test/flutter_test.dart directly, so a generated unit test runs standalone with zero changes to the target project's pubspec.yaml, instead of depending on an import that generation previously omitted.
• Unit-test target selection (LogicAnalyzer.pureMethodClasses) was broadened from gating eligibility on a class name matching a conventional utility-class pattern (Validator, Formatter, Mapper, Converter, Parser, Helper, Calculator, Utils/Tools) to a structural check: a class is now an eligible unit-test target if it is publicly constructible, is not a widget, and exposes at least one public, synchronous, non-operator method or getter with synthesizable parameters — regardless of its name. Naming convention is now only a confidence signal (hasConventionalName), not a gate. Classes that are structurally eligible but whose name or file imports strongly suggest external I/O (Service, Repository, Client, Api, Http, Database, Db, Gateway, DataSource; or an import of dart:io, http, dio, shared_preferences, sqflite, or firebase) are excluded and reported via a recorded skip reason instead of being silently dropped.
• Deterministic repair of a BlocBuilder widget test missing its BlocProvider ancestor was fixed: the repairer now extracts the bloc's type from the BlocBuilder<...> generic, verifies the type is actually constructible from the source file before rewriting anything, wraps the harness in BlocProvider<T>(create: (_) => T(), child: ...), and normalizes any package:<project>/lib/... import produced by generation down to the correct package:<project>/... form.

0.1.0 - 2026-09-13

Added

• Initial local-first Flutter test analysis, generation, execution, repair, and reporting MVP
• CLI commands: doctor, analyze, generate, run, fix, report
• AST-based source analysis using the Dart analyzer package
• State management detection for: setState, Provider, Riverpod, BLoC/Cubit, GetX, Redux, MobX, ValueNotifier, Signals, and custom/unknown
• Dedicated test harness adapters for each supported state management approach
• Generic fallback for unknown state management with behavior-based widget tests
• Unit test generation for: pure functions, validators, formatters, JSON models, utility classes, and state containers
• Widget test generation with proper test harnesses (MaterialApp, ProviderScope, BlocProvider, etc.)
• Conservative test plan with confidence scores and skip reasons
• Safe test writer that never modifies lib/ or overwrites handwritten tests
• Generated-test ownership marker for safe refresh and repair
• flutter test execution with machine-readable (JSON events) and human-readable output parsing
• Deterministic test repair for common failures (missing imports, harness wrappers, finder issues)
• Sanitized report generation (Markdown and JSON) with credential redaction
• Optional AI provider interface with disabled-by-default implementation
• OpenAI-compatible provider stub (opt-in, disabled by default)
• Comprehensive test suite with fixture projects for each supported state management approach
• Support for Windows, macOS, and Linux