tarpit
Find the widget-test code that hangs your CI, before it runs, and make a
stalled testWidgets fail in seconds with a reason instead of sitting until
the timeout.
Why
testWidgets runs the test body in a FakeAsync zone. Timers there fire
only when the test pumps, and the results of real async work (files,
sockets, isolates, the engine's image pipeline) wait in the fake zone for a
pump that never comes. So one innocent line:
await File('test/fixtures/user.json').readAsString();
turns into a test that never finishes. Locally it looks like a frozen runner; in CI the job sits until the ten-minute test timeout or the job timeout, and the log does not say which line (flutter/flutter#5728, flutter/flutter#139870).
tarpit has two parts:
- A checker (
dart run tarpit) that reads your tests with the Dart analyzer and points at the lines that will stall, with why and how to fix each one. - A watchdog (
package:tarpit/flutter_test.dart) that fails a widget test which makes no progress for 30 seconds, says what state it was stuck in, and lets the rest of the file run.
Rules
| Rule | Severity | Reports |
|---|---|---|
fake-delay |
error | await Future.delayed(...), Future(...) or pumpEventQueue() |
real-io |
error | an awaited dart:io call: files, directories, processes, sockets |
isolate-hop |
error | an awaited compute, Isolate.run/spawn, ReceivePort read |
engine-async |
error | an awaited toByteData, toImage, getNextFrame, precacheImage |
blocking-sleep |
warning | sleep(...) from dart:io |
endless-settle |
warning | pumpAndSettle() right after pumping an indeterminate progress indicator |
asset-string |
info | an awaited AssetBundle.loadString (uses an isolate at 50 KB and up) |
bad-suppression |
warning | a tarpit:allow comment without a known rule id or a reason |
Only testWidgets (and tarpitWidgets) bodies are checked, and anything
inside tester.runAsync(...) is allowed. The same pattern inside a nested
closure the test may not await, or in a file whose imports do not resolve,
is reported as info rather than an error. doc/rules.md has
the details, examples and what each rule deliberately leaves alone.
Install
flutter pub add --dev tarpit
Or install the command once for all projects (with the dart that comes
with Flutter):
dart pub global activate tarpit
Run the checker
dart run tarpit # checks test/ (every *_test.dart)
dart run tarpit test/widgets # or given directories and files
tarpit # when activated globally
Run flutter pub get first: tarpit uses your project's resolved types, so
file.readAsString() is known to be dart:io and not your own repository
class. Output from the example project:
$ cd example && dart run tarpit
test/stalls_test.dart:22:11 error fake-delay
Awaits Future.delayed(...); its fake timer fires only when the test pumps, and the test is waiting instead of pumping.
fix: Advance fake time with `await tester.pump(duration)` instead. If you really need wall-clock time, wrap it in `tester.runAsync(...)`.
test/stalls_test.dart:28:24 error real-io
Awaits File.readAsString() from dart:io outside tester.runAsync; the result arrives on the real event loop and the test never resumes.
fix: Wrap the call in `await tester.runAsync(() => ...)`, or do the I/O in setUp/setUpAll, or use an in-memory fake (for example package:file).
test/stalls_test.dart:40:11 warning endless-settle
pumpAndSettle() right after pumping an indeterminate CircularProgressIndicator (line 38); the indicator animates forever, so this never settles.
fix: Use `await tester.pump(duration)` while the indicator is visible, give the indicator a `value` in the test, or wait for the content with a bounded loop of pumps.
Why:
fake-delay: testWidgets runs the body in a FakeAsync zone. Future.delayed, Future(...) and pumpEventQueue() schedule fake timers, which fire only when the test pumps. While the body is awaiting one, nothing pumps, so the await never returns.
real-io: dart:io completes on the real event loop, but the continuation is queued in the FakeAsync zone, which only drains when the test pumps. The test waits for I/O that has finished but is never delivered.
endless-settle: An indeterminate CircularProgressIndicator, LinearProgressIndicator, RefreshProgressIndicator or CupertinoActivityIndicator animates forever, so pumpAndSettle keeps pumping frames until its own timeout and then fails with "pumpAndSettle timed out".
3 findings (2 errors, 1 warning, 0 info) in 2 files.
$ echo $?
1
Options and exit codes
| Option | Meaning |
|---|---|
-f, --format text|json|sarif |
Output format (default text). |
--fail-on error|warning|info |
Lowest severity that makes the exit code 1 (default warning). |
-o, --output <file> |
Write the report to a file instead of stdout. |
--rules |
List the rules. |
--version, -h, --help |
Exit code 0: nothing at or above --fail-on. 1: findings. 2: bad
arguments, a path that does not exist, or a report that cannot be written.
--format json prints {"format": "tarpit-json", "formatVersion": 1, ...}
with one object per finding (rule, severity, path, line, column,
message, why, fix). --format sarif prints SARIF 2.1.0 for GitHub
code scanning:
- run: dart run tarpit --format sarif --output tarpit.sarif || test $? -eq 1
- uses: github/codeql-action/upload-sarif@v4
with:
sarif_file: tarpit.sarif
- run: dart run tarpit
Suppressing a finding
When a pattern is fine in one place, say why next to it:
await File('golden.txt').exists(); // tarpit:allow real-io runs only with the live binding
// tarpit:allow fake-delay,real-io this helper pumps from a fake timer
await waitForSync();
The comment works on its own line or on the line above. // tarpit:allow-file <rule-id> <reason> covers a whole file. The reason is required; without
one, or with an unknown rule id, the comment is ignored and reported as
bad-suppression.
The watchdog
Change one import in a test file:
import 'package:tarpit/flutter_test.dart'; // instead of package:flutter_test
Every testWidgets in that file now runs under a watchdog. Or use
tarpitWidgets(...) from package:tarpit/tarpit.dart for single tests,
and watchTester(tester, description, body) inside a custom wrapper.
A test that stops making progress (the fake clock does not move and no
frame is pumped) for Tarpit.stallAfter (30 seconds) fails with a
TarpitStall, and the next test runs normally. A test that keeps pumping is
never stopped, however long it takes. From flutter test in the example:
══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following TarpitStall was thrown running a test:
tarpit: "reads the name from a file" made no progress for 3.0s and was stopped.
real time in the test body: 3.0s
fake clock moved: 0.0s (it moves only when the test pumps)
frames pumped: 0
running tickers: 0
frame requested, not pumped: no
The test body is awaiting a Future that nothing in the FakeAsync zone of testWidgets will complete.
Usual causes:
- Future.delayed, Future(...) or pumpEventQueue() awaited instead of tester.pump(duration)
- file, socket or process I/O, compute() or Isolate.run() awaited outside tester.runAsync(...)
- engine image work (toByteData, toImage, getNextFrame) awaited outside tester.runAsync(...)
- a Completer or stream that waits for a platform channel, plugin or service the test does not
fake
Run `dart run tarpit` on the test file to find likely lines. If the test was inside tester.runAsync,
flutter_test still waits for that real work before it moves on. If the test is just slow without
pumping, raise the limit with stallAfter: or Tarpit.stallAfter.
Set the limit for a whole suite in test/flutter_test_config.dart:
import 'dart:async';
import 'package:tarpit/tarpit.dart';
Future<void> testExecutable(FutureOr<void> Function() testMain) async {
Tarpit.stallAfter = const Duration(seconds: 20); // Duration.zero turns it off
await testMain();
}
or per test with stallAfter:.
Engine image work awaited outside tester.runAsync (toByteData,
toImage, getNextFrame;
flutter/flutter#50783)
does not block the test isolate. The engine finishes and the bytes arrive,
but on the real event loop, outside the fake zone, so the test stops at its
next await, or never finishes after the body returns when that call was the
last one. The watchdog fails both; the second says "The test body returned,
but the test never finished". test/runtime/watchdog_e2e_test.dart checks
this on Flutter 3.47.7 (macOS 27.2) and in CI on Flutter 3.44.0 and 3.47.7
(Ubuntu 24.04).
What the watchdog cannot do:
- Inside
tester.runAsync, the failure is reported on time, but flutter_test waits for the real work before it starts the next test. experimentalLeakTestingis not passed through; use flutter_test'stestWidgetsfor those tests.
How it works
The checker builds an AnalysisContextCollection over the files, gets each
file's resolved AST from package:analyzer, finds testWidgets calls whose
body is a function literal, and walks the body: awaited expressions are
classified by the library and class of the element they call (dart:io,
dart:isolate, dart:ui, package:flutter), not by text. Without
resolved types it only reports what it can recognise by name, as info.
The watchdog runs a real timer in the root zone next to the test's fake
one. When the fake clock and the frame count stay still for the limit, it
reports a TarpitStall into the test's zone, which flutter_test's binding
handles like any other test failure, and drains the binding's pending
microtasks so the test completes. It keeps watching until the test's
tear-down, not just until the body returns.
Contributing
Issues and pull requests are welcome; see CONTRIBUTING.md.
New rules need a reproduction that hangs or fails a real widget test.
tarpit is the successor to imaji_test_hang.
License
MIT, see LICENSE.
Libraries
- checker
- The static checker behind the
tarpitcommand, for use from Dart code. - flutter_test
package:flutter_test/flutter_test.dartwith atestWidgetsthat runs under the tarpit watchdog.- tarpit
- Runtime side of tarpit: a watchdog for widget tests.