twk_test 0.1.0
twk_test: ^0.1.0 copied to clipboard
Tests for twk apps: the real app on a terminal in memory under a fake clock, input as a terminal's bytes, the screen read by an emulator.
twk_test #
The testing package for widgets of twk:
a test runs the real app — runApp's binding on a terminal in memory —
under a fake clock. Keys, the mouse and pasted text go in as the bytes a
terminal sends, decoded by the core's parser, and the screen is rebuilt
from the frames' bytes by a strict terminal emulator, which matchers and
finders read.
dart pub add dev:twk_test
Writing a test #
import 'package:test/test.dart';
import 'package:twk/twk.dart';
import 'package:twk_test/twk_test.dart';
void main() {
testTwk('Enter submits what was typed', (t) async {
await t.pumpWidget(const MyForm(), size: (40, 10));
await t.type('привет');
await t.press('enter');
await t.pump();
expect(t.lines.first, 'sent: привет');
expect(t.cell(0, 0).fg, Color.green);
});
}
testTwk(description, body) is a test of package:test, and body gets a
TwkTester. Every call of the tester returns a future to await. Each call
runs the app's microtasks before it returns.
The app #
pumpWidget(widget, size: (columns, rows), mode: ScreenMode.inline): the first call starts the app on an 80 × 24 terminal in the fullscreen mode, unlesssizeandmodesay otherwise. A later call shows the new widget in place of the old one (elements whose widgets match keep their state). Given anothersize, it resizes the terminal first. Then it pumps (see below).- The first call also takes
history, lines the screen shows before an app in the inline mode, which the app must not write over. It takeskittyKeyboard(the terminal has the kitty keyboard protocol on),backgroundRgb,multiplexer(the app runs inside tmux or GNU screen, so what it sends through them, the clipboard and notifications, comes wrapped for them) andnotifications(the protocol the terminal shows desktop notifications in, by default none:AppBinding.notifythen writes nothing, and withNotificationProtocol.osc777its bytes are interminal.frames),graphics(the protocol the terminal shows images in, by defaultGraphicsProtocol.none, which leaves their cells blank; through themultiplexeras a terminal's go) andcellPixels(a cell's size in pixels, width and height, as the terminal reports it, the emulator's too; without it the terminal reports none and the emulator's cells are 10 × 20) too, and passesmaxFps,maxInlineRows,clearInlineOnExitandexitOnCtrlCto the binding.Multiplexer,NotificationProtocolandGraphicsProtocolare twk_core's, whichpackage:twk/twk.dartexports with the rest aHeadlessTerminalis made with. quit()quits the app and runs it until it has ended: the terminal is restored and the tree unmounted.isRunningsays whether the app is still running.binding(theAppBinding),terminal(itsframes),logs(the app'sLogSink) andstderrLines(what the app wrote to stderr: its warnings and errors once it ended) are there to look at.act(fn)runsfnin the app's zone, as the app's own code runs, and pumps a frame: how a test changes what the app listens to (a controller, a notifier, a cubit) and sees the frame that follows. A timerfnstarts waits for the clock.
Time #
The app's timers, frames and the pause after a lone ESC wait for the clock,
a FakeClock (t.clock):
pump()moves the clock on by one frame gap (a second overmaxFps, 16.666 ms at 60): a frame asked for before the call has run when it returns.pump(duration)moves it byduration, running everything due by then in order: timers, frames, input timeouts.- Frames are spaced by the clock:
setStatecalls between two frames make one build,1 / maxFpsafter the frame before.pumpWidget(..., duration: Duration.zero)runs only the app's first frame, at time zero. t.clock.elapsedis the time since the test began.pumpUntilIdle()pumps one frame gap after another until the app is idle: a gap passes in which it writes no frame, and none of its timers waits. A frame the app asked for is such a timer, and so is the next tick of an animation. If the app is still busy aftertimeout(10 s on the clock by default), the call fails the test. The message says how many frames the app wrote and when its next timer is due, and shows the screen and the render tree. An app with a timer that never stops (a spinner, a poll) never becomes idle: pump such an app withpump(duration). One whose animations end (anAnimationControllerat the end of its run or stopped, a spinner removed) has no timer left and goes idle again, whicht.clock.pendingTimersshows.
Only timers and microtasks of the app are fake: Stopwatch() and
DateTime.now() read real time in the app too. The binding spaces frames
by AppBinding(clock:), which the tester sets to t.clock.stopwatch. Code
of the test itself keeps real time. The clock does not wait for real I/O:
a file the app reads arrives whenever the operating system delivers it.
Input #
type('text')sends the UTF-8 bytes of the text as one read, which the parser makes into a key for each character.input(bytes)sends raw bytes.press('ctrl+s')sends the bytes a terminal sends for each key, a read each. The keys are named as twk's key bindings name them (KeyCombo.parseSequence), one space between two keys ('g g','ctrl+x ctrl+s'). Each key is its modifiers and then its name, joined by+. The modifiers arectrl,alt,shiftandsuper, in any order, each once. A name is either one character, which stands for itself (a,?,я,+, which comes last:ctrl++), or one of these, in lower case:space,enter,tab,backspace,escape,up,down,left,right,home,end,pageup,pagedown,insert,delete,f1tof12. Only a character has a case:Aisshift+a, which types the capital, andctrl+Aisctrl+shift+a. Shift alone with a character that has no case (shift+1,shift+space) names no key.- A name that is not one throws a
FormatExceptionnaming the part that is wrong, and the name meant where that is plain (Enter,esc,cmd). - The event is what the parser makes of the bytes:
ctrl+iarrives as Tab, as on a real terminal. - Without the kitty protocol, some keys have no bytes of their own
(Ctrl+Shift with a letter, Ctrl+Enter, Super, Alt and Shift with a
character that has no case). Pressing one throws an
ArgumentError; a test of such a key passeskittyKeyboard: truetopumpWidget. - Escape without the protocol is a lone ESC, which the parser resolves
only after a pause. So
press('escape')moves the clock on by the escape timeout, 30 ms.
- A name that is not one throws a
paste(text)sends a bracketed paste.mouse(MouseKind.down, x, y, button: 2, modifiers: Mod.ctrl)sends an SGR mouse report, andclick(x, y)sends a press and a release. Neither works in the inline mode, where the terminal keeps the mouse for itself.resize(columns, rows)resizes the terminal. The app gets aResizeEvent, and the next frame repaints the screen.
A lone ESC typed with type('\x1b') stays pending in the parser. It
becomes the Escape key after pump(const Duration(milliseconds: 30))
(500 ms with the kitty protocol), unless more input comes first.
The screen #
linesgives the screen's rows as text, without the blanks that end them, with a wide glyph shown once.scrollbackgives the lines scrolled off the top in the inline mode, oldest first.cell(x, y)gives one cell:text(''for the right half of a wide glyph),fg,bg,ul(the underline color) andattrs. The colors and attributes are numbered asColorandAttrnumber them, with0for the terminal's default color:expect(t.cell(0, 0).fg, Color.red),expect(t.cell(0, 0).attrs, Attr.bold).fgColor,bgColorandulColorgive the colors asColors,Color.defaultColorfor0.cursoris the cursor's (column, row) andcursorVisiblewhether it is shown.vtis the emulator itself (Vt).written()is what the app wrote to the terminal, as text: the bytes of every frame decoded as UTF-8, a byte that is not as U+FFFD; andwritten(since: n)those of the frames fromterminal.frames[n]on. It is where a test looks for what the screen does not show: a mode set, the clipboard written (OSC 52), a notification, a scroll.
The emulator is strict: a sequence it does not know, text that wraps or
scrolls the screen, a write to the inline mode's history, an absolute
row in the inline mode, or an image it can't take (a kitty command
without q=2, a placement of an image not sent, not sized in cells or
without C=1; an image over text, past the screen or one that scrolls
it) throws when the screen is read. t.vt.images lists the images on
the screen (VtImage: the protocol, a kitty image's id and placement,
its bytes, the cells it still has and, for kitty, the part shown), top
to bottom.
Matchers #
The matchers of the screen take the tester: expect(t, containsLine('ok')).
-
containsLine(line): a row of the screen isline, aslinesgives the rows. Given a matcher (startsWith('sent:')), a row it matches. -
hasTextAt(x, y, text): the cells from columnxof rowyon showtext. As many cells are read astextis wide. -
styleAt(x, y, matcher): the cell at columnxof rowymatchesmatcher. The matchers of a cell arehasAttr(Attr.bold)(every attribute given; an underline matches only the same kind of underline),hasFg(Color.red)andhasBg(color). They take a cell on its own too:expect(t.cell(0, 0), hasFg(Color.red)). -
cursorAt(x, y): the cursor is shown, at columnxof rowy. -
matchesScreen(snapshot): the screen issnapshot, a string in the test whose lines are the rows of the screen from the top. The blanks that end a line and the empty lines that end the screen don't count, on either side:expect(t, matchesScreen(''' sent: привет [x] done '''));Indentation counts, so the snapshot's lines start at the start of the line in the source.
-
matchesScreen(snapshot, styles: true)compares styles too. Each run of cells of one style is in[style]…[/]:[bold]sent:[/] привет [red on #102030]ok[/].- A style lists the attributes first, by their names in
Attr(bold,curlyUnderline). Then come the color,onand the background, andul=and the color of the underline. Each is left out when it is the default. - A color has its name in
Color(red,brightBlack, andbrightRedand so on for 9 to 14),indexed(n)for the rest of the palette, or#rrggbbfor a 24-bit color. - A
[of the text is written[[. - A space shows only its background, unless it has an attribute that
shows on a blank (
Attr.visibleOnBlank: underlines, reverse, strike, overline). Between two cells of one style with that background, the space is in their run. Elsewhere its style is its background, and spaces with no background at the end of a row are left out.
- A style lists the attributes first, by their names in
-
matchesScreenFile(path): the screen is the one the file atpathholds (a golden file), relative to the package. A whole screen of a themed app is a run of styles in nearly every cell, which inline marks make unreadable, so the file keeps the text and the styles apart:screen 14×3, cursor hidden text │sent: привет │ │[x] 界 │ │ │ styles │aaaaa │ │bbbcbb │ │ │ legend a bold b red on #010203 c on #010203Each row is between
│, so that no editor drops its blanks; the text a wide glyph once, the styles a character a cell, a space for no style. The legend names each character's style asmatchesScreendoes, a hyperlink aslink=and its target, and a space that shows only its background by the background alone. The size and the cursor count; images do not. The characters are letters, digits and the rest of ASCII first, then letters of other scripts a column wide: about 2000, enough for a picture drawn in blocks; a screen of more styles fails.Run the tests with
TWK_UPDATE_GOLDENS=1to write the files from the screens instead (TWK_UPDATE_GOLDENS=1 dart test): the matcher then writes and matches. A style keeps the character the file gave it, so the diff of a file written again is what changed. One that fails lists the rows whose text differs, the rows whose styles differ (?for a style the file has no character for), and the first cells whose style differs, with both styles.
When a matcher of the screen fails, the message says what the screen has
instead: the line it lacks, the text or style found at the cell, where the
cursor is, or the rows that differ from the snapshot (- the snapshot's,
+ the screen's). Then it shows the screen, rows numbered and styles
marked as above, with the empty rows at the end summed up
(matchesScreen(styles: true) leaves it out: its diff shows it). Then
comes the widget tree, from the widget pumpWidget showed down, as
twk's debugDumpTree prints it (none before the first frame and once
the app ended), and last the render tree, from the binding's root render
object down, which alone shows each render object's offset, constraints
and flags.
Expected: the cell at column 0, row 0: fg red
Actual: TwkTester:<fullscreen, 20×3>
Which: has 'S' there, which is bold yellow
screen 20×3:
0│[bold yellow]Save[/] [reverse]Cancel[/]
(rows 1–2 blank)
widget tree:
Text ‹"Save Cancel", spans: 3› 20×3
render tree:
_RenderView ‹constraints: tight› 20×3 ◇
└─ RenderParagraph ‹"Save Cancel", spans: 3, constraints: tight› 20×3 ◇
◇ relayout boundary
Finders #
Finders look for the app's widgets: those of the widget pumpWidget
showed and those below it, in the order of the tree, not the binding's
own. They work only inside testTwk (a StateError outside it or before
pumpWidget) and find nothing before the app's first frame or once it
ended.
find.byType(Text)(exactly the type, not its subtypes),find.byKey(key),find.text('Save')(aText's string, or the spans of aText.richjoined) andfind.byWidgetPredicate(predicate, description:).find.descendant(of:, matching:), whatmatchingfinds below whatoffinds (not itself), andfind.ancestor(of:, matching:), what it finds above, the nearest first, up to the widgetpumpWidgetshowed..first,.lastand.at(index)pick one of what a finder finds.
findsNothing, findsOneWidget, findsWidgets and findsNWidgets(n)
count what a finder finds: expect(find.text('Save'), findsOneWidget).
A count that fails lists what was found, each with its cells
(Tile-[<'save'>] at (3, 3), 6×1), then the widget tree.
The tester reads the one widget a finder finds, a StateError naming
what it found when it found none or more: t.element(finder),
t.widget<T>(finder), t.state<T>(finder), t.renderObject<T>(finder)
(of the widget, or of the nearest widget below it that has one), and
t.rect(finder), its cells (x:, y:, width:, height:) as it was laid out
last: those of the screen, or in the inline mode of the region, row 0
its first. t.clickOn(finder, button: 0) clicks the cell in the middle
of those the widget shows, cut where a render object above it cuts them,
as a hit test does (a StateError when it shows none); like click, it
does not work in the inline mode.
Frames #
t.frames are the app's writes to the terminal so far, in order
(RecordedFrame): its number (from 1), its time on the clock, its bytes and the lines
of the screen it left, replayed on an emulator of their own that resizes
where the terminal did. A frame prints as frame 3 at 0.050 s, 120 bytes: and the screen as a failing matcher shows it.
A test that fails prints the last frames of its app that changed the
screen, 3 of them, or as many as testTwk(..., framesOnFailure: n) says
(0 for none).
t.writeTrace(path) writes the frames so far as an output trace, each at
its time on the clock, at the size the terminal had when the app started,
with no resizes and no inline history; addTearDown(() => t.writeTrace(path)) keeps the frames of a test that fails. In a
checkout of the twk repository,
dart run tool/replay.dart --fullscreen --trace file shows the last
screen, with --step each frame's and with --frame=N frame N's.
Errors and the end of a test #
- If the app ends with an error, the tester call during which it ended
throws that error (for example a
WidgetErrorfrom anInputListener). A build that fails does not end the app: it shows anErrorWidget, and the error is inlogs. - When
bodyreturns, an app still running is quit, and the rest of its frames go through the emulator. An error the app ended with that no call threw fails the test, and so do bytes the emulator rejects. If the test had already failed, its own error is the one reported.
Other libraries #
package:twk_test/vt.dart(Vt,VtCell,VtImage) is the emulator; twk_core's own tests are held to it. AVtCellprints as a snapshot with styles shows it ([bold]a[/]). A DCS or APC string ends at ST alone: a sixel image (DCS) and a kitty graphics command (APCG) are taken as a terminal takes them, what aDCS tmux;string holds as the terminal outside tmux takes it (an ESC ESC in it is one ESC, and an ESC not doubled is an error), and any other changes nothing on the screen. Images (images): kitty's, transmitted and kept by id (kittyImages, their bytes), placed, moved with their cells as they scroll and staying over them as text is written there, deleted, and freed byED 2; sixel images, sized by their raster attributes over cells ofcellPixelWidth×cellPixelHeight(10 × 20 by default), the cursor left after them as DEC mode 8452 says; iTerm2's (OSC 1337File=); the pixels of the last two taken away with what is written or erased in their cells. The queries of sizes in pixels (CSI 14 t,16 t,18 t) are taken, unanswered.screenFrames(width, height, writes)plays a program's writes to a terminal (TerminalWrites: an output or a resize, at a time) through the emulator, and gives aScreenFrameafter each output that leaves no synchronized update open: its time, size, rows, the cursor, and what the emulator could not take up to it. twk_dev'sCast.writesare a recording's, which so becomes a regression test.package:twk_test/vt_terminal.dart(VtTerminal) is a terminal backend in the inline mode that feeds its output to the emulator, below lines printed before the app ran.package:twk_test/cli.dart(FakeCliTerminal, andFakeClockandVtagain) is the terminal a test of a program on twk's console (package:twk/cli.dart) hands aCliorCli.run'sopen:: it captures what is written, answers the plain prompts a line each, types keys into the interactive ones, and shows the inline session's screen through the emulator.Shot(widget, columns, rows)shows a widget tree without an app: on a screen of its own, built, laid out and painted as a frame is, each frame written to an emulator (vt). No binding, clock or input: a frame runs whenpump(widget)orframe()is called.rows,colors(y)(each cell's glyph, color and background),attrs(y)andanyWideread the emulator back;owner,pipeline,rootandtopare the tree's.Shot.render(renderObject, columns, rows)shows a render object no widget made, and withunder: '.'fills the screen with that character first, which a repaint boundary that leaves a cell unpainted shows. Both takeambiguousWidth, the screen's and the emulator's.- The screen as text, as the matchers and their failure messages give
it:
rowText(vt, y, styles: true)andscreenRows(vt)(rows plain or with styles marked),describeScreen(vt)anddescribeApp(vt, root)(the screen numbered, and the render tree),styleOf(cell),markedCell(cell),colorName(color),colorOf(color)andattrNames(attrs). FakeStderrstands in for stderr underIOOverrides: it keeps each write (writes) and their lines (lines), each write after what itstagsays when it comes (FakeStderr(tag: () => 'on screen')).FakeClockworks on its own too.run(body)runs code in its zone, andelapse(duration)fires the zone's timers as the clock reaches them.flushMicrotasks()runs the zone's microtasks. It also lets the real event loop run its own, since code awaiting a future completed outside the zone resumes there.stopwatchis aStopwatchthat reads the clock.pendingTimerscounts the timers that wait, andnextTimerDueis the time the next of them fires at.- A program a test runs in a process of its own (a signal, a crash, a
pseudo-terminal, a VM service) is compiled once per suite, in
setUpAll, bycompiledProgram(path, packages: ..., cache: ...), and run asdart <kernel>: a kernel without the platform, kept in the directorycachenames (.dart_tool/twk_test/programs, say) with the names of the files it was compiled from and a hash of what they hold. A later run with them unchanged compiles nothing; one of them edited, added or gone, or another Dart, compiles it again.definesare the-Ddeclarations it is compiled with, a kernel of their own. - A process a test starts ends with the test, however the test ends
(it returns, fails, throws or times out):
startProcessandrunProcessareProcess.startandProcess.runwith a tear-down that kills the process if it still runs then and waits for its exit, with a bound.group: true, for a process that starts processes of its own (a shell,dartrunning an app, a compiler), starts it on Linux in a session and process group of its own (setsid), seen to lead it before it runs, and the tear-down kills that group too, even once the process itself has ended. Nothing else is signalled: no process is looked for by name, environment or listing, and every signal goes through one guard, which refuses a pid that is not the process started, 0, 1, -1, and any group but the one made for it, never the runner's own.
Contributing #
How the package is tested, and the checks a change passes, are in the twk repository: its CONTRIBUTING.md.
License #
MIT, copyright 2026 Andrii Ivanov: LICENSE.