flutter3d_testing 1.0.0-rc.1

Renders a flutter3d scene with no GPU and compares it against a reference image, so a game gets pixel regression tests on a machine with no display.

1.0.0-rc.1

• Breaking: QualityMeasurement is a class with the same setting, cost and flip, so a row can later carry more measures.

• Breaking: FrameSubject and RenderedFrame are classes. A replay subject and a frame builder return FrameSubject(scene:, camera:); RenderedFrame keeps pixels, width, height and drawCalls. Read the same way; a record could not have grown.

• A frame and a replay are drawn on the device a test names. renderFrame, testReplay and expectReplayMatches take device:, a BackendOpener (WebGpuDevice.open, GpuRenderBackend.open, an outside backend's own), and draw there instead of on the software rasteriser: a frame's device is disposed once its pixels are read, a replay's at the end of the run. drawOnce, the half both share, is exported for a test that holds its own device and renderer. Null keeps the software rasteriser.

• Breaking: renderFrame(clearColor:) takes a LinearColor, linear as RenderView.clearColor is, where it took an sRGB Vector4: LinearColor.fromSrgb(r, g, b, a) draws the same picture, and the default is still opaque black.

• Breaking: a replay steps the run through the engine's loop. ReplaySubject lost restore, step and save, and has loop (an EngineLoop built on ReplayStart.input, the genre installed and its run set) and genre (the GenrePlugin whose run the tape recorded) in their place. expectReplayMatches restores the tape's start as the genre's part of the loop's snapshots (loop.rewindTo(0, state: genre.loopStateOf(start))), plays the tape as the loop's playback one runSteps(1) at a time, and digests the genre's part of loop.capture() at each checkpoint — the one path a rollback takes, rather than a second copy of the game's step written for the test. A tape's start and checkpoints are still the run's own snapshot, so every recorded tape keeps passing. A subject whose loop reads another InputState, or lacks the genre, is refused before a step.

• Breaking: a boolean reads as a question, and no bool is positional. PacingReport.even is isEven. dart fix carries the renames.

• Breaking: PacingReport and replayPacing speak seconds (docs/CONTRACTS.md): millis and limitMillis are seconds and limit (0.05 by default), and p50, p99 and max are seconds. toJson still writes milliseconds, under the keys it always had.

• 1.0.0 is a promise: strict semver from there. This release candidate already keeps it. A patch fixes bugs and breaks nothing, a minor adds, and a break waits for a major. The whole public API is stable, with no experimental exceptions, and is held to the snapshot in api/. A deprecated name stays until the next major and for at least six months, and says what replaces it. CONTRIBUTING.md has the rules, and SUPPORT.md says which releases get fixes and on which platforms.

• MaterialProgramStage moved to flutter3d_app, which a game can depend on; this package carries flutter_test.

• testReplay turns a recorded run into a test that needs no GPU. testReplay('test/tapes/level.f3drun', start: …, digestAt: …, goldensAt: …) registers a test that plays the tape into the game on the software backend. At each step in digestAt (every checkpoint the tape holds, by default) the digest of the game's snapshot must equal the recorded one; a divergence names its step and the last checkpoint that agreed. At each step in goldensAt the frame is compared with test/goldens/<tape>-<step>.png. The game implements ReplaySubject (levelHash, restore, step, save, frame) and gets a ReplayStart with the tape, the input to read and the device to upload to. A step past the end of the tape, a digest where the tape has no checkpoint, a test that checks nothing and a level changed since the recording all fail before the first step. expectReplayMatches does the same for a Demo in memory, and readTape reads a .f3drun or says why it cannot.

• Breaking: FramePacing is PacingReport. flutter3d_core has a FramePacing of its own, the renderer's pacing policy, and a file that imported both met two types of one name. dart fix renames it.

• Breaking: ReplaySubject is an abstract base class, extended rather than implemented, so a member added in a minor release can arrive with a body. levelHash is null by default.

Its flutter3d_* dependencies ask for ^1.0.0.

0.8.0

• Two pictures can be compared the way an eye would. flip takes two 8-bit sRGB frames and flipLinear two in linear light, and both return a FlipResult: the per-pixel error map and its mean, nought for identical and one for as different as green is from blue. It is LDR-FLIP (Andersson, Nilsson, Akenine-Möller, Oskarsson, Åström and Fairchild, HPG 2020), ported line for line from the reference C++ so a value here is the value there, and a test holds it to the error maps that implementation publishes. The viewing condition defaults to defaultFlipPpd, about 67 pixels per degree. A count of moved pixels cannot say how much worse a frame looks: a lower render scale moves nearly every pixel a little and looks almost the same. The reference code's BSD licence is kept in third_party/flip.
• expectMatchesGolden takes flipBudget. Null, the default, compares as before. Given, the mean FLIP error replaces the share of differing pixels, for a frame meant to differ a little everywhere, such as a lower render scale or a stochastic effect against its sorted reference.
• measureQualityTable measures the table AdaptiveQuality in flutter3d_core chooses from. It renders each QualityScene at every setting of QualitySetting.grid on the device it is handed, gives each setting its own renderer and settle frames, then takes the fastest of samples interleaved frames, and returns a QualityMeasurement per setting: its cost relative to full quality and its mean FLIP against it. The time is the passes' GPU time where the device reports it and the renderer's wall clock where it does not. qualityTablesSource writes the source of flutter3d_core's tables/quality_tables.dart from a full grid per DeviceClass, with a flag for whether each was measured on that class. It is an offline tool: it draws every scene at every setting many times over.
• replayPacing plays a recorded run a frame at a time and times every frame. A frame is one step of the tape through onStep and one drawFrame, back to back with the CPU and GPU halves serialised, which errs long and pins each spike to the frame that caused it. gpuSettled closes a frame and waits for the device's onFrameComplete, so the GPU's share is counted. The result, FramePacing, keeps the median, the 99th percentile, the worst frame and its index, and every frame over limitMillis (50 by default); a frame rate would hide the one stop a player notices. repeats, rewind and warmUpFrames play a tape several times from its start or skip a level's first frames.

Its flutter3d_* dependencies ask for ^0.8.0.

0.7.1

• A zero tolerance was never byte for byte, and now the README says so. tolerance is the share of pixels allowed to differ, but a pixel only counted as differing when red, green or blue moved by more than 8 steps, and alpha was not compared at all. The README called that "zero". The defaults stay as they were, and expectMatchesGolden takes channel and alpha, so a test with byte-exact references passes channel: 0, alpha: true.

Its flutter3d_* dependencies ask for ^0.7.1, and it asks for vector_math ^2.4.3.

0.7.0

• Breaking for code that builds a RenderedFrame by hand. The record gained a fourth field, drawCalls, read off the FrameResult every render already produced. renderFrame fills it, and a test can now assert how many draws a scene cost as well as what it looked like. Code that only reads pixels, width and height is unaffected; a literal (pixels:, width:, height:) passed where a RenderedFrame is wanted no longer has the type.
• replayGolden: a recorded run, replayed to one step, held to a picture. It drives an InputState from a Demo's tape, calls the caller's onStep once per entry, draws the caller's frame at atStep through renderFrame and compares with expectMatchesGolden. A DigestTrace already says whether a replay reached the same numbers; this is for a renderer that changed under numbers that did not. The step is the caller's because a genre's simulation is not this package's to know. A tape that ends before atStep throws StateError, so a short recording is not reported as a wrong picture. flutter3d_sim ^0.7.0 is a dependency for it.
• MaterialProgramStage draws a material written as source on the software backend. It implements flutter3d_cpu's CpuFragmentShader over a MaterialProgram from package:flutter3d_core/formats.dart that has been through specialiseMaterial, and evaluates the same tree the GLSL emitter writes from. It lives here because flutter3d_cpu's lib/ may not reach the engine and the engine may not reach a backend. flutter3d_core ^0.7.0 is a dependency for it.
• The archive carries a skill for a coding agent, skills/flutter3d-testing-pixel-regression/, which a project depending on this package installs with dart run skills@ get.
• The floors on flutter3d, flutter3d_cpu and flutter3d_hardware are ^0.7.0. renderFrame, expectMatchesGolden and the tolerance of zero are unchanged.

0.6.0

• Floors, and no code. Drawing a frame through the software backend and holding it to a reference image works exactly as it did in 0.5.2. The floors on flutter3d, flutter3d_cpu and flutter3d_hardware are ^0.6.0 — and they matter more here than in most places, because a golden helper resolved against a different engine than the game under test compares two different renderers and calls the difference a regression.

0.5.2

• ^0.5.2 on both halves. This package is the one place the engine and the software backend are named together, so it is where "these two move as a pair" can be written at all — see flutter3d_backend 0.5.2 for what a mismatched pair does with morph targets.

0.5.1

• Its floors move to flutter3d and flutter3d_cpu at 0.5.1, both of them. This package pairs the engine with the software backend, and 0.5.1 changed a contract they share: what the surface buffer's alpha holds, and the members of the FogInfo block. Either half at 0.5.0 against the other at 0.5.1 draws a picture that is quietly the old one — the software backend reads a missing uniform member as zeros rather than refusing, so nothing says anything. Naming both floors is what closes it in both directions, which flutter3d_backend cannot do because it does not depend on the engine.

0.5.0

Breaking. The frame builder takes one object.

• FrameBuilder is FrameSubject Function(FrameRequest), so telling a builder the size it is drawing at, or which backend it is on, does not break every builder anybody has written.

0.4.0

• No changes of its own; the version moves with the workspace, whose sibling constraints name a single release. The README's closing section now says what the engine around this package is.

0.3.0

• First release. renderFrame draws a scene through the software backend, with no GPU, no driver and no display; expectMatchesGolden holds the result to a reference image, recording one when there is none.
• The tolerance defaults to zero, because a software rasteriser draws the same scene the same way twice and a test that allows drift has stopped watching it.