verdictGapOf function

String? verdictGapOf({ String? scenariosNote, String? previewsNote, Iterable<ComparedState> scenarioStates = const [], Iterable<ComparedState> previewStates = const [], int inconclusiveScenarios = 0, bool narrowed = false, })

Why a comparison's verdict is incomplete, or null when it is whole.

The exit-code question, and deliberately answered in the published reader rather than in fw alone: fw compare exits 1 on exactly this sentence, and a consumer's script reading index.json — or an agent reading the compare reply — has to reach the same conclusion from the same rule, not re-implement it. A consumer's workflow once grew fifteen lines of guards around a comparison that could not run recording itself as clean; the rule exists so nobody writes those lines again. ComparisonIndex.verdictGap asks it of a file.

Three gaps. A half that recorded a scenariosNote never produced a verdict — its harness would not build, say — and whatever next prevents a half from running is covered by the note the day it appears. A half whose rows all came out ComparedState.failed is the same gap wearing a different record: failed means neither side rendered, so a half made of nothing else answered no question about the branch — and it is near-always environmental, one missing native library rather than fifty-one simultaneous regressions. And a half of nothing but ComparedState.wasBroken is that gap's mirror: wasBroken means the base alone would not render, so not one pair of pictures was ever compared — and every row at once is near-always one base-side cause (a fresh base checkout whose native assets did not build, a pin the base cannot run under), which would otherwise read as "already broken before this branch" and exit 0. The genuine mass repair exists — a branch fixing a crash in main fixes every flow — and it pays one red check whose sentence names what to look at, then clears when it merges and the base moves on; the silent wrong verdict would be paid on every branch until someone noticed.

All, not any, for both: one pre-broken or one repaired flow among compared ones is a half that did its job, and its row is an ordinary finding. ComparedState.broke deliberately gets no such rule: mass breakage on the head side is the branch's problem either way — environmental or not, a dependency the branch added that will not build is the branch's to fix — and a verdict full of broke findings is already loud.

narrowed switches the all-one-state rules off — the note rule stands. Uniformity is evidence of environment only over the whole suite; a run narrowed to named entries compares the rows somebody picked, and picking the one pre-broken flow would otherwise turn its ordinary finding into a permanent gap no change on the branch can lift.

inconclusiveScenarios is how many of the scenario rows were replayed and not compared — see ScenarioComparison.inconclusive. A half in which that is every replayed scenario compared nothing, which is the same gap an all-failed half is, and near-always the same kind of cause: a host that could not run anything to the end.

Implementation

String? verdictGapOf({
  String? scenariosNote,
  String? previewsNote,
  Iterable<ComparedState> scenarioStates = const [],
  Iterable<ComparedState> previewStates = const [],
  int inconclusiveScenarios = 0,
  bool narrowed = false,
}) {
  if (scenariosNote case var note?) {
    return 'the scenario half produced no verdict — ${note.split('\n').first}';
  }
  // The previews half records one for the same reason the scenario half does,
  // and only since a comparison could span several packages: a package whose
  // catalog will not compile used to end the whole run, which is the right
  // answer when it is the only package and the wrong one when three others
  // compared cleanly. Its rows are simply absent, and absence is what a note
  // exists to explain.
  if (previewsNote case var note?) {
    return 'the previews half produced no verdict — ${note.split('\n').first}';
  }
  if (narrowed) return null;
  if (inconclusiveScenarios > 0 &&
      !scenarioStates.any(
        (state) => state != ComparedState.skipped && !_settledUnrun(state),
      )) {
    return 'the scenario half produced no verdict — all '
        '$inconclusiveScenarios replayed scenarios were inconclusive';
  }
  return _uniformGap('scenario', 'scenarios', scenarioStates) ??
      _uniformGap('previews', 'entries', previewStates);
}