saturationStatus method

SaturationStatus saturationStatus({ SaturationThresholds thresholds = const SaturationThresholds(), })

Saturation snapshot for THIS run. See Anthropic Step 7. Capability suites that come back with a high saturatedTaskRatio are signaling that easy tasks should graduate to a regression suite and harder tasks should be added.

This is a single-run computation; for cross-run analysis (graduation candidates that have been mature in N consecutive runs, broken-task detection across runs) use SuiteHealthAnalyzer.

Implementation

SaturationStatus saturationStatus({
  SaturationThresholds thresholds = const SaturationThresholds(),
}) {
  final byTask = trialsByTask();
  final mature = <String>[];
  final stragglers = <String>[];
  for (final entry in byTask.entries) {
    final passes = entry.value.where((tr) => tr.allGradersPassed).length;
    final total = entry.value.length;
    final passRate = total == 0 ? 0.0 : passes / total;
    if (passRate >= thresholds.matureTaskPassRate) {
      mature.add(entry.key);
    } else if (passRate <= thresholds.brokenTaskPassRate &&
        total >= thresholds.minTrialsForBrokenJudgment) {
      stragglers.add(entry.key);
    }
  }
  final ratio = byTask.isEmpty ? 0.0 : mature.length / byTask.length;
  return SaturationStatus(
    saturatedTaskRatio: ratio,
    suiteSaturated: ratio >= thresholds.saturatedSuiteRatio,
    matureTasks: mature,
    stragglerTasks: stragglers,
  );
}