taskPassRate property

double get taskPassRate

Overall task pass rate. Definition depends on suite kind:

• regression: task passes only if every trial passes (pass^N where N=trialsPerRun)
• capability: task passes if at least one trial passes (pass@N)
• mixed: same as regression

A trial "passes" for this metric according to EvalSuite.taskPassThreshold:

• 1.0 (default): binary — TrialResult.allGradersPassed
• < 1.0: partial credit — mean of non-null score values (TrialResult.meanScoreValue) must be >= the threshold

Implementation

double get taskPassRate {
  final byTask = trialsByTask();
  if (byTask.isEmpty) return 0.0;
  final passing = byTask.values.where((trs) {
    if (suite.kind == SuiteKind.capability) {
      return trs.any(_trialPassesForTaskRate);
    }
    return trs.every(_trialPassesForTaskRate);
  }).length;
  return passing / byTask.length;
}