countsTowardTaskPassRate method

bool countsTowardTaskPassRate(TrialResult t)

Whether a single trial counts as passed for taskPassRate.

Report diffs use the same rule so a partial-credit threshold does not disagree with the headline task pass rate.

Implementation

bool countsTowardTaskPassRate(TrialResult t) {
  // Scores can exist on imported or manually constructed failed executions.
  // Partial credit must respect the same execution boundary as binary grading.
  switch (t.trial.status) {
    case TrialStatus.errored:
    case TrialStatus.timedOut:
    case TrialStatus.skipped:
      return false;
    case TrialStatus.passed:
    case TrialStatus.failed:
      break;
  }
  final threshold = suite.taskPassThreshold;
  // Default binary path preserves historical all-or-nothing semantics.
  if (threshold >= 1.0) return t.allGradersPassed;
  final mean = t.meanScoreValue;
  if (mean == null) return false;
  return mean >= threshold;
}