EvalRunnerOps extension

on

• EvalRunner

Methods

runSuite({required String runName, required EvalSuite suite, int concurrency = 8, int? trialsOverride, bool filter(EvalTask)?}) → Future<EvalRunReport>

Available on EvalRunner, provided by the EvalRunnerOps extension

Run all tasks in suite, honoring concurrency and per-task trialsPerRun. Returns the aggregated report.

runTask({required String runName, required EvalTask task, required String agentName, int? trialsOverride}) → Future<List<TrialResult>>

Available on EvalRunner, provided by the EvalRunnerOps extension

Convenience: run a single task. Useful for ad-hoc debugging or for rerunning a flaky task with extra trials.