ModelGrader class abstract

Convenience base for LLM-as-judge graders.

Subclasses provide judgeClient (an LLM client to use as judge) and rubric (a prompt template). The rubric should include an explicit "Unknown" escape hatch (Anthropic Step 5) so the grader can return Score(value: null) instead of fabricating a score.

Implemented types

• Grader

Constructors

ModelGrader()

Properties

hashCode → int

The hash code for this object.

no setter, inherited

judgeClient → LLMClient

LLM client used for judging.

no setter

kind → GraderKind

Anthropic kind: code / model / human.

no setter, override

name → String

Stable name. Used as score key in reports.

no setter, inherited

passThreshold → double

If a Score.value meets or exceeds this threshold, Score.passed is true. Defaults to 1.0 (binary) but graders may override to support partial credit thresholds.

no setter, inherited

rubric → String

Rubric prompt template. Must include an "Unknown" escape hatch.

no setter

runtimeType → Type

A representation of the runtime type of the object.

no setter, inherited

Methods

grade({required Trial trial, required Transcript transcript, required Outcome outcome, required EvalContext context, ReferenceSolution? referenceSolution}) → Future<Score>

Compute a score for one trial.

inherited

noSuchMethod(Invocation invocation) → dynamic

Invoked when a nonexistent method or property is accessed.

inherited

toString() → String

A string representation of this object.

inherited

Operators

operator ==(Object other) → bool

The equality operator.

inherited