HumanGrader class abstract

Anthropic Step 5 / 8: human graders are gold-standard for subjective dimensions, used both for direct scoring and for calibrating LLM judges.

Concrete implementations connect to an application-provided HumanReviewQueue. The default grade flow is:

1. Push the trial to the queue (non-blocking).
2. Poll for a verdict; if absent, return a pending Score(value: null).

Implemented types

• Grader

Constructors

HumanGrader()

Properties

hashCode → int

The hash code for this object.

no setter, inherited

kind → GraderKind

Anthropic kind: code / model / human.

no setter, override

name → String

Stable name. Used as score key in reports.

no setter, inherited

passThreshold → double

If a Score.value meets or exceeds this threshold, Score.passed is true. Defaults to 1.0 (binary) but graders may override to support partial credit thresholds.

no setter, override

queue → HumanReviewQueue

Where to enqueue trials and where to read back human verdicts.

no setter

rubric → String?

Optional rubric prompt shown to human reviewers in the UI.

no setter

runtimeType → Type

A representation of the runtime type of the object.

no setter, inherited

Methods

grade({required Trial trial, required Transcript transcript, required Outcome outcome, required EvalContext context, ReferenceSolution? referenceSolution}) → Future<Score>

Compute a score for one trial.

override

noSuchMethod(Invocation invocation) → dynamic

Invoked when a nonexistent method or property is accessed.

inherited

toString() → String

A string representation of this object.

inherited

Operators

operator ==(Object other) → bool

The equality operator.

inherited