ScenarioRun class

Orchestrates the evaluation of a single iteration of a scenario.

Call evaluate once with the model response to score, then call dispose (or use a try/finally) to persist the ScenarioRunResult to the EvaluationResultStore.

Annotations

• @Source(name: 'ScenarioRun.cs', namespace: 'Microsoft.Extensions.AI.Evaluation.Reporting', repository: 'dotnet/extensions', path: 'src/Libraries/Microsoft.Extensions.AI.Evaluation.Reporting/')

Constructors

ScenarioRun(String scenarioName, String iterationName, String executionName, Iterable<Evaluator> evaluators, EvaluationResultStore resultStore, {ChatConfiguration? chatConfiguration, EvaluationMetricInterpretation? evaluationMetricInterpreter(EvaluationMetric)?, ChatDetails? chatDetails, Iterable<String>? tags})

Creates a ScenarioRun.

Properties

chatConfiguration → ChatConfiguration?

Optional ChatConfiguration used by AI-based evaluators.

final

executionName → String

The execution name shared across all runs in one evaluation batch.

final

hashCode → int

The hash code for this object.

no setter, inherited

iterationName → String

The name of the iteration within the scenario.

final

runtimeType → Type

A representation of the runtime type of the object.

no setter, inherited

scenarioName → String

The name of the scenario.

final

Methods

dispose() → Future<void>

Writes the ScenarioRunResult to the EvaluationResultStore.

evaluate(Iterable<ChatMessage> messages, ChatResponse modelResponse, {Iterable<EvaluationContext>? additionalContext, CancellationToken? cancellationToken}) → Future<EvaluationResult>

Evaluates modelResponse against all configured evaluators and returns the aggregated EvaluationResult.

noSuchMethod(Invocation invocation) → dynamic

Invoked when a nonexistent method or property is accessed.

inherited

toString() → String

A string representation of this object.

inherited

Operators

operator ==(Object other) → bool

The equality operator.

inherited