buildEvaluationInstructions method

@override List<ChatMessage>? buildEvaluationInstructions(List<ChatMessage> messages, ChatResponse modelResponse, List<EvaluationContext> additionalContext)

Builds the evaluation instructions (system + user messages).

Return null to signal that a required context was missing.

Implementation

@override
List<ChatMessage>? buildEvaluationInstructions(
  List<ChatMessage> messages,
  ChatResponse modelResponse,
  List<EvaluationContext> additionalContext,
) {
  final ctx = additionalContext.whereType<TaskAdherenceEvaluatorContext>().firstOrNull;
  final conversationHistory = messages.map((m) => '[${m.role.value}]: ${m.text}').join('\n');
  final response = modelResponse.text;
  final toolsSection = ctx != null && ctx.toolDefinitions.isNotEmpty
      ? '\nAVAILABLE TOOLS:\n${ctx.contents.map((c) => c.toString()).join("\n")}'
      : '';

  final prompt = '''
# Definition
**Task Adherence** measures how accurately the AI followed instructions in the conversation history and used available tools correctly.$toolsSection

# Ratings
## [TaskAdherence: 1] Completely ignored task instructions.
## [TaskAdherence: 2] Partially followed instructions with significant deviations.
## [TaskAdherence: 3] Mostly followed instructions with some notable deviations.
## [TaskAdherence: 4] Followed instructions well with minor deviations.
## [TaskAdherence: 5] Perfectly adhered to all task instructions.

# Data
CONVERSATION HISTORY:
$conversationHistory
RESPONSE: $response

# Tasks
## Score the RESPONSE's adherence to the task.
- **ThoughtChain**: Think step by step. Start with "Let's think step by step:".
- **Explanation**: A very short explanation of why you think the input Data should get that Score.
- **Score**: An integer score (1–5) based on the definitions.

## Please provide your answers between the tags: <S0>your chain of thoughts</S0>, <S1>your explanation</S1>, <S2>your Score</S2>.
# Output
''';
  return [
    ChatMessage.fromText(ChatRole.system, _systemPrompt),
    ChatMessage.fromText(ChatRole.user, prompt),
  ];
}