buildEvaluationInstructions method

  1. @override
List<ChatMessage>? buildEvaluationInstructions(
  1. List<ChatMessage> messages,
  2. ChatResponse modelResponse,
  3. List<EvaluationContext> additionalContext
)
override

Builds the evaluation instructions (system + user messages).

Return null to signal that a required context was missing.

Implementation

@override
List<ChatMessage>? buildEvaluationInstructions(
  List<ChatMessage> messages,
  ChatResponse modelResponse,
  List<EvaluationContext> additionalContext,
) {
  final response = modelResponse.text;
  final prompt = '''
# Definition
**Fluency** refers to the quality of individual sentences in a response, measuring whether they are well-written and grammatically correct. A fluent response uses correct grammar, appropriate vocabulary, and clear sentence structure without errors.

# Ratings
## [Fluency: 1] (Incoherent)
Responses with severe grammar errors, unclear meaning, and language that makes the text almost impossible to understand.

## [Fluency: 2] (Disjointed)
Responses with frequent grammar errors, limited vocabulary, and simple sentences that reduce clarity.

## [Fluency: 3] (Mostly Fluent)
Responses with some grammar errors, adequate vocabulary, and generally understandable sentences.

## [Fluency: 4] (Fluent)
Responses with minor grammar errors, good vocabulary, and clear, well-formed sentences.

## [Fluency: 5] (Highly Fluent)
Responses with no grammar errors, sophisticated vocabulary, and complex, elegant sentences.

# Data
RESPONSE: $response

# Tasks
## Please provide your assessment Score for the previous RESPONSE based on the Definitions above.
- **ThoughtChain**: Think step by step. Start with "Let's think step by step:".
- **Explanation**: A very short explanation of why you think the input Data should get that Score.
- **Score**: An integer score (1–5) based on the definitions.

## Please provide your answers between the tags: <S0>your chain of thoughts</S0>, <S1>your explanation</S1>, <S2>your Score</S2>.
# Output
''';
  return [
    ChatMessage.fromText(ChatRole.system, _systemPrompt),
    ChatMessage.fromText(ChatRole.user, prompt),
  ];
}