buildEvaluationInstructions method
List<ChatMessage> ?
buildEvaluationInstructions(
- List<
ChatMessage> messages, - ChatResponse modelResponse,
- List<
EvaluationContext> additionalContext
override
Builds the evaluation instructions (system + user messages).
Return null to signal that a required context was missing.
Implementation
@override
List<ChatMessage>? buildEvaluationInstructions(
List<ChatMessage> messages,
ChatResponse modelResponse,
List<EvaluationContext> additionalContext,
) {
final response = modelResponse.text;
final prompt = '''
# Definition
**Fluency** refers to the quality of individual sentences in a response, measuring whether they are well-written and grammatically correct. A fluent response uses correct grammar, appropriate vocabulary, and clear sentence structure without errors.
# Ratings
## [Fluency: 1] (Incoherent)
Responses with severe grammar errors, unclear meaning, and language that makes the text almost impossible to understand.
## [Fluency: 2] (Disjointed)
Responses with frequent grammar errors, limited vocabulary, and simple sentences that reduce clarity.
## [Fluency: 3] (Mostly Fluent)
Responses with some grammar errors, adequate vocabulary, and generally understandable sentences.
## [Fluency: 4] (Fluent)
Responses with minor grammar errors, good vocabulary, and clear, well-formed sentences.
## [Fluency: 5] (Highly Fluent)
Responses with no grammar errors, sophisticated vocabulary, and complex, elegant sentences.
# Data
RESPONSE: $response
# Tasks
## Please provide your assessment Score for the previous RESPONSE based on the Definitions above.
- **ThoughtChain**: Think step by step. Start with "Let's think step by step:".
- **Explanation**: A very short explanation of why you think the input Data should get that Score.
- **Score**: An integer score (1–5) based on the definitions.
## Please provide your answers between the tags: <S0>your chain of thoughts</S0>, <S1>your explanation</S1>, <S2>your Score</S2>.
# Output
''';
return [
ChatMessage.fromText(ChatRole.system, _systemPrompt),
ChatMessage.fromText(ChatRole.user, prompt),
];
}