parsingFunction property
Required. Python function for parsing results. The function should be defined within this string.
The function takes a list of strings (LLM responses) and should return either a list of dictionaries (for rubrics) or a single dictionary (for a metric result).
Example function signature:
def parse(responses: liststr) -> list[dictstr, Any] | dictstr, Any:
When parsing rubrics, return a list of dictionaries, where each
dictionary represents a Rubric.
Example for rubrics:
{
"content": {"property": {"description": "The response is
factual."}},
"type": "FACTUALITY",
"importance": "HIGH"
},
{
"content": {"property": {"description": "The response is
fluent."}},
"type": "FLUENCY",
"importance": "MEDIUM"
}
When parsing critique results, return a dictionary representing a
MetricResult.
Example for a metric result:
{
"score": 0.8,
"explanation": "The model followed most instructions.",
"rubric_verdicts": ...
}
... code for result extraction and aggregation
Implementation
final String? parsingFunction;