llm_eval 0.1.1
llm_eval: ^0.1.1 copied to clipboard
A Dart test harness that runs assertion checks over LLM outputs, adds an optional LLM-as-judge, and caches responses so CI stays deterministic.
0.1.0 #
Initial release.
EvalCase,EvalSuite, andEvalReportwith Markdown and JSON output.- Built-in checks:
contains,notContains,matches,isValidJson,predicate, and LLM-as-judge scoring withCheck.judge. ResponseCacheinterface in the core and a file-backedFileResponseCacheinpackage:llm_eval/io.dart(atomic writes) for deterministic reruns in CI.- Concurrent case execution with stable result order.
- Repeat runs with a flakiness rate.