llm_eval 0.1.1 copy "llm_eval: ^0.1.1" to clipboard
llm_eval: ^0.1.1 copied to clipboard

A Dart test harness that runs assertion checks over LLM outputs, adds an optional LLM-as-judge, and caches responses so CI stays deterministic.

0.1.1 #

  • Docs: tightened the README wording and visuals.

Changelog #

0.1.0 #

Initial release.

  • EvalCase, EvalSuite, and EvalReport with Markdown and JSON output.
  • Built-in checks: contains, notContains, matches, isValidJson, predicate, and LLM-as-judge scoring with Check.judge.
  • ResponseCache interface in the core and a file-backed FileResponseCache in package:llm_eval/io.dart (atomic writes) for deterministic reruns in CI.
  • Concurrent case execution with stable result order.
  • Repeat runs with a flakiness rate.
1
likes
0
points
924
downloads

Publisher

verified publisherdeveloperyusuf.com

Weekly Downloads

A Dart test harness that runs assertion checks over LLM outputs, adds an optional LLM-as-judge, and caches responses so CI stays deterministic.

Repository (GitHub)
View/report issues

Topics

#llm #ai #testing #evaluation #ci

License

unknown (license)

Dependencies

crypto

More

Packages that depend on llm_eval