agentic_test 0.2.0

Test agents without paying for every run: record real model answers once and replay them offline, and score agent behaviour with repeatable evals.

Changelog

0.2.0

• Released with the rest of the framework at 0.2.0, which it now depends on. No changes to this package's API or behaviour.

0.1.2

Initial release. The version follows the sibling packages it ships with.

• Cassettes. RecordingChatModel records a real model's answers, ReplayChatModel plays them back offline, and cassetteModel chooses between them: it records when the file is missing or AGENTIC_RECORD=1 is set, and replays otherwise. Replay fails with a CassetteMismatchError naming the first difference when a prompt or tool changes, and verifyExhausted catches a run that makes fewer calls. A Redactor keeps secrets out of the file.
• Evals. EvalSuite runs EvalCases repeatedly against an agent and reports pass rates. Checks cover success, answer text, tools called and not called, step and token limits, custom functions, and a model-graded rubric. EvalReport renders as a summary, Markdown or JSON, and requirePassRate turns it into a test assertion.