evalctl

Local-first evals for agents, not just prompts.

evalctl scores what an agent does — not just what it says. The files it writes, the diffs it produces, the commands it runs, the exit codes it returns.

Eval cases are files. Scorers are shell commands. Every run is a sealed, rerunnable artifact you can diff, commit, and hand to the next agent — with no gateway, no dashboard, and no login. Deterministic scorers ship first; LLM judges are optional, never required.

Delegate execution to spoolctl for crash-safe async runs: bounded concurrency, retries, and resume-where-it-died across a crash. Model routing can delegate to inferctl, recorded per case.

Start with evalctl init.

Pre-release. Source and install instructions coming soon.