Run a coding agent twenty times on the same task and you get several different outcomes. Reliability lives in the shape of that distribution, and specs, hooks and retries reshape it. A way to read the shape, a lens on what interventions do to it, and 400 runs of Claude Code on a small Laravel app.
A line in CLAUDE.md is advice that Claude follows most of the time. A hook is a script that runs every time. Five hooks for a Laravel app (Pint, migrations, destructive Artisan commands, Pest, session context), written partly as Artisan commands, and for each one the other places the same rule could live.
Why one run of an agent eval proves little, what pass@k and pass^k measure, how to give the agent a world it cannot damage, and how evals fit next to production monitoring and continuous integration.
Where the cases of an agent eval come from, when a check should be code, a model or a person, and why the outcome of a run matters more than the path the agent took to reach it.
Why an AI agent needs evals before its next prompt or model change, what makes an agent harder to evaluate than a single LLM call, and the task, run and grade loop every eval is built on.