Skip to content

tags /

Evals

Every agent run is a sample

Run a coding agent twenty times on the same task and you get several different outcomes. Reliability lives in the shape of that distribution, and specs, hooks and retries reshape it. A way to read the shape, a lens on what interventions do to it, and 400 runs of Claude Code on a small Laravel app.

Without evals, every change is a guess

Why an AI agent needs evals before its next prompt or model change, what makes an agent harder to evaluate than a single LLM call, and the task, run and grade loop every eval is built on.