Working notes
Understanding Evals
Making sense of AI evaluations: what to test, how to test it, and what the results actually tell you.
What I’m exploring
I’m trying to understand evals as evidence about how a system behaves. Ordinary software still needs tests. With an agent, I also want to inspect the decisions, tool use, and outcome across realistic tasks and repeated runs. Model-based features without agents can benefit from evals too.
Follow the task, then inspect what happened.
A customer-support draft is the teaching example here. Choose a case and follow it through the loop.
Illustration only · fixed examples, not live agent runs or measured results
01 / Define the case
I’d write down a realistic input, the expected outcome, and the boundaries the agent must respect.
The input
A customer asks for a delivery update but does not supply an order ID.
What success means here
Ask for the missing information. Avoid guessing which order belongs to the customer.
The next version goes back through the same cases.
Start with the job the system needs to do
I’d start with the workflow and the actions the system is allowed to take. That gives me something concrete to test: whether the right information is available, the intended steps are followed, and the proposed action is appropriate.
Look beyond one overall score
If 40 out of 50 test cases pass, I want to understand the other ten. Were they missing information? Did the system take the wrong step? Did it try to act when it should have asked for help? Those numbers are an illustrative example; the useful part is investigating the failures.
Make the handoff rules explicit
I want a clear view of the conditions where the system can act and where a person needs to review the decision. A confidence threshold is one possible rule to investigate, but the decision also needs to account for risk and the consequences of getting it wrong.
Use the findings to improve the next version
The point is to learn something I can act on. I’d group failures, adjust the workflow or operating rules, and test again before expanding what the system is allowed to do.
Questions I want to work through
- What should an evaluation measure for a particular use case?
- How do you turn realistic examples into a useful set of tests?
- How do you use failures to improve the system?
Working notes. I’m continuing to develop and test this understanding.