Hamel Husain explains how to win support for evals, tests of an AI application’s behavior, in this four-minute video on his channel.
Start with real failures
- Review 50–100 traces, the records of user interactions and the application’s responses or actions.
- Take notes, group recurring failures, and count how often high-impact errors occur in the reviewed examples.
- Explain surprising user behavior and how those failures obstruct business progress.
Show the improvement loop
- Fix the issues you find, then report what changed and how the error rate fell.
- Repeat the analysis and updates on a regular cadence; Husain leaves the exact schedule open.
- Pair metrics with examples and explanations so the team understands what needs fixing.
Let results make the case
- Husain says demonstrated improvements generate interest in the evaluation process behind them.
- He argues that return on investment usually cannot be quantified upfront; experiments establish value over time.
- His 50–100 traces are a starting point for finding problems, not proof of a representative production-wide error rate.
“You want to let the results lead the conversation on evals.”