Hamel Husain distills recurring mistakes from teaching AI evaluation to more than 5,000 engineers and product managers and working with over 50 companies (12:47; figures are his own).

Start with observed failures

  • 1. Writing tests before reading data: inspect real interactions before brainstorming an unlimited list of possible problems.
  • 2. Using generic metrics: replace abstract helpfulness scores with specific failures such as missed handoffs, incorrect recipients, or broken scheduling.
  • His leasing example: a polite goodbye looked successful but missed the opportunity to offer a cheaper qualifying property.
  • Favor non-trivial, observed or reproducible failures and verifiable pass/fail checks—not giant catch-all prompts.

Validate the evaluator and design for review

  • 3. Trusting an untested AI judge: compare its decisions with domain-expert labels before trusting the score.
  • His suggested split is 20% examples, 40% development, and 40% held-out testing; tune on development, then check unseen cases.
  • Examine which failures the judge misses and which acceptable results it wrongly flags, not just overall accuracy.
  • 4. Building opaque products: show sources, intermediate calculations, and relevant comparisons so experts can inspect an answer efficiently.
  • For lesson plans, show changes against a trusted plan and let teachers accept or reject them.

Keep criteria simple and humans accountable

  • 5. Using one-to-five ratings: narrowly scoped pass/fail judgments are easier to align with a “good enough to ship” decision.
  • 6. Asking the wrong people to review: lawyers judge legal assistance; medical experts judge medical assistance; developers are not automatically domain experts.
  • Build a custom annotation interface that makes the expert’s review easy.
  • 7. Outsourcing discovery to automation: automated tools can catch obvious defects while missing business context and interaction failures.
  • In his vendor comparison, tools missed leasing objections, markdown in texts, voice interruptions, and human handoffs; this is a reported study, not a universal ranking.

“It’s impossible for you to know what good looks like without looking at some data.”

Bottom line: automate known checks after experts have established what matters; keep human review in the loop to discover what the checks miss.