Hamel Husain on his own channel — a 1-minute clip on the “evals take too long” objection and why your users eval your product whether you do or not.

The “evals take too long” objection

  • One of the main reasons people skip evals: reading a trace takes forever — a lot of human hours you “don’t have time to do”
  • His example: an AI data agent that answers business questions (supply, revenue) — “the most popular internal tool I’ve seen so far”
  • The question: how do you eval that?

If you can’t eval it, it probably sucks

  • One principle to keep in mind: if you can’t eval your product, your product probably sucks — because your users need to eval the product
  • When you present an AI answer to a user — say, an answer to a business question — you need to figure out a way to convey trust

“If you can’t eval your product, your product probably sucks. Why is that? It’s because your users need to eval the product.” — Hamel Husain