Hamel Husain on his own channel — a 57-second answer to the question every team doing evals eventually asks: how do you steer an agent to do this work well — skills, MCP servers, agent markdown files?
Start with the person, not the agent
- The question: how do you constrain or steer the agent to do evals well as a human partner? Is it equipping it with skills and MCP servers you built and maintain, or agent markdown files?
- Hamel’s answer does not start with tooling: “my answer starts with the person, not the agent”
- You have to educate the human first, and the human has to understand the process — what good looks like, and what you are actually trying to do
- Only after that do the skills help; the agent is not the bottleneck the question assumes
Why skills have a ceiling
- Skills “can only help you so much” — there is an upper limit to what they can do for you
- The reason is structural: a published skill is generic by design, because it has to apply across a lot of different use cases
- Getting value out of it means putting in “a little bit more thinking” to customize it for your own problem
- The whole thing “comes down to learning what good evals look like, plus using the skills, and then customizing the skills” for yourself
The loop he describes
- Learn what good evals look like — the judgment, not the tooling
- Use the skills
- Customize them to your use case
- Same frame he applies to coding: the skill is scaffolding around the process knowledge, not a replacement for it
The resources in the clip
- The evals skills are open — hamelsmu/evals-skills on GitHub, with updates landing as recently as the day before this clip
- The next AI Evals cohort with Hugo (Bowne-Anderson) is open at maven.com/parlance-labs/evals
“It’s just like anything else, it’s just like coding or anything else. You need to educate the human and you need to give them some skills, and the human needs to understand the process of what good looks like. The skills can only help you so much.” — Hamel Husain