https://intelligentartifact.com/tags/hamel-husain/ Recent content in Hamel-Husain on Intelligent Artifact Hugo en-us Tue, 25 Aug 2026 21:30:10 +0000 https://intelligentartifact.com/posts/dont-build-agents-build-environments-instead/ Tue, 25 Aug 2026 21:30:10 +0000 https://intelligentartifact.com/posts/dont-build-agents-build-environments-instead/ The hard part of background agents is the background: dev boxes over sandboxes, sessions over jobs, and keeping the agent in the control plane away from the risky work. https://intelligentartifact.com/posts/how-to-build-better-ai-evals-with-claude-code/ Sun, 23 Aug 2026 13:00:04 +0000 https://intelligentartifact.com/posts/how-to-build-better-ai-evals-with-claude-code/ Shreya Shankar and Hamel Husain walk through the practical evals loop: top-down vs bottom-up criteria, an open-source Claude Code "error discovery" workflow that turns your taste into a rubric, and why automated eval tools get you a baseline but never replace reading your own data. https://intelligentartifact.com/posts/how-multi-vector-retrieval-works-at-scale/ Fri, 21 Aug 2026 21:30:02 +0000 https://intelligentartifact.com/posts/how-multi-vector-retrieval-works-at-scale/ Why single-vector embeddings fail agentic retrieval, how late-interaction multi-vector scoring preserves low-level detail, and how sparse encoding makes it fast enough for production. https://intelligentartifact.com/posts/how-to-turn-evals-into-a-better-model/ Mon, 17 Aug 2026 20:30:11 +0000 https://intelligentartifact.com/posts/how-to-turn-evals-into-a-better-model/ Evals are the bottleneck: the three parts of an evaluation, the hidden variables that skew scores, and when to reach for fine-tuning and RL instead of prompt tweaks. https://intelligentartifact.com/posts/cut-llm-classification-costs-by-90/ Wed, 12 Aug 2026 17:07:59 +0000 https://intelligentartifact.com/posts/cut-llm-classification-costs-by-90/ A live whiteboard lecture on model cascades: send every record to a cheap proxy model first, accept its answer when confidence is high, route the rest to the expensive model — and pick the threshold from a labeled sample to hit your accuracy target at minimum cost. https://intelligentartifact.com/posts/how-to-use-open-models-effectively/ Fri, 07 Aug 2026 21:30:06 +0000 https://intelligentartifact.com/posts/how-to-use-open-models-effectively/ Zach Mueller (Lambda) on when open models beat APIs: the current open-model landscape, token-speed targets, quantization trade-offs, and why open weights are good enough for ~90% of tasks for ~90% of people. https://intelligentartifact.com/posts/how-to-build-agents-that-answer-data-questions/ Fri, 31 Jul 2026 21:30:38 +0000 https://intelligentartifact.com/posts/how-to-build-agents-that-answer-data-questions/ The DAB benchmark for data agents: why text-to-SQL isn't enough, the four real-world data challenges (messy joins, multi-database, free text, domain knowledge), and why frontier agents only pass 57% of the time. https://intelligentartifact.com/posts/how-to-make-codex-run-itself/ Mon, 27 Jul 2026 21:30:14 +0000 https://intelligentartifact.com/posts/how-to-make-codex-run-itself/ Codex's hidden superpower: threads that spawn threads, talk to each other, and orchestrate themselves. Hamel demos running 16 parallel coding tasks with computer use and mobile control. https://intelligentartifact.com/posts/how-to-choose-the-right-ocr-model/ Fri, 24 Jul 2026 21:30:07 +0000 https://intelligentartifact.com/posts/how-to-choose-the-right-ocr-model/ A practical framework for picking an OCR model: text-blocks vs document structure, API vs self-host, and why you should test on your own data instead of trusting benchmarks. https://intelligentartifact.com/posts/how-to-build-ai-evals/ Fri, 17 Jul 2026 00:00:00 +0000 https://intelligentartifact.com/posts/how-to-build-ai-evals/ Lucas Rocha, an engineer at Brazilian edtech Nova Escola and alum of Hamel Husain's AI Evals course, tells his evals rollout story backwards: from a team that now runs LLM judges with no tickets and no engineer in the loop, back to the messy first spreadsheet and the three barriers — data, methodology, and judgment — that every team hits along the way. https://intelligentartifact.com/posts/how-to-reduce-llm-latency/ Sat, 11 Jul 2026 00:00:00 +0000 https://intelligentartifact.com/posts/how-to-reduce-llm-latency/ Abi Aryan breaks down where LLM latency actually goes: the compute-bound prefill phase vs the memory-bandwidth-bound decode phase, and why decode — not prefill — dominates latency and cost, especially in multi-step agentic systems. A live Colab walkthrough with TinyLlama on a T4 measures both phases across chat, RAG, creative, and agentic request shapes. https://intelligentartifact.com/posts/how-to-build-ai-that-earns-user-trust/ Tue, 07 Jul 2026 00:00:00 +0000 https://intelligentartifact.com/posts/how-to-build-ai-that-earns-user-trust/ Before worrying about evals, ask if you are building the right thing: AI that dumps an unverifiable answer creates more work. Hamel walks through design patterns — provenance, progressive disclosure, expert signals, modular steps — across three worked examples. https://intelligentartifact.com/posts/how-to-run-ai-coding-sessions-from-your-phone/ Mon, 06 Jul 2026 00:00:00 +0000 https://intelligentartifact.com/posts/how-to-run-ai-coding-sessions-from-your-phone/ A walkthrough of Codex desktop app remote control: connect your agentic-coding servers, kick off sessions from your phone, and watch agents work anywhere — with live session views, subagent summaries, and follow-up prompts. https://intelligentartifact.com/posts/how-to-automate-ai-evals-correctly/ Fri, 03 Jul 2026 21:30:24 +0000 https://intelligentartifact.com/posts/how-to-automate-ai-evals-correctly/ Part 1 of Hamel and Shreya's AI product engineering series: the analyze → measure → improve eval lifecycle, and three mistakes to avoid when using AI to automate evals. https://intelligentartifact.com/posts/how-to-hire-an-ai-consultant-the-right-way/ Thu, 02 Jul 2026 00:00:00 +0000 https://intelligentartifact.com/posts/how-to-hire-an-ai-consultant-the-right-way/ Hamel argues that hiring a consultant to implement AI and walk away is almost always a mistake — the underlying problems remain and the AI itself is designed to remove the middleman. Consultants work only for education or when they have skin in the game. https://intelligentartifact.com/posts/how-to-eval-an-ai-product-that-seems-impossible-to-eval/ Mon, 29 Jun 2026 00:00:00 +0000 https://intelligentartifact.com/posts/how-to-eval-an-ai-product-that-seems-impossible-to-eval/ When a product seems impossible to eval, that is usually a sign the product itself is weak. Hamel suggests infusing user-facing signals into the product so users — and you — can assess quality. https://intelligentartifact.com/posts/if-you-cant-eval-your-ai-your-users-cant-trust-it/ Fri, 26 Jun 2026 00:00:00 +0000 https://intelligentartifact.com/posts/if-you-cant-eval-your-ai-your-users-cant-trust-it/ Hamel Husain counters the 'evals take too long' objection with one principle: if you can't eval your product, your users can't either — so evals are how you figure out how to convey trust in AI answers. https://intelligentartifact.com/posts/why-theres-no-llm-judge-you-can-trust-100-of-the-time/ Mon, 22 Jun 2026 00:00:00 +0000 https://intelligentartifact.com/posts/why-theres-no-llm-judge-you-can-trust-100-of-the-time/ Hamel Husain explains how to scientifically measure an LLM judge's noise against ground-truth human labels — and why every LLM judge is a black-box classifier with edge cases, so you have to decide how much noise is tolerable. https://intelligentartifact.com/posts/why-ai-rating-scales-make-it-harder-to-ship/ Sun, 21 Jun 2026 00:00:00 +0000 https://intelligentartifact.com/posts/why-ai-rating-scales-make-it-harder-to-ship/ Hamel Husain argues that ROUGE/BLEU string-similarity scores and 1-to-10 rating scales are holdovers from traditional NLP that make evals harder to act on — nobody knows what a 4.2 means, and binary pass/fail evals are what actually let you ship. https://intelligentartifact.com/posts/why-the-1-to-5-scale-is-where-ai-evals-break-down/ Sat, 20 Jun 2026 00:00:00 +0000 https://intelligentartifact.com/posts/why-the-1-to-5-scale-is-where-ai-evals-break-down/ Hamel Husain explains why he converts every 1-to-5 rating scale eval into binary pass/fail checks, scoping each failure into a specific, actionable criterion instead of hiding noise behind middle values.