Youtube

Zach Mueller (head of DevRel at Lambda, ex-Hugging Face) on how to think about open models — which ones, where they run, what they cost, and how to serve them. ~41 minutes on Hamel Husain’s channel. The big claim Open models are good enough for ~90% of queries from ~90% of people — the 10% exception is people pushing the frontier (OpenAI, Google, Anthropic, frontier research labs) Compared to a year ago, massive improvement in intelligence per parameter; small models now generalize, not just hyper-specialize The current open-model landscape Qwen non-MoE (27B, e.g. Qwen3.6 27B) — “the most capable model for an everyday agent”; 4-bit ≈ 14GB → fits a 16GB card; hundreds of people run Hermes agent purely on it Qwen MoE (~30B) — better than bigger Qwen dense models GLM-5.2 (~800B) — his daily driver for coding; needs 6×96GB cards (~$60k) or a big Mac DeepSeek V4 Flash — capable smaller model, a solid Haiku replacement Kimi (3T) — great for writing/review but 8×B200 ≈ 1.5TB VRAM MiniMax — strong but enterprise license changed since 2.6 Nemotron — US-built, US data; easier to get past enterprise “scary Chinese model” objections Speed targets (tokens/sec) 20 tok/s — baseline usable: background tasks, CPU offloading of a giant model 50 tok/s — usable workflow, roughly Opus-level; GLM 5.2 self-optimized its own deployment from 20→47 tok/s in a day (200–300M tokens through it in the first week and a half) 100 tok/s — ideal for sub-agents 750 tok/s (GPT-5.6 + Cerebras) — too fast to even review what the model writes; the joke is that becomes the chain of thought, presented at 50 tok/s Quantization FP8 native — most expensive; needed when RL rollouts must match training precision NVFP4 / MXFP4 — near-lossless; NVIDIA ships its own NVFP4 weights via quantize-aware training; Kimi K2 (1T) goes from 8–16 B200s in FP8 to fitting on 4 in NVFP4 His setup: GLM 5.2 NVFP4 on half a B200 cluster, Kimi K2.7 code on the other half Cost and security thinking Don’t point company data at OpenRouter — it routes to providers anywhere (including outside the country) and routers can carry “upload your code” instructions; even American providers (Grok) did it. If you can run it yourself, that’s the lowest-risk variable Buying hardware is usually a flawed equation — GPUs run 40–60% utilization, so “pays for itself at 100%” math doesn’t hold; try 2–3 month spot instances first, track token usage, then decide Self-hosting skills advance your career (his path: Accelerate at Hugging Face → GPUs at home) Serving vLLM — out-of-the-box experience, broad model support, an “oracle” that auto-picks kernels; home/single-node default SGLang — disaggregated inference, cache-aware routing for 100–1000 users; multi-node territory llama.cpp — fine for single-user local; he ignores it for batched/multi-user work Model routing and harnesses KB cache + shared context are everything — cold caches rebuild from scratch on every provider switch, so he won’t use “magic routers” unless he picks the models, the tasks, and stays on one provider Numena Ncode (XDR’s harness): a Claude Code fork with a model stack — Soul (driver) + GLM 5.2 (implementer/overseer) + DeepSeek V4 Flash (writes the code); XDR fine-tuned Kimi K2.6 for it and swaps models to measure how much code each frontier model removes Pi — minimal open-source harness; he replaced 3–4 Claude Code workflows with GLM 5.2 + Pi Amp — opinionated routing baked in (GLM 5.2 workhorse + a “Soul Oracle” for hard problems), API-priced; “the Puck” = an agent over agents Evaluating open LLMs: “vibes” plus trusted people (e.g. XDR); Lambda publishes the LLM index — deploy recipes (Docker) + tokens/sec benchmarks Codex side-note Codex’s computer use operates any app on your Mac; mobile support shows all running sessions/threads on your phone; and Codex can control Codex — fan out a project into 12–15 threads, let them talk to each other, orchestrate “Open weight models are good enough for about 90% of queries from 90% of people.” ...

Chris (Build Great Products) walks his full “Product OS” system end-to-end — a 6-hour definitive course for building and launching real software with Claude Code/Codex/Cursor, following one live product (Eyedropper, a cloud design system served to agents over MCP) through four phases with mini-launch validations at every step. ...

Shreya Shankar (UC Berkeley PhD) presents DAB — the Data Agent Benchmark (“Can AI agents answer your data questions?”), from her PhD work at Berkeley. ~27 minutes on Hamel Husain’s channel. Why data agents matter Huge real use case: AI answering business questions — much office work is this Enterprises struggle to use Claude Code / Codex off-the-shelf for BI tasks, so they build their own data agents: Uber’s Query GPT (1.2M queries/month at launch), OpenAI’s in-house agent, Databricks, Snowflake Existing benchmarks fall short: Text-to-SQL — assumes clean data in one Postgres/Snowflake; real enterprises have fragmented data; SQL alone isn’t enough (need domain knowledge, reasoning over outputs) Table QA — context table + question; doesn’t scale to enterprise data volume The formative study — 4 real-world challenges Interviews with enterprises + Hasura (PromptQL) surfaced four properties missing from existing benchmarks: ...

Hamel Husain demos a Codex capability most people don’t know about: Codex can control itself — spawning parallel threads, letting them coordinate, and driving computer use. ~5 minutes on his own channel. The demo setup Working project: AEO — optimizing one of his websites for AI discovery (making it findable by AI search engines) Codex listed 16 high-value tasks from the project file (aeo-to-do.md) — the fuel for the orchestration demo Spawning threads Prompt: “Open a new thread for each task and explain how you’d tackle it, along with prerequisite steps” — Codex spawns 16 parallel threads in the sidebar Codex can also rename and delete threads itself Value: manage separate tasks completely independently, no window-jumping Threads talking to threads Inside any thread you can query another: “What is AEO 1 doing? Does it need any help?” Great for orchestrating when things get stuck, or starting a supervisor thread that manages others and unblocks them Steering and queues Ask for a status table when threads finish: which can run in parallel, which need human intervention or input Broadcast guidance to all threads: “Direct threads that can work independently with computer use to start — don’t start work if you need other threads to finish first” One thread inventories the active threads and coordinates the rest — “this starts to become super powerful” Computer use A thread opens the browser itself: checks Bing Webmaster Tools, Google Search Console, etc. It tells Hamel what it needs (accepting a verification), keeps going, and reports when it gets stuck Mobile Same thread list appears on your phone — manage all parallel threads remotely, even away from the computer “You can have Codex control Codex and become a power user to do a lot of things faster and parallelize your work.” ...

Joe Barrow (ex-Amazon/Adobe, ML lead at Pattern Data, now Adobe Research’s Document Intelligence Lab) on choosing an OCR model for AI document processing. 24 minutes on Hamel Husain’s channel. Why OCR matters Your app sits downstream of OCR quality — garbage in, garbage out, no matter what the LLM does after It’s not solved — Anthropic shipped bad PDF handling for a year because it was pulling text, not doing OCR; users noticed OCR is sticky — once you build on a vendor, swapping models is painful (Pattern learned this the hard way) Documents are evil — multi-column layouts, rotated scans, no reading order; TeX-compiled PDFs have no spaces (glyph glue), so naive text extraction gives you one run of characters The decision grid: two axes Text blocks vs. document structure Text blocks: word/line bounding boxes → grounding, evidence highlighting, cheapest Structure: headings, reading order, grouped paragraphs, tables, figure alt text, chart de-rendering → much better LLM input (LLMs are trained on markdown-like structure; raw line runs look like garbage to them) API vs. self-host API: ease of use, vendor support (startups retrain on your bad docs), minimal time — right for ~95% of teams Self-host: control throughput/concurrency (APIs cap concurrent docs — a real bottleneck), stable weights, domain fine-tuning, no lock-in, cheaper at bulk — but only if your time ≈ $0 or you run huge batches The four quadrants Big cloud APIs (AWS Textract, Google Cloud Vision, Azure) — $0.60–1.50 / 1k pages; word+line boxes only; tables/forms a la carte at $10–15 / 1k Document startups (Reducto, Data Lab, Extend, LlamaIndex) — $5–20 / 1k pages, “fast” vs “accurate” tiers; structure included (markdown/HTML, tables, figure boxes) Open pipelines (PaddleOCR, Nemo Tron, Tesseract) — 10–100M params, nearly free, edge-deployable (PaddleOCR runs on phones/e-ink); text lines only, post-process with layout models Open VLMs (LightOn OCR 2, GLM OCR, GOT-OCR, Chandra/Surya) — 600M–8B params, native document structure, ~$0.20–0.30 / 1k pages on a saturated H100; hallucination risk exists but clouds hallucinate on crusty scans too How to actually choose Ignore benchmarks (OmniDocBench, CR Bench) — they’re not run on your data Build a 50–100 page sample of your own representative PDFs Run a few candidates, diff the returned text (catches junk-on-handwriting fast), visualize the boxes ~a day of effort total — then pick Watch the license Chandra/Surya (Data Lab): free only if org < $2M revenue AND not competing with Data Lab LightOn OCR: Apache. GLM OCR: MIT (but relies on PaddlePaddle’s Doc Layout model — Apache — both apply) Self-hosting, for the ~5% who should Inference engines: VL (default, OpenAI-style client) or SGLang; infra: Modal (request-queue scaling beats SageMaker), BaseTen, or big cloud for one-off batches 1B-param models (LightOn, GLM) on H100 → ~10k pages/hr, 20–30¢ / 1k pages; 4×3090 ≈ one H100 → 3–4 pages/sec His 7M-page local-laws dataset: ran over a weekend at ~30¢ / 1k all-in “You can process 1,000 pages per second, but it doesn’t matter if they’re all wrong — then your entire app’s output is going to be garbage.” ...

Jeff Su’s counter to the default “jump in, pick a template, start prompting” tutorials — that path gives you generic output and burns tokens fixing unusable slides. His fix: three files prepared ahead of time, demonstrated with the actual deck he used for a paid workshop. ...

Futurepedia’s full-platform guide to Claude Design after its big upgrade — the host skipped covering it at launch because usage limits made it barely usable; that’s fixed (usage now bundles into your existing Claude credits). The overview: 15+ template types (mobile apps, slides, documents, wireframes, animations, UI mockups, resumes, 3D objects, HTML email, flyers), and the design-system workflow that stops output from looking like generic AI slop. ...

Shreya Shankar (Stanford CS professor, co-creator of the AI evals course with Hamel) kicks off the 12-part AI product engineering series. 27 minutes on Hamel Husain’s channel. Why this matters Output quality is the biggest barrier to productionizing agents (LangSmith annual report) — and figuring out how to evaluate models is genuinely hard Vendors (LangChain, Braintrust, Arize) are selling end-to-end automated eval tools: point an LLM at your traces, it finds and fixes your bugs The catch is epistemic: what “good” means lives in your head, not in the traces — if a tool could fully fix your product, it could fix everyone’s, and there’d be nothing left to differentiate yours AI’s real job: help you express and apply your judgment faster, not replace it The eval lifecycle (analyze → measure → improve) Error analysis — the hardest step: take traces and find failure modes. No perfect definition of “mistake” (you can’t define slop, but you know it when you see it) Measure — how prevalent is each failure mode? Pareto applies: ~80% of issues come from ~20% of failure modes — prioritize those Improve — fix the product: prompt instructions, model switch, fine-tuning. Iterate forever AI is weak at the front (taste-specific error analysis) and strong at the back (measurement, prompt optimization, hill-climbing). ...