YouTube videos

Cole Medin demonstrates two feedback loops for production agents: learn from human corrections, then propose trace-backed changes to instructions and skills.

Jeff Crume uses Jev to explain why automated decisions need independent evidence, audit trails and checks for subtle changes in behavior.

Al Chen of Cisco explains how exploit tests, verified restores and privacy checks help banks move AI into production. The bottleneck also includes image approvals, GPU procurement and internal ownership.

Moshi founder Anekha Sokhal argues that AI should gather and check market information while investors keep the decision. She describes a six-agent system, traceable claims and the hidden cost of losing apprenticeship.

Momento's Khawaja Shams explains realistic coding tasks, leaked answers, repeated trials, and the traces that make agent scores worth trusting.

Brooke Lacey shows Claire Vo how she uses Claude Code, FFmpeg, and Whisper to turn phone footage into short-form videos, learn her delivery, and mine audience comments.

Martin Keen explains the goals, triggers, context, tools and human checks around an agent's act–observe–adjust loop.

Maddy Zhang's eight rules for reviewing AI-generated code, preserving error signals, limiting production risk and checking dependencies.

Review 50–100 traces, find recurring failures, and show how fixes reduce errors. Build support for evaluation through visible product improvements.

Ben Carsley explains context-specific writing styles, the limits of prompt-only personalization, and shared model serving with a small adapter per person.

Keep evaluation inside the development loop, give a trusted domain expert the final call on quality, and build annotation tools around their work.

Wolfe’s weekly roundup contrasts Krea’s editing workflow, downloadable models, one-shot coding demos and the smaller updates behind product launch headlines.

Cole Medin keeps agents building in parallel, with people defining each release and testing the result before production.

Pi Durable, a live Lisp demo, and the cost of agents whose work is getting harder to inspect.

Hamel Husain argues for validating AI judges against human labels before adding another model provider: measure precision and recall, then optimize cost.

Martin Keen maps four context failures to four fixes: validate mistaken notes, compact long histories, curate tools and remove superseded information.

Joseph Enochs on reversible agent actions, creative control in AI video, and finding enterprise problems worth solving—with important limits to the interview's optimistic claims.

A personalized news pipeline: deduplicate and filter in code, use Jev for inexpensive triage, and reserve an LLM for the ambiguous items and final digest.

Hamel Husain's interview playbook: audit imperfect data, establish a baseline, run honest comparisons, challenge vague metrics, and explain product trade-offs.

A live test finds real failures in 100 leasing-assistant conversations, but premature test design, brittle grading and a poor review interface undermine the workflow.

Peter Yang compares code-built scenes, games, storytelling, websites, and writing—and finds different winners for different tasks. These are selected demos, not a controlled benchmark.

Hamel Husain's seven eval pitfalls: imaginary failures, generic scores, unvalidated judges, opaque interfaces, rating scales, the wrong reviewers, and over-automation.

Cole Medin argues that stronger coding agents increase the value of clear intent, concrete references, and human validation—not passive forwarding of AI output.

A Berkeley mathematician on AI-assisted research, the limits of quantum speedups, and why foundations still matter when answers are cheap.

Isolated workspaces, a readable code structure, before-and-after evidence, and bounded review loops—with a human still deciding what ships.

Ben Hylak explains change-focused agent testing: replay production traces in a reconstructed environment, inspect behavioral differences, and validate the simulation before trusting it.

When the mandate is 'add AI,' look for a workflow worth accelerating—not a chat box to bolt onto an existing screen.

A blind, small-sample writing comparison finds some improvements over Opus 5, but generic phrasing, misplaced detail, and bot-like replies survive.

Four places Cole uses a fast decision model in his AI second brain: email triage, injection screening, news filtering, and deduplication—with a clear boundary for when an LLM still does better.

If customer traces are sensitive, seek limited authorized access, recruit cleared domain experts, or carefully redact; synthetic examples are a last resort.

Hamel ranks 28 evaluation practices: inspect real failures, use human-validated product-specific checks, and stop mistaking generic dashboard scores for progress.

Three skills for turning a spec into reviewed work, making PRs easier to judge, and learning from real agent sessions—plus a breaking glossary filename change.

The 100-trace target is a starting cue, not a magic sample size; keep inspecting until examples stop teaching you something new.

The Linux maintainer separates useful AI-assisted bug finding from noisy security reports and asks who will actually review, fix, and deploy the patches.

Refresh AI eval datasets through error analysis; use product metrics, not static tests, for long-term comparison.

A document-agent answer can be wrong because OCR misread the source, not because the LLM hallucinated. Trace the failure back before changing the model.

Peter Yang builds a voice-first Japanese travel tutor from a spec and screen designs, then fixes the voice loop, improves the lessons, and deploys a small private beta.

Speakeasy COO Hector Hernandez on what breaks when agent use scales: invisible tool connections, ambiguous identities, overbroad permissions, and approval fatigue.

Lauren Tan (poteto) on the agent infrastructure behind a reported 2,500 production PRs in one month: verification, deterministic tools, constrained codebases, coordinator agents, continuous gardening, and review by sampling.

Granola co-founder Sam Stephenson on pre-meeting briefs, turning conversations into organizational memory, designing AI that lowers stress, and building an AI-native company without the 996 grind.

Joseph Enochs on enterprise AI from the boardroom to production: measure the baton rather than busy teams, replace scattered pilots with shared foundations, constrain agent autonomy, and optimize complete business outcomes instead of local activity.

Writing an evaluation rubric before examining real traces can lock in the wrong criteria. Hamel Husain argues for open-ended error analysis first, then rubrics and reference answers grounded in observed failures.

Accessible PDFs need more than extracted text: reading order, headings, formulas, tables, figures, and other document structure must survive. Hamel Husain and Isaac Flath examine DataLab's API and the human review still required.

Overall agreement can make a useless LLM judge look accurate when failures are rare. Hamel Husain shows why product teams need class-specific error metrics instead of one reassuring percentage.

Model benchmarks answer how a model performs in general. Product evals answer the operational question that builders actually own: whether their specific system does the right thing on their users’ tasks.

A live attempt to let Codex buy a $5 course with Mercury’s agent card shows both sides of delegated payments: tightly scoped spending and enough approval friction to erase the convenience.

Four places a fast decision model can sit inside an AI coding workflow: screening tool calls, playtesting games, driving browser tests, and routing GitHub work to the right process and model.

Hamel Husain tours an open-source eval toolkit built with Shreya Shankar: discover real failure modes before writing tests, audit weak eval pipelines, calibrate LLM judges against human labels, and build review interfaces tailored to your traces.

Hamel Husain and Isaac Flath compare three writing setups and get a messier result than expected: the elaborate anti-slop prompt, a plain-writing prompt, and no system prompt each win examples. Their small test points away from prompt mythology and toward context, real outputs, and editing effort.

Peter Yang tests code-generated motion graphics, product launches, talking-head edits, music videos, and anime—then extracts a practical workflow for making better AI-assisted video.