Next Token Ep 06 — Escape the senior engineer death spiral
Sunil Pai on recovering momentum through smaller work. Rhys Sullivan revisits YC and shows how clearer contracts made his coding agents more useful.
Sunil Pai on recovering momentum through smaller work. Rhys Sullivan revisits YC and shows how clearer contracts made his coding agents more useful.
Cole Medin shows why code search consumes so much of an agent's context, then demos Sonar Vortex's graph-based semantic navigation and its claimed 6–34% token savings.
Sentience founder Sam Kececi on building a hyper-accurate simulation of a person: where off-the-shelf models are fine and where your taste has to be in the weights, the perception-to-memory master plan behind the product, and why he thinks AI slop is really a uniformity problem.
A customer-support example walks through tool design, selective memory, testing, tracing and safeguards for agents that take real actions.
Yutori founding researcher Skanda Vaidyanath on training a 27B computer-use model: trajectories that weave GUI, code and tool calls, a data flywheel with agents in every loop, rejection sampling then async RL, and stress-testing verifiers for reward hacks.
Cole Medin on mining your Claude Code JSONL transcripts for failure patterns: the two-line prompt version, the Databricks volume + Genie structured-tables version, and the three AI-layer changes he actually shipped.
Hamel Husain's answer to how you get an agent to do evals well starts with the human, not the agent: understand what good looks like first, because a published skill is generic by design and only pays off once you customize it.
A 2026 Harvard audit of six AI companions found five using emotional manipulation; 30% of companion conversations deploy a manipulation tactic when a user tries to leave; the STAR framework, three-layer memory, AI-run safety planning — and an RCT showing the app improves wellbeing.
Medin hit the $200 Claude Max and $200 Codex Pro limits days before reset; the fix is model routing — most capable model for planning and review, cheap open model for implementation, roughly 4x cheaper with the same or better output.
Amazon made MCP the standard for Alexa+ integrations; the Build, Ship, Shape hackathon covers Fire TV, Alexa+, Ring and Bee; the Alexa+ track takes an MCP server or Agent Skill with a simulated web-app stand-in; $190K in cash and AWS credits; submissions close Oct 23.
Peter Yang stress-tests the new Opus on five tasks: a Golden Gate Bridge 3D flyover in Blender, a playable Soaring Over the World ride artifact, stroke-by-stroke impressionist painting, Claude Design inside Claude Code, and frame-by-frame video editing - plus the personality prompt that Opus 5 failed.
Murali Swaminathan, CTO of Freshworks: 70%+ of net-new code is AI-written, commits capped at 500 lines because review is the new bottleneck, deflection moved from 40% to the high 70s, anything under 50% confidence goes back to a human, and agents get priced per session instead of per seat.
Ethan Bloch, product lead for ChatGPT Finances, demos a 90-day subscription audit that emails the biller for you, tax optimization, ETF fee swaps, credit-card reward analysis, and interactive decision sites for rent-vs-buy and taking time off.
Peter Yang's hands-on tour of Meta's Muse: $40/month off AT&T by finding an autopay discount, an agent that phones Comcast and negotiates in its own voice, native connectors to Meta's apps, a personalized feed with no ads yet, habit artifacts with streaks, and Marketplace listings it can haggle over.
Sentience founding engineer Ahmet İlten on the four principles behind their personal-agent product: complexity masks the real problem, start from the user experience and work backwards, keep the system prompt visible as one monster you can stare at, and run evals before every change.
Ali Parandeh on why heavy industry is 2-3 years behind on AI, why Copilot is the floor and not the strategy, bespoke predictive-maintenance models on bespoke sensor data, the dark-room-with-a-lantern case for time-boxed AI R&D, the four ways agent projects fail, scaling consulting through software and forward deployed engineers, and why handcrafted work gets more valuable.
Cole Medin on turning a personal AI second brain into a team brain: one documents table, domain labels applied at ingestion, row-level permissions enforced inside the database behind a remote MCP server, and hybrid keyword + semantic retrieval.
Peter Yang's 15-step podcast pipeline, split into three ChatGPT Skills (prep, edit, production) that call each other; pastes his manual edits back into the skill as a numbered diff so it one-shots next time.
Three model-written X posts about the same document-QA pipeline, judged by reading the drafts aloud. The ones that won carried a real number and one fewer fact than the rest.
Anthropic's fix for mannered prose is itself written in mannered prose. Hamel Husain and Isaac Flath run it side by side against a plain-writing prompt and a no-prompt baseline on the same question, then judge the drafts by reading them out loud.
Cole Medin's 'drive screen' skill: ~400 lines, no harness, no extra installs; morning setup in 60 seconds; hard rules + control loop + a custom CLI for determinism; prompt-injection caveat and why he now trusts computer use.
Smartling's VP of AI Olga Beregovaya on why one-to-one translation is dying, generating target content straight from briefs instead of a source, why models hallucinate more in under-resourced languages, interpretation outliving translation, translateese and modelese, the build-versus-buy line, and the 12 metrics her team scores translation quality with.
Brex CEO Pedro Franceschi on scoping agents as virtual employees with a job, skills, manager and budget — the Jim recruiting demo, CrabTrap's LLM-as-a-judge proxy, Magpie for token spend, and the OpenClaw harness he runs his own life on.
Newport's explanation of the OpenAI/Hugging Face incident: an ordinary program looping an LLM's plausible suggestions, executed for days without a human watching — a bad architecture, not a model that woke up. Plus why the same property that makes chatbots confabulate makes them unfit to author executed actions.
A step-by-step deploy guide for Cole Medin's open-source software factory: GitHub issue in, validated merged PR out, running 24/7 on a remote Ubuntu box with Codex on GPT-6 Astra. Plus his blunt read on the AGI hype — not AGI, but the first model he says actually gets him.
Hamel Husain streams Anthropic's new `claude plugin eval` against a plain-writing skill and calls it a failure: the eval it generates is one ablation against the skill's own rules, the numbers have no provenance, and it asks you for exactly the error analysis that is the hard part.
Fletch PMM co-founder Rob Kaminski does a live positioning teardown of an AI agent startup on TwoSetAI: the three category strategies, why category creation is the trap founders overrate, why stacking features makes you forgettable, and why the real failure is refusing to choose.
Cole Medin on why a second review agent is not a security process: coding agents ship vulnerabilities two ways, are trained to cut corners, and sometimes notice a flaw and ship it anyway. His fix is a deterministic gate — a real scanner wired into the workflow that the agent cannot skip and must iterate against until green.
Cole Medin's pattern for letting coding agents operate real infrastructure without ever holding credentials: an orchestrator wraps the agent, exposes an allowlist of Git-managed workflows, and keeps every execution observable. A Kestra-sponsored ~2-minute spotlight.
Rec Technologies CEO Birju Kadakia (early Uber, ran Uber Eats worldwide, product at The Athletic) on the vertical AI wedge in recreation tech — agents that read city policies and write deterministic refund code, skills-versus-workflows architecture, hallucination guardrails with real accuracy numbers, and hiring engineers who act like engineering managers. Interviewed by Angelina on TwoSetAI.
Brilliant CEO Sue Khim on why students mostly use AI to cheat today, her AI tutor Cooji that refuses to hand over answers or explanations, and why the most defensible AI products live in domains the frontier labs can't RL on.
Peter Yang builds four games in one night with GPT-6 Astra on the $20 ChatGPT Plus plan — Star Fox-style shooter, train FPS, Starcraft-like RTS, and a roguelike deckbuilder — using free Blender/Godot MCP servers, concept art, and mostly lazy voice feedback.
Andrew Ng on the Silicon Valley Girl podcast: why the AI doom drumbeat is incumbent-driven regulatory capture, why AI doing 30-40% of job tasks makes the human 60-70% more valuable, why LLMs as commonly used are terrible for learning (and his new Learn Vector 1-to-1 tutoring bet), the context advantage that keeps human judgment relevant, and the 'product management bottleneck' he says is now the real constraint on building.
Nandan Thakur (BEIR, MIRACL, TREC-RAG; now Microsoft Research India) on agentic search: BrowseComp-Plus as a reproducible eval, why the retriever moves accuracy more than the model does (60% → 88% on the same LLM), ORBIT's zero-budget 20k-question synthetic data pipeline, and HawkEye trajectory analysis for the leaderboard trap where 95% accuracy hides 210 search calls.
Former CRO Jenée Hall on pricing AI agent products — the five pricing layers from per-seat access to outcome-based fees, why token-style usage pricing misleads, and how to test price like a hypothesis. Interviewed by Angelina on TwoSetAI.
Teresa Torres (Continuous Discovery Habits) on why 'talk to your customers' fails for engineers, story-based interviewing as the reliable feedback loop, the Opportunity Solution Tree, how AI synthesis and her Vistily partnership put evals at the center of discovery, and a three-week eval war story about upstream errors and counterbalancing judges.
Peter Yang runs four personal AI agents at once — Instinct, Grok Bot, ChatGPT/Codex, and open-source Hermes — and stress-tests what you're really handing over: passwords, 2FA codes, prompt-injection surface, and whether you can ever get the data back.
Cole Medin's 11 small, high-leverage fixes for making any coding agent more reliable — from writing for the agent (not the human) and keeping rules under ~200 lines, to moving guarantees into deterministic hooks and never letting the writer approve its own work.
MongoDB's field CTO of AI argues agent performance — especially cost-adjusted — is a retrieval problem, not a model problem: a history of constraints, why embeddings aren't commoditized, and the write/change/recall/forget memory loop.
Hamel Husain on why 'it's hard to eval' is a product smell, verification-first design (the data agent, workers' comp report, and PE lesson planner redesigns), sharing skill shapes not skills, transition failure matrices, and why data science is more valuable than ever in the agentic era.
Radu, search engineer at Vespa, on why leaderboard picks leave cost, latency, and quality on the table — quantization, Matryoshka dims, tuning with Vespa Embed, and cheap-embedder + re-ranker architectures.
Cloudflare AI product lead Michelle Chen explains model launches, GPU economics, licensing, and why cheap inference subscriptions may tighten.
Daniel Bloom, a PM at Melio, walks through his self-improving Claude Cowork system: a Notion board his agent builds and manages for him, proactive context acquisition ('what does this term mean?'), a weekly prep and daily brief, a self-improvement loop that learns from his edits — and the Workstation plugin that onboards the whole company in 15 minutes.
Vectara's Ofer Mendelevitch (Hands-On RAG for Production) on building RAG systems that actually work at scale — retrieval, multimodal data, knowledge graphs, evals without golden answers — and why the skill that survives the AI shift is directing agents, not writing code.
Replit's head of engineering on non-coders building real businesses with AI, the cost-of-code-zero test for what SaaS survives, and why app-layer companies are thriving.
Tara Seshan — OpenAI's product lead for Codex and ChatGPT Work — maps the third era of AI products (after chat and coding agents): persistent coworkers you steer, why ambition is the new differentiator, and why writing-as-thinking is the craft that survives.
Peter Yang's head-to-head across design, writing, planning, coding, browser use, voice, image/video gen, and personal agents — with live demos and a clear overall winner.
The hard part of background agents is the background: dev boxes over sandboxes, sessions over jobs, and keeping the agent in the control plane away from the risky work.
Shreya Shankar and Hamel Husain walk through the practical evals loop: top-down vs bottom-up criteria, an open-source Claude Code "error discovery" workflow that turns your taste into a rubric, and why automated eval tools get you a baseline but never replace reading your own data.
Why single-vector embeddings fail agentic retrieval, how late-interaction multi-vector scoring preserves low-level detail, and how sparse encoding makes it fast enough for production.