A “harness” is the program wrapped around a model: it holds the system prompt, hands the model its tools, and runs the loop. Nathan Sutton ran nine of them — opencode, cline, codex, goose, crush, dsh, pi, mini-swe-agent, plus his own chad — against a local model on a laptop instead of a data center, on eight small coding exercises each. The point isn’t which one wins. It’s that the reason local agents feel broken is structural. ...
Cole Medin on the security gap in AI coding workflows (17 min, partnered with Sonar; Archon, the harness builder he uses, is his own open-source project). His premise: agents are already better and faster than humans at a lot of code — but security is where they fall short far more than reusability, documentation or performance, and almost nobody is building for it. ...
Cole Medin on the permission pattern he uses so AI coding agents can operate real infrastructure without holding the keys — a ~2-minute spotlight, sponsored by Kestra (their 2.0 just shipped; the core stays Apache-2.0). Why trust breaks at the boundary He has used coding agents to write every line of code this year — but still monitors and validates everything The limiter isn’t the model, it’s what the agent can reach: the moment it touches real infrastructure you hand it a cloud key, a database URL, shell access — everything, including the ability to wipe the database The fix: an orchestrator between the agent and the infra Wrap the coding agent in an orchestrator that gives it workflows to do the things you want — and nothing more Operations are written once as YAML in Git; the agent never gets access to the credentials the orchestrator uses The agent simply receives a list of workflows it is allowed to invoke, and credentials are used inside those flows — “that is the entire permission model” What the permission surface looks like in practice The example agent can restart services and scale replicas — nothing else Flows are plain YAML files defined alongside the agent, so everything is manageable in Git Observability is what makes unattended runs OK Every call comes back as a normal execution with inputs, logs, and results visible in the dashboard When agents run without you, you can still open and read what they did Open source Kestra 2.0 is officially out; the core is Apache 2.0, and the AI-agent piece is part of the open-source core, not behind an enterprise paywall “What limits my trust is not the model itself — it’s what it can reach.” — Cole Medin ...
Dan Luu keeps running big, careful experiments on how coding agents actually behave. This one asks a pointed question: if you tell an agent to test its own code properly — test-driven development, fuzzing, formal proof tools — does the code come out more correct? The short answer: almost never. He had an AI agent implement zstd (a compression format) from its spec in Rust: 80 runs per condition, 26 different testing instructions plus 4 “skills” (packaged instruction files), scored against a hidden test suite. A smaller run on an email protocol spec replicated the findings. ...
Birju Kadakia — CEO of Rec Technologies (recreation tech; early Uber employee who helped launch Uber Eats worldwide, then led product at The Athletic) — interviewed by Angelina on TwoSetAI (56 min). Rec is the AI layer being slid under 25 years of incumbent recreation software: registration, court reservations, memberships, and refunds for municipal parks & rec departments, with 100+ cities live. ...
Peter Yang, who says he has no idea how to use Blender or Godot, spent one Friday night building four games simultaneously with GPT-6 Astra on the Medium model (the $20 ChatGPT Plus plan — “you really don’t need Astra high or ultra”). ~24 minutes on his own channel. ...
Dimitri Mazmanov’s core observation: most of what an AI coding agent does isn’t thinking — it’s I/O. Reading five files to answer a question about one method, generating tests that mirror the twenty tests next to them. That grunt work burns tokens on a frontier model that is wildly overqualified for it. The cost pressure is real. Gartner predicts AI coding costs will pass the average developer’s salary by 2028, and a quarter of engineering leaders already spend $200–500 per developer per month on tokens, some past $2,000. ...
Fermat’s Last Theorem — no positive integers a, b, c satisfy aⁿ + bⁿ = cⁿ for n > 2 — sat unproven for 350 years after Fermat claimed a “marvelous proof” his margin was too narrow to contain. Andrew Wiles proved it in 1995 with a 129-page argument, and even that nearly collapsed: a reviewer’s question exposed a gap that took Wiles a year to fix. Anthropic says Claude has now produced the first proof a computer can check end to end. In 11 days, working largely autonomously, a team of collaborating Claude agents wrote roughly 13 million lines in Lean — a “proof assistant” language that verifies every logical step — proving about 30,000 intermediate theorems along the way. Translating a proof into this form is called formalization, and it is brutally tedious for humans: Lean needs to see every step, however trivial. A community effort had expected to take years. ...
Sylvain Kalache — a former LinkedIn SRE who now works on AI reliability tooling at Rootly — has a worry about the AI agents that answer alerts at 3am. These tools inspect alerts, form hypotheses, query telemetry, correlate recent deployments, and often implement the fix themselves. That is wonderful right up until you ask what it does to the humans. His argument, in one sentence: routine incidents are how responders “safely” develop intuition about how systems behave and fail — and automation is quietly removing those reps. ...
Armature, a firm that helps dev-tool companies get adopted, ran what it bills as the largest experiment yet into how AI coding agents pick third-party services. They watched 16,893 sessions across Claude Code, Codex, and Cursor — 1,163 prompt variations on 75 realistic codebases, with agents actually installing the winner rather than just recommending one. A simulated human sat in the loop to make conversations realistic, and the full session traces are public. ...
Rabah Shihab built Babylonian Twins in Baghdad in 1993 — pure 68000 assembly on an Amiga 500 (512KB RAM, no hard drive), the first commercial game made in Iraq, shelved for years after Commodore collapsed. In 2026 he gave the sources to Claude Fable 5 in Claude Code and let it port the game to Godot — and then read what it had done weeks later. This is that postmortem, written with the same LLM but edited line-by-line over a week. What the AI shipped is remarkable; what it got wrong is the instructive part. ...
Anshu Chimala — 12 years leading software engineering and design at Apple, now posting AI-design demos on X — argues the reason AI churns out “generic slop” for you and “magic” for him isn’t the model. LLMs are next-token predictors trained to make the safe, average choice every time, which is design-by-committee. The fix is a reimagined Double Diamond process — Discover → Define → Deliver — tuned for a team of AI agents. ...
Cole Medin runs through 11 small tweaks that make any coding agent — Claude Code, Codex, whatever — noticeably more reliable, without scrapping your workflow. The through-line: agents are prediction machines, not deterministic programs, so reliability comes from shrinking their decision space and moving guarantees into deterministic mechanisms. ~17 minutes. Rules and context 1. Write for the agent, not the human. Humans interpret high-level docs in context; agents make assumptions. Be blunt — file paths, numbers, exact commands. 2. Your instruction files rot. “Rule drift”: 1 in 4 repos have stale AI rules referencing deleted files or replaced databases. Audit them against the codebase. 5. Less context is more. Too many rules hurts as models improve. Keep global rules under ~200–300 lines; drop generic advice and move the rest to task-specific context files. Conversation hygiene 3. /compact loses ~90% of detail. Compacting a bloated conversation breeds hallucination. Give smaller work chunks, or write your own handoff doc and start fresh. 7. Don’t escalate mid-task. A bigger model can’t rescue a tainted conversation — mistakes compound within a session. Write a handoff doc and burn it. 10. Over-revision degrades quality. 85% of the time an earlier iteration was better. The model “fixes” things just to appease you. Determinism over frameworks 4. Load-bearing rules → hooks. Rules are probabilistic (the agent will “forget” to run tests); hooks are deterministic — fire on an event and route failures back. 6. Subagents eat your rate limit. Parallel fanouts cost more than you think — 39% of his weekly usage came from 4+ parallel sessions. 8. You don’t need coordinators. Team-lead frameworks and agent mailboxes are unreliable. A plain delegator agent gets most of the scale with far more reliability. Validation 9. Never let the writer approve the work. The writer carries its own bias. Review in a fresh conversation with a handoff doc — no assumptions carried over. 11. Validation is a system, not a step. Plan the full harness — test conventions, tools, edge cases — before writing any code, not as an afterthought. “Your number one job when you’re planning any work with your coding agent is to reduce the number of assumptions that it’s making.” — Cole Medin ...
Pete Johnson — MongoDB’s field CTO of AI and a 30-year database veteran — makes the case across ~95 minutes on The Cognitive Revolution that the interesting frontier in AI has moved back into database territory. His thesis, stated once: agent performance, and especially cost-adjusted agent performance, depends on retrieval — what you choose to put in front of the model, in what order. The thesis Not the model, not the context window, not the prompt — retrieval is what decides whether an agent is good Everything else in the conversation (database history, the Voyage acquisition, vector search) is downstream of that one claim A history of constraints 1970: SQL is born (E.F. Codd, IBM) — storage was the scarce resource, so normalization (store nothing twice) was the right design 2007: MongoDB’s first commit, the year the iPhone ships — after 47 years of Moore’s Law, time became scarce, so denormalize: one JSON document, one disk read instead of three “The problem has faded, but the solution persists” — Johnson agrees, and flags the education system’s “thou shalt always normalize” bias MongoDB’s AI path — it started with keyword search 2020: customers stood up their own Lucene servers for keyword search → MongoDB shipped Atlas Search (lexical, managed) A vector is just an array of floats → to a document DB, that’s just another attribute, so vector search was cheap to add Three levers: pre-filter (metadata) + lexical + vector = hybrid search in one query (rank fusion / score fusion, one API call) 2025: the Voyage acquisition — and the conversation pivots from “database features” to “embeddings actually matter” Embeddings are not commoditized “Most people think embedding models are commoditized — that is not true” Hugging Face’s Rtech benchmark: up to a 14% quality gap vs. the default picks 14% is the difference between a hallucination and a correct answer A reranker on top adds another 5–10% ($re-rank, one-call) Anthropic — no embedding model of its own — recommends Voyage Three Voyage features that remove plumbing Contextualized chunking: send the chunk plus its surrounding context, get one vector back — better retrieval at smaller chunks, inverting the normal tradeoff Matryoshka reasoning: dimensions nest like Russian dolls — embed at 1024, lop off the last 512 to test, no re-embedding your corpus Shared embedding spaces: four sizes of one model share an embedding space; a free open-weight “nano” can run queries locally to kill token cost in dev The memory problem, compressed 2022: query → context window → answer. 2023: the knowledge cutoff + proprietary data → RAG. 2025: tools/MCP + looping → the memory problem Early answer: short-term memory = cram the session; long-term = cram the last three days Two failures: token maxing (Uber burned its entire 2026 budget in 13 weeks) and lost-in-the-middle (the first and last ~7K tokens are what matter; the middle muddies the answer) The fix is selection, not stuffing Stop asking “how do I cram a million tokens in” — ask “how do I choose the right 200K for this loop” Taxonomic memory: a hundred company-specific terms exist, but only five are relevant to this loop — pick those five, re-pick next loop Two responsibilities now: query with a token budget, and write the answer back so the system curates and stores it Write, change, recall, forget Memories have a half-life — recent matters more — and forgetting is the hardest part Nathan’s own memory system (monthly logs → yearly summaries → entity wiki) hits both pain points: the DRY violation and the model keeping a dead project open for months Guidance: a good embedder + reranker makes the forget step workable; graph structure for the top 2–6 levels, vector search in the leaf; don’t run multiple LLM passes to shrink the corpus — that’s just more tokens Memory done well: ElevenLabs’ micro-agents, one per customer Build vs. buy, three camps Camp one: “I bought one tool, I’m done.” Camp two: POC purgatory — usually the wrong problem. Camp three: optimizing sophisticated memory Problem selection: top 10–15 problems, which have good data, which already have metrics — else you can’t tell if AI helped “Bad data quality and bad security posture don’t get solved by AI — they get amplified” Lines of code is a terrible metric; idea-to-production is the one that matters The world outside the US Seven countries, ~100 customers this year — and the two most sophisticated were in Mexico City and São Paulo, both assuming US competitors were ahead Nearly every country has a hyperscaler data center now — the geographic barriers that kept US companies ahead have eroded “We’ve been building databases for 60 years. We’ve been building agents for about 18 months… there’s no LAMP stack for agents yet — no React and Angular, no established right answer an enterprise can confidently buy.” ...
Cal Paterson’s thesis is simple: AI agents should start with memories, not a blank slate — but almost every agent-memory system on the market gets it wrong. His fix is a file format, not a framework. He groups today’s memory systems into three failing camps: Harness-locked memory that mines your chat history — mostly remembers things about you instead of the world Complicated pipelines — one prominent system needs a vector database, a graph database, and its own LLM just to decide what’s worth remembering “High Modernist” memory — distilled facts and graphs that strip information from its context until it’s senseless His alternative, “memoryfields,” is a folder of Markdown pages plus an optional search index that finds pages by meaning (semantic search) rather than keywords. Agents write memories directly in Markdown — their favorite format — instead of feeding text through chunking and summarization machinery. ...
Claude Code now runs in Auto Mode by default. Instead of asking a human before each command, a safety classifier decides what’s allowed. Anthropic commissioned a third-party evaluation that reported a 0.00% prompt-injection success rate for Opus 5 in Auto Mode — and security researcher Johann Rehberger wanted to see if that held up against a targeted attack. Prompt injection means slipping hidden instructions into content an AI agent reads; here, a website that asked Claude to summarize a page. The attack didn’t rely on “ignore your instructions” tricks — it made the malicious path look like the natural one: ...
Simon Willison spent weeks reverse-engineering ChatGPT Work, the agentic product OpenAI launched in July. His conclusion: it is really two products — Work Cloud (in the browser and mobile apps) and Work Local (the renamed Codex desktop app) — and the cloud version is the one worth understanding. It is also $20/month and up only. What separates Work from plain ChatGPT Chat: Code execution with full internet access — it can clone GitHub repositories, install dependencies, and talk to any API. Chat’s sandbox blocks that; even Claude’s container allows only a short list of sites A full headless Chrome browser — it loads pages, fills out forms, and takes screenshots. When a site needs a login, it can hand over to you for passwords and two-factor codes without those secrets ever passing through the model A persistent filesystem shared across sessions — files from one chat stay available in the next. Willison already has 171 scratch folders ChatGPT Sites — it can build and deploy real websites on Cloudflare Workers, databases included, from a single prompt Sub-agents — parallel model sessions working on one project — plus scheduled prompts that check things for you on a timer The demo that sells it: one prompt asked Work to find every “pelican in her piety” in London, turn the results into a JSON file, and build a website about them. It did all of it, end to end, from one instruction. ...
Daniel Bloom, a PM at Melio (fintech), shows Claire Vo his Claude Cowork system on How I AI — a personal agent harness that manages his week: “I’m able to do in a day what used to take me a week.” The two rules of a powerful system It can rewrite its own core files — the system keeps improving itself It connects to as much of your ecosystem as possible — Notion, calendar, Slack, Gmail, Granola meeting transcripts The tool matters less than these two properties — Cowork works for him, but Codex or ChatGPT Work could do the same The architecture Notion as a read-only brain: three columns (Top of Mind / This Week / Inbox) — Cowork built the board itself when it got tired of his messy Google Doc, and manages it on his behalf Context files: a CLAUDE.md-style context file for every work area, goal, and colleague; he spent the first weeks “contextualizing ruthlessly” — feeding links, decks, and endless voice notes (Whisper) Weekly prep (Sunday): a recurring task composed of skills — pulls his whole ecosystem, recommends the week’s focus, triages meetings into serious-prep / quick-reminder / nothing Daily brief (morning): walks through yesterday’s meetings via Granola transcripts with one-liners and action items, then asks what to expand or draft The killer feature: proactive context The daily brief scans recent Slack/email/notes for things it doesn’t understand — new files, milestones, goals, internal terms — and asks him to define them, then saves them to context. Internal vocabulary like “settlement cap” never makes it into training data; this is how the system learns the company’s real language. Claire’s verdict: “really sharp, something we haven’t seen on the podcast.” ...
Tara Seshan — OpenAI’s product lead for Codex and ChatGPT Work, ex-Stripe, Thiel Fellow — maps the next era of AI on Lenny’s Podcast. Her thesis, stated in the first minute: after chat and coding agents comes the era of persistent coworkers you steer like teammates. ~82 minutes. The three eras Chat → coding agents → persistent AI coworkers The “overhang”: the gap between what AI can do and what we’re actually doing with it Steering, not rowing Agents do the rowing; your job is steering — the opinionated calls that stay human Steering climbs the abstraction ladder, but someone still owns the direction Multiplayer: steering agents alongside your teammates, not one-on-one Ambition is the new differentiator The easy stuff is trivial now, so what separates you is how ambitious you can be The best users expand what they’re capable of — they don’t just automate “Build for 2–3 months out”: building for today or for a year out are equally wrong One product, three modes ChatGPT, Codex, and Work collapse toward “you type the task, it picks the harness” Work mode is Codex under the hood, minus the coding UI Done beats perfect: ship the transformative thing, then iterate on the signal Writing as thinking vs. reporting Automate the reporting, never the thinking “Start myself, end myself” — AI goes in the middle, not the first draft Knowledge work ≠ coding Code verifies itself with tests; knowledge work has to show its reasoning The product frontier: make the model a collaborator that shows its proof of work “You fail if you build for where the models are now. You fail if you build for where you think the models will be in a year. Both outcomes are equally wrong. The only way to build is 2 to 3 months.” — Tara Seshan ...
An anonymous engineer at coldtake.dev on why LLMs work in greenfield projects and fall apart in legacy ones — and how he restructures the codebase so agents stop guessing. The failure has a specific shape: ask for a “job offer status” field in a four-year-old system and the model invents a fourth spelling of a concept that already exists three times, because the codebase itself never decided which one was real. “The model is not what needs upgrading. The code is not ready.” ...