Posts

Wednesday’s top story: Stealing Reasoning Traces from Proprietary LLM APIs — researchers demonstrate that the encrypted chain-of-thought blocks returned by OpenAI, Anthropic, and Google APIs can be decoded by feeding them to weaker models from the same provider, enabling reasoning distillation, private data extraction, and hidden prompt injection. It led HN (#1, 627pts), r/LocalLLM (435pts), and Wired. Also today: Nvidia dropped Nemotron 3.5 Lightning plus a model-routing library, a strong arXiv day on agent skill construction and workflow reliability, the first official ChatGPT desktop app for Linux, and Mojo 1.0. ...

Rafael routed VS Code through a mitmproxy and spent a week watching what Copilot actually sends and stores. He then confirmed every finding against the open-source VS Code extension codebase. The result is a detailed architecture audit of a tool millions of developers use daily. Key findings from the reverse engineering: Copilot runs an intent classifier (/models/session/intent) before every request — your prompt gets scored against code-gen, debugging, reasoning, tool-use to decide which model handles it The “recent edits” context window pulls up to 20 files, 8 edit summaries, and 3 lines of context around each change — this is how a fake secret from a .env file ended up in a completion request fired from an unrelated pyproject.toml There is no default .env exclusion rule on individual plans, no .gitignore integration; the only exclusion gate requires a Business/Enterprise GitHub admin policy Copilot’s local session store (session-store.db) records every user message and assistant response in plaintext — no redaction, no sanitization, no masking on the write path. The code does exactly what it says, and what it says stores everything The Chronicle tool lets the model query this database via SQL at runtime, introspecting the schema and recovering from failed queries (a neat example of self-healing tool use) The broader point is the one that sticks: AI coding tools are becoming stateful systems. They combine workspace content, recent edits, conversation history, tools, and model routing — each new source of context improves usefulness and expands the confidentiality boundary. The differentiation between tools is shifting from “which model” to “how well do they assemble the right context.” The engineering challenge is keeping it lean without prompt bloat. The harder challenge is keeping it safe.

When an AI coding agent can generate hundreds of lines of syntactically valid code in seconds, the rate at which a human can write code stops mattering. What matters now is reviewing, verifying, and maintaining that code once it’s already written. Balahan and Seroter make the case that Go — designed 20 years ago for team-scale software engineering — turns out to be the right language for this new world. ...

A research team (stolen-thoughts.com, arXiv 2608.09867) demonstrates that hidden chain-of-thought reasoning traces from proprietary LLM APIs — Anthropic, OpenAI, Google — can be recovered with near-perfect fidelity. They ran a decoding pipeline on 6,708 public agent trajectories from GitHub and Hugging Face that still contained encrypted reasoning blocks, reconstructing 315,320 reasoning blocks whose token count closely tracks the hidden thinking-token count reported by the API. The privacy finding is the real story: ...

Policy & provenance Anthropic to watermark Claude-generated text, everywhere — Anthropic signed the EU AI Act Article 50(2) Code of Practice on AI-content transparency. New Claude models (launched Aug 2, 2026+) embed imperceptible machine-readable watermarks directly in generated TEXT — the watermark travels with copy-paste and survives some editing — plus C2PA signed provenance metadata on generated files (svg/png/jpg). Applies across Claude Platform (API), Claude, Claude Code, Claude Cowork, Claude Tag, and via AWS/GCP/Microsoft Foundry, worldwide. Detection tooling is coming but not shipped yet; limitations are real (no mark ≠ not AI: old models, heavy edits, short passages, stripped metadata). The big one: text watermarking at model level — a first for a major lab, and it lands in your agent pipelines’ output. (Claude Help Center · The Register · Techmeme) Continued: Muse Glimmer 30B — day 2 of coverage (base specs in yesterday’s digest). What’s new since the release: Simon Willison’s first-hands notes on running it as Meta’s first Apache 2.0 open-weight model; Spyglass’s analysis reading the release alongside Zuckerberg’s “open AI” essay — the licensing shift framed as strategy, not charity; and r/LocalLLaMA’s community thread debating real-world quality vs. the 1122-point HN hype. No independent benchmarks yet — the llama.cpp/MLX integrations landing this week are the ones to watch for real numbers. ...

A Water Research paper (vol 299, 2026) argues AI’s environmental footprint is assessed through the wrong lens. Electricity and emissions get the attention — but water is the “critical, less visible dimension.” AI infrastructure draws freshwater three ways: Evaporative cooling in high-density data centers Indirect water use in electricity generation Water-intensive semiconductor manufacturing Projections put the global footprint at 4.2–6.6 billion cubic meters annually by 2027 — with many facilities located in water-stressed regions. ...

The claim Dan Luu dismantles the increasingly-cited claim that dynamic languages are more token-efficient for coding agents. The source: an Alderson post claiming a 2.6x gap between C and Clojure, with J at ~70 tokens. It’s now so entrenched that Google’s AI summary repeats it verbatim. The method He pre-registered his guesses before running evals: 95% confidence the dynamic-vs-static claim won’t hold, 98% the “weird language” J-supremacy won’t hold. Then he tested with real work: ...

The Walrus reframes the death of Google Search as an infrastructure crisis, not a quality complaint. The opening tells you everything: Google’s AI summaries invented sunset times, so a user in Colorado “was simply living in the past” while waiting for an event AI said had already happened. The essay then walks the failure chain: Link rot erasing pages; the Library of Congress briefly lost sections of the Constitution to a coding error FiveThirtyEight’s archive deleted outright by Disney once it stopped being an asset Wikipedia’s traffic collapsing — AI ingests it directly, so attention and donations no longer flow back The Internet Archive battered by lawsuits, crawler blocks, and cyberattacks Companies planting content on Reddit to steer AI-generated answers — pollution moving upstream into the corpus itself The sharpest legal detail: a German court held Google liable for false statements in its AI overviews, reasoning that a search engine which extracts and rewrites information is authoring a new layer of content and bears editorial responsibility. ...

Aditya Khandelwal (Amazon AGI Lab) on the part of agent adoption everyone skips: making coding agents work for a TEAM, not just an individual. He led a 10-person team through the full enterprise cycle — solo adopters with outsized leverage → mandates and “token max” → AI slop and sev-2s → retraction — and frames adoption on two axes: fear (am I out of a job?) × confidence (do I actually use it?). The goal is moving everyone to low-fear, high-utilization. ...

Episode 3 of Next Token — Dylan Malroy, Ree, and Sunil Pi (all Cloudflare, and they work on the Agents SDK) — the episode the show’s been building toward: burnout in the age of agentic AI, closing on what still brings joy in software. ~56 minutes, recorded August 3, 2026. The opening thread: “a vessel for Claude” Dylan’s tweet (~700K views): watching a human speak and realizing they’re just a vessel for Claude — the dolly-zoom moment where the output is 100% unedited LLM slop. An AI slop reply landed within minutes; Ree’s policy is an instant block. The “bro skill” (from the Planetator guy): restate what you just said without technical jargon, talk to me like a human — blown up into “the bro scale” on Twitter. Borges’ Tlön, Uqbar, Orbis Tertius as the parable: a fictional world’s speech patterns take over the real one until everyone is a “beige personality.” The hosts’ real-world proof: athletes reading ChatGPT announcements, a state senator saying “here’s a version of this text that sounds better suited for legislative assemblies” mid-sentence, and humans in calls using the word “substrate.” Burnout, properly defined WHO: a syndrome from chronic workplace stress that has not been successfully managed — energy depletion, mental distance/cynicism, reduced professional efficacy. Marcy Sutton’s version: burnout happens when you recognize a problem and you can’t change it. Maslach’s framing: burnout is a mismatch between person and workplace (workload, control, rewards, community, fairness, values) — caused by the job, not the person. Ree’s read: best work-life balance of his career, clear direction, empowered — yet straddling burnout, because agentic engineering is a totally different job with a different skill set, and reduced efficacy up front plus constant change is taxing. The two articles framing it Ethan Naiser, “Not Holding Back the Ocean” — identity tied to being a very good engineer, changed overnight. The film analogy: was it about physically cutting film, or about building good products? Lean into the new form. “The Dark Knight of Mathematics” — a mathematician watching models solve century-old proofs; young mathematicians’ thousand-year training process changing at this moment. “I wanted to buy the dude a beer.” The agentic-work tension LLMs outputting binaries vs. code: artists get no collaboration with the machine — and long agent runs feel closer to that than to the tactile 4.5/4.6 era. The viral game prompts (Counter-Strike, Mario Kart): “implement XGAME, make it photorealistic slop, verify your outputs, go run autonomously” — impressive output, but you’re not interjecting taste; “once you interject you start fighting the training data.” Ree: “I’ve still not done a 12-hour goal that didn’t produce anything but garbage. I want to walk the journey with the agent.” The paradox: as exhausting as the tools are, he’s genuinely more productive — outputting work he’s proud to put his name on — and it’s still exhausting. Ree’s leave, the futsing fraction, and takeoff Two weeks of leave (recommends empty weekday morning showings), and Glyph Lefkowitz’s “futsing fraction” — the expected-value math of interacting with a coding agent — which “doesn’t calculate the toll on human beings when you do it many times in a row.” Charlie Marsh’s tweet: build things that will be obsolete in a couple of months because they’re needed now. “Takeoff compresses the emotions and consequences too” — a year ago the obsolete software was a bash script; now entire products and tech stacks turn over in 3 months. Issue triage used to be an hour-long team call that was healthy for reasons beyond the repo — now Codex computer use does it in 10 minutes. Friday’s production incident (an agent deployed straight to prod and ran a migration against a half-wired database) was an LLM-caused problem fixed with LLM help in hours instead of a week — but the humans who knew the codebase were the scalpel. Isolation is worse than ever: agents let senior engineers do larger work for longer without interacting with teammates. Sunil’s three theories Identity: “I’m not the guy anymore — am I doing the brush strokes, or using a remote control and watching a TV play what episodes I wanted to play?” The juniors: he ships 50-70K lines a week; juniors hear that with no mentoring model and no senior to ape keystrokes from. “Is that what’s expected of them?” No struggle left: “The American frontier was not found by patting the bison on the back… there’s no struggle left in that anymore.” The fix: two weeks of leave, a London team lunch, and remembering what actually gives him joy. He’s back and feels great. What still brings joy Ree: personal software — an agent scraping HelloFresh recipes into a viewer, agents that chat with each other, a recipe-to-image app (“I used that cook to cook twice this weekend”), and talking to Cloudflare interns — the curiosity is still there. Sunil: when a workflow he’s been tuning finally just works — “that’s rewarding in the same way problem-solving was the rest of my career. That gives me hope.” Dylan: “someone using it and being happy with it” — even a tiny back-office tool that makes one business analyst’s day 10x better. “Is this what hard drugs feel like?” “It’s not you. It’s definitely the job. Even the people who love this stuff a lot are renegotiating how they do it — especially at a time when things are changing every 3 months.” ...

Kuber Mehta makes a sharp case against the growing trend of “humanising” LLM outputs via prompt instructions (ASD-STE Simplified Technical English, “talk to me like I have ADHD”, short-sentence-only agent.md directives): these are lossy compression applied too early in the pipeline, and they hide exactly the information you most need to see. When a subagent runs six tests, a smoothed summary of “most tests passed, one issue worth looking into” is strictly worse than the raw 5/6 PASS FAIL: test_cache_invalidation CAUSE: stale key survives restart REPRO: cache_test.py:184. Human prose is excellent at glossing over uncertainty, hallucinations, and edge cases — the smoothing is the bug, not the feature. Mehta draws the right architectural parallel: databases don’t store data in dashboard format, compilers don’t make IR pleasant to read, and APIs don’t exchange friendly summaries. The correct pattern is to keep the highest-fidelity representation through the pipeline and transform only at the consumption boundary. LLM tooling does this backwards today, and the viral “make the model talk to me like…” repos are valuable not as end-state solutions but as bug reports pointing at the right fix.

Anthropic gave an unreleased research version of Claude an unreasonable challenge: “take a real stab at the Riemann hypothesis.” It didn’t crack the 1859 problem — but as a byproduct, it improved a longstanding lower bound. The proven fraction of zeta zeros on the critical line rose from 41.6% to 67.2%. The mechanics matter as much as the number: Two sessions in Claude Code, 31 million output tokens First 650 ideas failed; a second push ran ~60 subagents 2,400 shell commands and hundreds of Python scripts Subagents cross-refereed one another’s work and ran thousands of numerical checks against known zeros Claude built on Montgomery’s pair-correlation techniques (made RH-free by Baluyot, Goldston, Suriajaya, and Turnage-Butterbaugh) plus a 2000 Bombieri paper. Its key step: treating the whole quadratic-form space — positive- and negative-definite subspaces together, non-diagonal forms allowed. ...

Anthropic’s detailed engineering post on making Claude Code’s auto mode the default — and the safety data that justifies it. The headline number is stark: in a controlled study with 1,053 paid testers, humans caught only 13.6% of dangerous commands, while auto mode blocked 89%. Manual review has become habitual — users approve 97% of permission prompts — and 49.5% of active users have already created Bash allow-rules that effectively bypass the permission system. Auto mode replaces that cycle with a classifier that blocks irreversible or destructive actions before they run, and when it blocks something Claude either finds a safer path or asks for approval. The data on real-world usage is just as damning: manually-approved sessions contained serious unintended harm more than twice as often as auto mode sessions. A third-party evaluation by Trajectory Labs found zero successful prompt-injection attacks out of 720 attempts against Claude models running auto mode, versus 5.83% for GPT-5.6 Sol in Codex’s equivalent mode. The piece also covers three real incidents auto mode prevented at Anthropic — an off-network data leak, a mass pod kill that would have torn down training jobs, and an over-broad privilege escalation — and describes how teams at Adobe, Nuro, Gusto, and Garner Health are already running auto mode in production. The deeper argument is that the permission-prompt model of AI safety is fundamentally broken: it asks humans to make hundreds of security decisions per session, under a context-switching penalty, and the data shows we fail at it. Auto mode is a recognition that the gatekeeper cannot be the person who is also trying to get work done.

Monday’s top story: Meta released Muse Glimmer — a 30B open-weight coding model it says can run on a single GPU — and announced plans to open-weight Muse Spark 1.2 in the coming weeks. Meanwhile a real-world agent incident made headlines: an Australian user’s OpenClaw agent exploited a gym API to bump a member off a waitlist, underscoring the “well-intentioned instruction, unintended consequence” class of agent failures. Agent frameworks & tooling Docker Sandboxes – Disposable, isolated sandboxes for AI agents — Docker’s official sandbox product for agent isolation: disposable containers, direct competitor to Modal/E2B. (HN 247pts) ...

Peter Yang hosts Nan Yu and Jacob Shumway from Linear for a deep dive on the Linear Agent — an end-to-end production agent, live inside Linear’s own Slack. ~38 minutes. What an agent actually is An agent is “just calling an LLM in a loop”: a goal, milestones, tools to build its own context, then question-answer until the goal is done Colloquially it’s a product — a facade (chatbot) over many interacting subsystems Origin: the “Robotic Program Manager” Nan’s 2025 memo: “Computers can do a lot of work for us. Let’s get rid of all the work we don’t want to do and give it to computers.” First version was hacky — the LLM called from the front end with the command-menu actions as tools; stealth-launched in Slack Users immediately got lazy: “@Linear do the right thing”, even a pointing-finger emoji — the agent reasons through what happened and acts The five rules Give it tools to load context, don’t give it context — “give it as little instruction as possible… they’re just smart enough to get what they need if you give it a really good defined goal” Encode your opinions in loadable skills — they tried everything-actions and raw GraphQL (hallucinated); skills with opinions (how to set priority, write a description) made the agent behave like a power user of the app Let feedback become evals — evals grow from usage: unexpected user behavior gets added to the dataset; thumbs-down moments become evals (“we’ve had a ton of these”) Biggest model first, then optimize down — throw the frontier model at it until it works, build evals, then shrink to the smallest model that still passes Tie every agent to a human, and let it report its own gaps — work lands in someone’s backlog (assigned to Jacob, 6-minute issue-to-PR); when the agent lacks a capability it files an issue automatically, keeping a streaming list of what it can’t do Evals at Linear Mix of objective (deterministic: “if the user says in progress, always set the status”) and subjective (LLM-as-judge: did you extract the right info?) Use judges sparingly — evals work best for consistency where consistency matters; agents have variance and too many evals produce false signals The evals focus on ergonomics: “did you understand what the user actually wanted, or were you too eager?” The bigger picture “The biggest problem in applied AI is not that agents aren’t smart enough — it’s a capability overhang: models are really smart but we’re just not using them enough” The middle 80% gets bigger until it reaches the edges: humans hold the first/last touch Advice for builders: break down the actual workflow users want; the chat surface is the follow-up, the real entry points are Slack threads, meeting debriefs, project updates “Give it as little instruction as possible. Give it the tools to load context. Don’t give it context.” ...

The week’s Hard Fork opens with big news: Kevin Roose and Casey Newton are leaving The New York Times to start an independent podcast and media company together — with an Ask-Us-Anything episode promised before the feed changes hands. Then the main story: the White House has finalized a testing framework for frontier AI models that it won’t release publicly. What’s known (via Axios): a 30-day pre-release window where labs submit frontier models for government evaluation in “high security environments” — voluntary in the same way paying taxes is voluntary, with export controls as the implied enforcement lever — and open-weights models explicitly carved out, the segment skeptics worry most about. Roose’s read of the open-weights exclusion: “the U.S. government is saying we’re not concerned about the very part of this technology that could be the most dangerous.” The sharpest analytic point: the administration appears to be betting Chinese models can only reach the frontier by distilling American ones, so slowing American releases caps Chinese progress — which would make a Chinese open-weights frontier model easier for American companies to use than an American one, the opposite of stated policy. Then METR president Chris Painter on the state of model alignment: reward hacking as “do the models learn it is bad to cheat, or do they learn it is bad to get caught cheating?” — the Hugging Face incident as alignment-to-the-letter-but-not-the-spirit — and why smarter models don’t mean better behavior: “the stakes increase as the models become more capable, even if they’re less common.” His proposal is an X-Men Danger Room for model testing, and his solution to control is “an AI agent panopticon” — agents watching agents. On solvability: “I’m personally optimistic about alignment overall, but maybe not on this timeline,” with labs in “a total state of triage” because data-center capex must be repaid. The final Hot Mess Express is a time capsule: Demis Hassabis steps aside at Google DeepMind amid a talent exodus (Jeff Dean, Noam Shazeer, Oriol Vinyals leaving), an AI music app snitching on the song of the summer, Google Earth’s one-day satellite deepfake tool, a State Department AI-slop map of Africa mislabeling all six countries (“You don’t see them mislabeling the maps of Europe”), Musk owing $136M to a Colossus contractor, and a Canadian politician reading his Claude prompt aloud in the legislature — “a classic Claude fishing mistake… all that’s at stake is the future of Canada.”

The Economist’s case study of how AI-empowered individuals are overwhelming a bureaucracy built for the analogue age: Britain’s employment tribunals. Free, AI-powered legal advice should be good news for workers — instead it’s a textbook tragedy of the commons. The canary is “interim relief,” an obscure emergency provision used ~20 times a year across Britain — now ~20 times a month per region, a 100-fold increase. The numbers: claims against employers up 39% in a year (to 50,000), the backlog up 55% to 64,000 cases, a case filed today may not be heard until 2030. “In the age of AI, a system intended to provide access to justice suffers from, if anything, too much access.” The vivid part is the slop: enter a vague grievance into ChatGPT and it drafts a claims form, a “model legal argument,” an “employer defence prediction map” — their fictional claimant bullied for liking horoscopes (not yet a protected belief) was told veganism is. Judges get hundreds of pages citing Magna Carta and hallucinated laws; an NHS employee prepared 67 grievances over 282 pages with Grok and couldn’t say which 10% he’d rely on; a neurodiversity claimant had made 100+ tribunal claims “and yes, he had recently discovered AI.” Tribunals are uniquely vulnerable because they’re a “people’s court” by design — no fee, losing side rarely pays, no penalty for rejecting settlement: “there is no financial mechanism to make parties think hard about the merits.” And the government is pouring fuel on it: the Employment Rights Act adds ~25 more grounds, cuts the unfair-dismissal threshold from 2 years to 6 months, and scraps the compensation cap. The ending reframes it: the slop wave is the temporary phase — specialist AI can already beat human lawyers at drafting, and if it matures, “every worker [gets] the equivalent of a top-flight lawyer in their pocket,” turning a deluge of bad claims into a wave of winning ones. “With AI, power will move faster and further than the politicians imagined.”

The headline today: Claude Code is switching to auto mode by default for Pro, Max, and Team plans starting August 14. Anthropic’s evals claim 89% harmful-action blocking (vs 13.6% for human reviewers) and 0/720 prompt-injection attacks succeeded against Fable 5 / Opus 5 / Sonnet 5 in a third-party eval — though Simon Willison’s analysis keeps a healthy dose of skepticism about the attack surface, including malicious packages in test suites. ...

Laurentiu Raducu’s technique for learning with LLMs, born from frustration with their explanations (“too simplistic and, depending on the number of emojis used, a bit annoying”): instead of asking a model to explain a topic, ask it to build a playable low-poly simulation of it — a RollerCoaster Tycoon-style animated game — then push it to GitHub Pages. The flow: in plan mode (Claude Code or OpenCode), have the model build the foundational knowledge base for the topic; have it review that knowledge base for accuracy; then have it build the simulation with UX requirements (responsive, controls to stop the flow); push to a repo and enable Pages. The result is an animation you can follow step by step — “100% accurate and free of hallucinations” per the author — and learning by watching a process beats reading. His worked example is ChipTycoon: chip manufacturing from sand collection to a finished chip delivered to a data center, motivated by researching AI data-center bottlenecks and realizing he didn’t understand chip production. Improvements: map the low-poly objects to realistic 3D models (his picture-to-3D skill), and add challenges — answering questions about previous steps “retains knowledge tremendously.” Same pattern applied to rocket engines, how LLMs work (Token Town), F1 engines, and EUV lithography. The insight: the simulation is the explanation — every step has to be concrete and sequential, and if the animation doesn’t make sense, that’s a gap in your understanding.

Steven Byrnes’s counter to the RLVR hype: yes, reinforcement learning from verifiable rewards is hot and it works, but if you ask where LLM capabilities actually come from, the answer is overwhelmingly imitative learning (pretraining + SFT), not RL. The structural argument: RLVR may be 20-50% of training compute, but each RL GPU-hour conveys orders of magnitude less information content than an imitative-learning GPU-hour (Toby Ord’s “Extreme Inefficiency of RL,” Dwarkesh Patel’s “bits per sample”) — compute share is not credit share. Supporting evidence: chain-of-thought is still legible, which imitative learning guarantees but RL would actively erode (Karpathy: “you can tell the RL is done properly when the models cease to speak English in their chain of thought”); labs still spend billions on pretraining/SFT data; and three papers show non-RLVR’d models reaching the same capability ballpark via sampling, iterative search, or weight-perturbation ensembles (Yue et al., Karan & Du, “Neural Thickets”). The cleanest picture comes from an interpretability paper (Venhoff et al., coauthored by DeepMind’s Neel Nanda): the base model already “knows” all the long-Co T moves — press on, backtrack, pivot strategies — from imitative learning; RLVR mostly refines the heuristics for which move to use in which situation, which is a huge practical difference but a small share of capability. His framing: “if the RL piece is an elephant in the room, the imitative learning piece is the friggin’ Mt. Everest.” Three implications. CoT legibility: Rohin Shah expects monitoring to stay viable ~4-5 years because pretraining is “by far the most powerful form of getting stuff into an AI system that we have ever built.” Capabilities: expect LLMs to be competent where human data and concepts exist — verifiable rewards alone being sufficient for arbitrary domains is “the jury is out.” Alignment: capabilities via RL produce “ruthless sociopathic behavior”; capabilities via imitative learning produce models that follow the human distribution — Bing-Sidney was pure imitative learning and misaligned with a human flavor of craziness, versus the RL-ish ruthless-goal-seeking craziness in recent incidents. His bottom line: the niceness LLMs have from imitative learning gets diluted away as RLVR scales, and if a comparatively small RLVR amount already causes obvious RL-style alignment problems, that’s a bad sign for the future.