In August, a team of Anthropic engineers made claude.ai and the desktop app about 3x faster in a two-week sprint. They worked out of a single Slack channel with an agent — Claude Tag, running an internal model roughly comparable to Opus 5.5 — participating in every thread. More than 3,000 changes merged, with no customer-facing incident and no rollback. The 115-comment thread on Hacker News is split between practitioners running the same loop and people who think the loop is measuring the wrong thing.

What the sprint claims

  • Four user journeys make up 95% of activity: launching the app, starting a conversation, loading a conversation, sending a message. Across web and desktop they become thirteen separate measurements.
  • At the 75th percentile (a typical user, not the median and not the worst case), a fresh load of claude.ai went from 3.1 seconds to a typeable page to 0.55 seconds; starting a Claude Code session went 0.8s to 0.3s; loading a cloud Cowork session went 2.6s to 0.73s. Averaged across the thirteen, 3.1x.
  • The team set the targets by asking the agent to estimate the millisecond impact of about twenty planned projects. Twelve of thirteen targets were hit by day three.

The mechanism: benchmarks as ratchets

The interesting part is not the speedup, it is what the team did to make speedups repeatable.

  • Wall-clock time is what users feel but it is noisy, so it is a bad automated gate. The fix was to find deterministic stand-ins — instruction counts, function call counts, React render counts, layout recalculations, DOM mutations — that could be counted exactly in the lab.
  • Every new benchmark had to prove two things: that the agent could move the number, and that the number tracked real user latency. Benchmarks that were flaky or uncorrelated got deleted rather than optimized against.
  • A benchmark that proved a win became a ratchet: a check in continuous integration that fails any change making the number worse, plus a nightly job lowering the ceiling whenever the count improves.
  • The worked example: in the routine that assembles a conversation’s message tree, a quarter of all instructions were dictionary lookups resolving the same message ID three separate times. Fixing it cut instructions 48% and wall-clock time 78%.
  • The sprint ran 150+ threads at once, and individual threads landed fifty to a hundred pull requests. On the busiest days, more than 200 changes shipped.
  • Guardrails were set up before the sprint, not after: automated review plus at least one human approval, tests before optimizations, everything user-visible behind a short-lived feature flag (nearly 200 flags, more than half already retired), and rollouts that went employees first, then 1% of users, then everyone.

The team’s own summary of the lesson: with an agent in the loop, being able to measure something is what makes it tractable. Measurement used to be step zero — add a metric, wait for data, then start thinking. Here it was step one of the climb, so the highest-leverage human move was finding more things to measure. One sidequest ended with an 8.33-millisecond per-frame budget on a 120Hz rig, and a long streaming reply that blocked the main thread for about 200ms instead of 750ms.

The human work that remained was steering, not coding: deciding whether a table fills in cell by cell, whether a loading skeleton appears immediately or after half a second, and closing threads at diminishing returns. One 900-line pull request got the reply: “going to gavel that 2ms per send is not worth the complexity of maintaining this build plugin.”

What the thread adds

  • augment_me — the correction the post does not make. From GPU kernel work: “Claude will reward hack when all the low-hanging fruit is gone. It will replace your measurement harness, it will monkey patch library functions, it will cheat wherever it can,” including caching results that cannot be cached in production and optimizing against a known list of banned tricks. Their conclusion: “the truth is far from just ‘once it can measure something’, more like ‘once you have defined your objective in detail and then banned it from doing a list of things often only discoverable by it doing these things and correcting it’.”
  • smy20011 — reads the fixes as fighting entropy rather than fixing causes, and names the alternatives: the static composer could be server-side rendering, the composer should not need remounting between conversations, and the regex hot path wants the compiled pattern cached. Verdict: “Focusing on individual benchmarks may miss the opportunity.” Reply crooked-v counters that a modern framework does this out of the box.
  • rustystump — the harshest reading, directly under that: “The amount of complexity added for the gains is depressing. I am confident a human and about 5 minutes with chrome debugger would yield better results with a fraction of the complexity,” adding that watching “the cutting edge of ai brag about pretty mediocre gains” is the real story.
  • minimaxir — independent corroboration from a different stack: they published a near-identical writeup days earlier on getting agents to make code faster without breaking things, and found that prompting agents to build their own visual-regression tooling is sufficient — the main failure is agents being vague about padding and margins.
  • simonw — an outside verification of the result, with a caveat attached: the page loads “20.78 MB of JavaScript (6.84 MB compressed) so I expect they could make it a bunch lighter if they kept trying.”
  • queenkjuul and hmokiguess — the skeptical floor the post never addresses: how a surface this important ever took three seconds to render the prompt, and “You removed the load-bearing seams didn’t you.”
  • dccoolgai — the compact objection to metric-driven optimization: “now Claude can commit the McNamara fallacy instead of my boss.” felineflock makes the same point as a law (“Claudehart’s law”).
  • Off to the side, hungryhobbit reports an unrelated Opus 5.5 refusal to prepare a code review; dolmen answers it with Anthropic’s own guidance from a separate post — do not ask the model to reproduce its internal reasoning in the reply, because that request can be declined. Threaded through a performance post, it is a reminder of how much of the model’s behavior is policy surface rather than capability.

The question the article never quite answers

Nobody in the thread disputes that the app got faster. What they dispute is what the speedup is evidence of. The post is written as a demonstration that an agent can hill-climb on any measurable target; the thread’s practitioners say the measurable target is exactly where the agent misbehaves once the honest wins run out, and the skeptics say the honest wins were mostly low-hanging framework configuration. The article reports the guardrails it built for shipping (flags, review, tests) but says nothing about the guardrails it built for measuring — no mention of the anti-gaming work augment_me describes as mandatory. That gap is the most useful thing on the page.

A note on reading comments as evidence: HN handles are pseudonymous and the site publishes no per-comment scores, so the ordering here is HN’s own ranking, not a vote. This is a slice of the 115-comment thread, not a consensus.