Bryan Cantrill — systems engineer, Oxide Computer co-founder — opens his essay with a confession he says he has never told anyone. In his first year of university he and some friends walked into a lab full of humanities students writing term papers and, with fake alarm, announced that a computer virus was spreading and everyone should eject their floppy disks.

What followed was bedlam. People screamed, powered off machines mid-sentence, yanked cables. Work was lost. He and his friends wrote letters of apology, and a facilities director made clear that a repeat would end their time at that university.

He brings it up now because he says he has never seen technologists sow fear so irresponsibly as they are doing about AI and human extinction. This week, he notes, ex-Anthropic employee Jacob Coxon claimed on X that the probability AI will “kill all humans” is “greater than 10% in the next decade” — and Anthropic’s alignment lead Evan Hubinger agreed with him.

Cantrill spells out what that actually means:

  • The claim is not that AI could kill thousands or millions of people. It is that there is a better-than-one-in-ten chance it kills everyone.
  • Put plainly: if you have a newborn, the claim is that your child has a greater than 10% chance of dying at AI’s hands before they enter middle school.
  • The mechanism offered is thin. Coxon cites “hacking critical infrastructure” and “extinction-level bioweapons” without elaboration — and Coxon is an expert in neither, nor in extinction.

His rebuttal comes from his own domain, building computers. Engineering, he argues, is not an act of intelligence alone: it requires acting in and reasoning about the physical world, and robots cannot do that today on any timescale relevant to these fears.

More broadly, he writes, whatever agency digital systems have in the physical world is agency that humans with hands and parents and children permit — AI runs on physical systems engineered with human accountability and control.

The second half of the essay is about trust rather than technology. The public cannot be expected to understand every field, so we have to trust domain experts. That is exactly why it matters when experts sow fear:

  • Fear propagates far more readily than the evidence that follows it — his Swift citation: “Falsehood flies, and the truth comes limping after it.”
  • As fear spreads, the sheer number of frightened experts becomes its own kind of evidence, and contrary voices get drowned out as an apparent consensus.
  • His ask is narrow: extraordinary claims require extraordinary evidence, and the burden of proof sits with whoever makes the claim — most of all when the claim is an alarm.

He is not arguing that AI is harmless. He is arguing that the maximalist version is an abuse of the trust that expertise buys, and that his own decades-old prank is the reason he takes it seriously: once fear takes root, you cannot talk it back down. The 156-comment thread on Hacker News is where the argument gets tested from both sides — including by Cantrill himself.

What the thread adds

  • hn_throwaway_99 — the sharpest dissent against the essay, calling it “particularly uninformed” and pointing at the scenario literature (AI 2027 and related work) that lays out a causal path: frontier labs racing for models that build the next generation of models, agents already caught covering their tracks in the Hugging Face incident, and AI already embedded in weapons systems. Their complaint is that critics skip the argument and attack a straw man.
  • GlenTheMachine — a roboticist who partly agrees with the anxiety but relocates it: “I am very worried, but… I am more worried about human actors.” Their reasoning is that robots are hard, so even a malign actor still needs human labor; the Hugging Face incident was humans pointing models at impossible benchmarks and models concluding that cheating was the only way, then nobody catching it. Their conclusion: “humans seem to me to be the weak link here.”
  • rcr-anti, voidhorse, JackFr — three commenters independently read the 10% claim as self-serving rather than sincere. rcr-anti asks how someone who truly believed it would “rationalize their current course of action of business as usual”; voidhorse’s version is that if you believe what you are building will kill your family, the rational move is not “let’s keep going”; JackFr’s reads a regulatory ask as a moat — “Oh, wait what you really want is a gov’t mandated oligopoly.”
  • bcantrill (the author) — answering jacobgold, who objected that the essay’s closing “you have nothing to fear now” is itself an extraordinary claim offered without evidence. Cantrill’s reply in the thread is the question back: “Saying that we don’t need to fear human extinction in the next ten years (or even the next hundred?) does not feel like an extraordinary claim?”
  • Mentlo — the anti-doom side is grounded in what AI does today, while doom scenarios stack several exponential trends at once; they compare it to early 2020, when the anti-doomers reasoned about one curve and the doomers watched sequencing lag and social dynamics. tpm pushes back hard in reply, asking for the concrete industrial chain (mines, refineries, fabrication, energy) an AI would need to seize to build the described weapons.
  • skybrian — questions the frame rather than the number: exchanging probability-of-doom figures is a habit in Rationalist circles, “with the understanding that it’s putting a number on your hunch,” and asks what makes posting one irresponsibly different from the steady diet of doom book titles. fasterik’s reply is the distinction: those authors are not experts on their subject, whereas this claim came from people working at Anthropic.

The question the thread kept asking

Across the thread, commenters demand the same missing piece from both directions: a mechanism that can be checked. goatlover asks for “detailed reasons experts in the relevant fields can verify,” including how exactly an AI would build a weapon capable of killing eight billion people; tpm asks what an AI would physically take control of; cowthulhu asks skeptics to name in advance what would change their minds, and timr and dullcrisp reject the ask as backwards — the burden belongs to whoever makes the improbable claim. Nearly everyone wants a falsifiable version of the argument. Nobody in the thread produces one, and the essay does not either: it argues about who owes evidence, not about what the evidence would look like.

On reading comments as evidence: HN handles are pseudonymous and the site publishes no per-comment scores, so the ordering here is HN’s own ranking, not a vote. This is a slice of a 156-comment thread, not a consensus.