When an LLM solves a hard math problem, the reflex is to say it’s getting smarter. Davide Piffer offers a simpler, more interesting explanation: the context window is an enormous external notebook, and that matters more for formal reasoning than superior reasoning itself.
- Human working memory is severely limited — try multiplying two three-digit numbers in your head; the difficulty is carrying partial results, not the operations themselves
- Studies show working memory predicts mathematical performance even after controlling for IQ (Alloway & Alloway 2010; Alloway & Passolunghi 2011)
- A context window is “augmented symbolic working memory” — weaker at private internal states than humans, vastly stronger at maintaining an explicit persistent record
- Mathematics is especially suited to this architecture because its symbols are precise, assumptions can be externalized, and every step can be checked
- AI’s advantage is largest on problems involving many interacting constraints, long calculations, case analysis, and exact bookkeeping
- The gap narrows on problems requiring a single short conceptual leap or an entirely new representation
Piffer’s framing matters because it shifts the question from “how smart is AI?” to “what cognitive architecture does it have?” — and that distinction has real implications. Reducing a model’s usable context or preventing it from writing intermediate steps should disproportionately hurt its performance on long math tasks. Expanding a human’s external memory (clear notation, software, structured notes) should narrow the gap. The fairest comparison isn’t AI vs. unaided human — it’s AI with its tools vs. a human with equally powerful external memory and verification systems. The closing analogy — von Neumann (speed, breadth, symbolic memory) vs. Einstein (reconceptualizing the problem) — is the cleanest distillation of where we actually are.