Nathan Barry starts from a line in the paper Language Modeling is Compression: every prediction model is inherently a compressor, and every compression algorithm is a prediction model. He then takes the least likely candidate he can find — gzip, the archiver that ships with your operating system — and asks whether it can write Shakespeare. No weights, no training, no neural network. The 122-comment thread on Hacker News supplies the prior art he left out, one sharp correction, and the question the post never answers.

The mechanism, in plain terms

  • Compression is a form of prediction. A compressor spends few bytes on text it expects and many on text it doesn’t, so the number of bytes needed to encode something is essentially a probability in disguise. DEFLATE (what gzip uses) compresses by finding repeated stretches in the last 32 kilobytes of input and pointing back at them instead of re-storing them.
  • That gives a scoring rule: concatenate a piece of context with a candidate continuation, compress the whole thing, and measure the length. The shorter the result, the more the candidate looks like something the model “expected.”
  • To give gzip any knowledge to work with, Barry primes it with a corpus — in his run, tiny Shakespeare — so anything that resembles the corpus compresses small.
  • Generation is a separate problem. Picking the single best next byte fails badly, because gzip reports whole bytes: many candidates tie, and the real signal drowns in rounding. The fix is beam search — evaluate whole spans of several bytes ahead, keep the best few partial continuations, extend and prune, then commit the winning span and repeat.
  • One deliberate restriction: only the most recent stretch of generated text stays in the scoring window. Without it, the cheapest thing gzip can do is repeat text it just emitted, and the output degenerates into runs of one letter. With it, the unedited output is nonsense but recognizably Shakespeare-shaped — character names in caps, “Pray now,” fragments of iambic register.
  • The whole thing is one file of standard-library Python, using zlib rather than shelling out to gzip. Barry’s footnote admits the paper that inspired him tried the same idea and performed poorly; his addition was the beam search.

What the thread adds

  • jll29 — the practical technique hiding in the premise: classify a document by compressing it against a corpus for each topic and picking the smallest result, as in gzip -9 sports.txt testfile.txt. They credit Witten’s group at Waikato as early work and point at the Hutter Prize as the place where compression and prediction are scored against each other.
  • stingraycharles — independent corroboration with a decade-plus of hindsight: they built language detection this way around twenty years ago by seeding each compressor with Wikipedia articles in a different language. Their own verdict: “Absolutely totally not the best approach, but very fast and super simple to implement.”
  • StilesCrisis — the correction. They argue the interesting-looking output is an artifact of the tail-window restriction, not evidence that gzip is modeling language: “Basically I think the entire premise falls apart due to that choice–they forced an interesting-looking outcome by adjusting the algorithm until gzip started picking random slabs of letters instead of ever-larger repeating runs.”
  • mg — the question that undercuts any strong claim: how would you know the search was any good, when the space of possible byte sequences is vastly larger than what beam search explores? On that reading the result is only a lower bound on gzip’s ability to judge a continuation. shoo replies with an experiment: if any span of the wanted length already appears in the context, DEFLATE can encode it as a back-reference, so the best-compressing continuation is often just a copy of prior text — great for shrinking a file, useless as a generative model.
  • montebicyclelo and Matumio (two commenters, same point) — don’t over-read the equivalence. Matumio puts the useful version: both methods “try to solve the same mathematical problem. It’s better than thinking of LLMs as magic,” while the wrong version is assuming gzip could ever “reach similar complexity or generalization.”
  • tromp — flips the question: how well does an LLM compress, compared to gzip? gkbrk’s answer: the top entrant in the Hutter Prize uses a neural network.
  • networked (the submitter) — reran the experiment with bzip2 and zstd after asking MiMo-V2.6-Flash to fork the code, reporting that bzip2 emits runs of alternating symbols and zstd mostly whitespace. That drew two of the sharper replies in the thread: maxidog asking whether they verified the model performed the unfamiliar task correctly, and jeremyjh asking whether they had AI write code they didn’t understand and then posted the output for other humans to read. The author’s own footnote over the same gap: the tool is named after gzip but calls zlib — def’s complaint that the post is “generated article text and then not even actually using gzip.”

The question the thread kept asking

The post is framed as “can gzip be a language model?” and delivers a demonstration, not a measurement. Nobody in the thread could point to a number that says how good a model it is — no bits-per-character against a neural baseline, no comparison of beam widths or search horizons. mg asked it directly and got an experiment about back-references instead of an answer. berkes took the same gap in the other direction, asking what it would even mean for an LLM to be a compressor; eru and evgpbfhnr answered with arithmetic coding and Fabrice Bellard’s ts_zip, neither of which the post mentions.

Thread caveat: HN handles are pseudonymous and the site publishes no per-comment scores — the ordering above is HN’s own ranking, not a vote. This is a slice of 122 comments, not a consensus, and shoo’s back-reference result is quoted as their experiment, not as a settled finding.