Can gzip be a language model? — Nathan Barry
Nathan Barry starts from a line in the paper Language Modeling is Compression: every prediction model is inherently a compressor, and every compression algorithm is a prediction model. He then takes the least likely candidate he can find — gzip, the archiver that ships with your operating system — and asks whether it can write Shakespeare. No weights, no training, no neural network. The 122-comment thread on Hacker News supplies the prior art he left out, one sharp correction, and the question the post never answers. ...