Can gzip be a language model?

This explores treating a standard DEFLATE compressor (gzip/zlib) as a simple language model by using compression as a proxy for prediction. Because compressors allocate fewer bits to expected byte sequences, continuations that resemble a priming corpus compress smaller; the score for a candidate continuation is the compressed length of (corpus + prompt + candidate). Priming gzip with tiny Shakespeare and searching for continuations yields recognizably Shakespearean fragments, showing the compressor captures distributional structure despite having no learned parameters beyond its sliding 32 KiB window and back-reference mechanism.

Generating text requires more than greedy next-byte selection because gzip reports integer compressed lengths and short increments often tie, producing poor local decisions and self-copying loops. The implementation runs beam search over byte sequences: at each step the context is the corpus window plus the recent tail of prompt+generated text; it expands bytes seen in the corpus, scores each by compressed length, keeps the top beam_width candidates, repeats for a horizon, then commits the best span (or samples with temperature). Keeping only the recent tail in context avoids trivial repeats. The system is a single Python file using zlib, and beam search substantially improves generation quality compared to naive approaches.