Tufa Labs, a small independent AI research group in Zurich, says it has shrunk 1 GB of Wikipedia to 105.39 MB without losing a byte. Its compressor, tufazip, is the best entry on the Large Text Compression Benchmark that uses a neural network as its only predictor, according to the group’s announcement on X. Running it takes about 7.5 days on two NVIDIA B200 GPUs in each direction.
Tufa Labs posted the announcement on 19 November and credited the build to Tommy He. The code is on GitHub under the MIT licence. The full write-up on tufalabs.ai is dated 14 October, and Matt Mahoney’s benchmark page lists the first version from 7 September and the improved v2 from 30 September. This is the longer look: what the number measures, how the compressor works, what Tommy He tried that didn’t pay off, and what the result doesn’t show.
In plain words
A compressor is a guessing game. If a program can predict the next piece of text, it needs only a few bits to record each piece it guessed well and more bits for the surprises. tufazip hands the guessing to a transformer, the kind of neural network behind chatbots, and a second tool called an arithmetic coder turns those guesses into bits. The transformer starts with only a small trained head start and learns from the text as it goes. The decompressor repeats the same learning in the same order, ends up with the same guesses and rebuilds the file exactly.
That turns 1,000,000,000 bytes into 105,388,353, about 9.5 times smaller, with the decoder and the trained starting weights counted in. The price is time and hardware. Each direction takes about a week, and the code’s requirements list exactly two B200 GPUs.

What the benchmark measures
The Large Text Compression Benchmark is a test Mahoney has run for more than 20 years. Entries compress enwik9, a billion-byte file of Wikipedia text. Under the benchmark’s rules, the score is the compressed file plus the zipped decoder, including anything the decoder needs to rebuild the text. That rule matters later, because it makes pre-trained weights count against you.
| Program | Total size (bytes) | What it is |
|---|---|---|
| gzip 1.3.5 -9 | 322,630,796 | Everyday compressor |
| xz 5.2.1, preset 9e | 197,368,568 | Everyday compressor |
| NNCP v3.2 | 107,261,318 | Fabrice Bellard’s transformer compressor |
| DNA-4 | 105,554,360 | RWKV-7 language model trained on enwik9 |
| tufazip v2 | 105,388,353 | Tommy He, Tufa Labs |
| ex-girlfriend 2.0 | 88,983,124 | Top listed entry, derived from cmix-neif |
Tufa Labs’ write-up says tufazip took second place when it was submitted. The table I read on 20 November lists 12 entries above it, between 88.98 MB and 97.62 MB. Tufa’s X post says everything above tufazip mixes a transformer with hand-built models. The benchmark’s own notes agree for the two I checked. ex-girlfriend 2.0 is derived from cmix-neif, which Tufa’s write-up describes as combining a transformer, an online LSTM and statistical models. fractal-zip is derived from cmix-lex-transformer. The next entry below tufazip, DNA-4, is a compressor built on the RWKV-7 language model, and it’s 166,007 bytes bigger.

How guesses become bits
Tufa’s write-up starts from one fact: predicting text and compressing it are the same job. A token is a chunk of text, usually a word or part of one. If the model gives the token that really comes next a probability of one in eight, writing it costs three bits. Double that probability and the cost drops by one bit. Training a language model to predict better is the same as shrinking the file.
Arithmetic coding does the writing. It starts with the range from zero to one and splits it into slices sized by the model’s probabilities. The slice for the token that actually occurs becomes the new range, and the process repeats. Likely tokens leave wide slices that take few bits to pin down, and surprises leave thin ones. The finished file is one very precise number between zero and one.
The model keeps learning while it works. In the online version, the encoder and decoder start from identical randomly initialised weights and repeat three steps for every token. Both predict the next token from the text they already know. The encoder writes the real token and the decoder recovers it. Then both train on that token. Because the decoder rebuilds the model as it rebuilds the text, the weights learned during the run never need to be stored. Tufa’s write-up adds one condition. The learning updates and the probabilities must be fully deterministic for the result to be lossless.

The model, and what moved the needle
The predictor is a 12-layer, decoder-only transformer with 163 million parameters, the adjustable numbers inside the network. It uses rotary position embeddings, SwiGLU feed-forward layers and RMSNorm, with a 4K-word vocabulary and a 192-token context. Tommy He says he started from Fabrice Bellard’s NNCP and improved the model and training recipe. Online learning alone got the file to 105.92 MB of compressed text, or 105.98 MB with the decoder, about 0.848 bits for every original byte.
The biggest lever he describes is replay. Every so often the model goes back over a window of text it has already decoded and trains on it again. That costs compute on both sides but needs no extra data in the file. With the same 163-million-parameter model, widening the replay window from 15 million to 35 million tokens cut the compressed text from 107.22 MB to 105.92 MB. That saved 1.30 MB, or 1.21 per cent, for about 40 per cent more estimated model operations.
The other tests produced mixed results. A smaller vocabulary helped. In a comparison with a five-million-token replay window, the 4K-word tokenizer beat the 8K and 16K ones. Dedicated XML and wiki-markup tokens made compression worse, even though the input is full of markup. Bigger models didn’t help either. On the full file, a 267-million-parameter model produced slightly larger output than a 188-million one. On the 100-million-byte enwik8, gains ran from 14 million to 77 million parameters, flattened at 163 million and reversed at 336 million. Tufa’s write-up hypothesises that a fixed document has an optimal model size.

The 4.33 MB head start
Online learning starts with no knowledge of English. So Tommy He also tried an offline prior: train a small model in advance, ship it inside the decoder and let it keep learning during the run. The benchmark counts those weights, so they have to save more text than they cost. Tufa tested full-precision models, low-bit models, LoRA adapters, which are small correction layers added to a frozen model, and compact layouts that reuse the same blocks. On enwik8 none of them beat the online baseline of 15.15 MB. The best compact layouts landed at 17.58 and 18.96 MB.
The full billion-byte file gives a prior ten times as much text to earn its cost back, and there the two-block layout won. Called C2, it trains two blocks and stores them once each, 4.33 MB in all. At run time they’re copied six times into 12 independent layers, which then learn on their own.

A six-layer prior predicted better but couldn’t pay for itself. Its compressed text was 99.17 MB, yet its decoder and weights cost 10.95 MB, so the total was 110.12 MB. C2 saved 4.97 MB of compressed text and added 4.38 MB of decoder, a net gain of 0.59 MB, or 0.56 per cent, over the online run. That gain is the gap between 105.98 and 105.39 MB.
| Setup | Compressed text (MB) | Decoder and weights (MB) | Total (MB) |
|---|---|---|---|
| Online only | 105.924 | 0.055 | 105.979 |
| Six-layer prior (L6) | 99.173 | 10.945 | 110.118 |
| Two-block prior (C2) | 100.958 | 4.430 | 105.388 |

What it takes to run
The tufazip README lists Linux on x86-64, exactly two visible NVIDIA B200 GPUs, GCC, CUDA 13.0, cuDNN 9.20, Python 3.12.3 and the uv package manager as requirements. It publishes SHA-256 checksums for the 100,958,324-byte compressed file, the weights file and the decoded enwik9, so anyone with the hardware can check the round trip. The README gives about 631,548 seconds to compress and 645,748 seconds to decompress. That’s roughly 7.3 and 7.5 days. Mahoney’s page says the compression time doesn’t include the offline training of the prior.
What the number doesn’t show
Mahoney’s page describes tufazip as tuned to enwik9. Tufa’s own write-up calls compression on one fixed file a limited measure of intelligence and raises Goodhart’s law, the idea that a measure stops being useful once you optimise it hard enough. Tommy He writes that a smaller complete artifact is evidence of a better representation of the same corpus, but that it doesn’t establish greater general intelligence.
The write-up also points out that the score can’t tell where the knowledge came from. cmix v21 combines 2,077 predictors, and the benchmark counts their savings just as it counts a transformer’s. Tufa’s interest in a single neural predictor is in how much structure one general learning procedure can recover. Its write-up adds that it would want automated research loops to explain which findings carry over, not just shave bytes off the payload.
The code is at github.com/Tufalabs/tufazip, and the benchmark table changes as entries arrive. The top entry, ex-girlfriend 2.0, went in on 18 November at 88,983,124 bytes.
Glossary
| Term | Plain meaning |
|---|---|
| Lossless | The decompressed file matches the original byte for byte. |
| Token | A chunk of text, usually a word or part of one, that the model predicts one at a time. |
| Transformer | The neural network design behind modern chatbots. It predicts the next token from the tokens before it. |
| Parameters | The adjustable numbers inside a neural network. tufazip’s has 163 million. |
| Arithmetic coder | The tool that turns a model’s probabilities into the bits of the compressed file. |
| Replay | Training again on text the model has already seen. Here it’s text already decoded. |
| Offline prior | A model trained in advance and shipped with the decoder as a head start. |
| Quantised | Stored with fewer bits per number, so the weights take less space. |
| enwik9, enwik8 | The billion-byte Wikipedia test file and its first 100 million bytes. |
| Payload | The compressed text on its own, without the decoder. |
| Deterministic | Gives exactly the same result every time from the same input. The decoder depends on it. |
| Ablation | A test that changes one part of a system to see what it contributed. |
Discover more from Sulat AI & Tech News
Subscribe to get the latest posts sent to your email.




