Sulat.com / Writing on AIFriday, November 20, 2026
Artificial Intelligence/20 November 2026

How tufazip squeezes 1 GB of Wikipedia into 105.39 MB: a deep dive into Tufa Labs’ compressor

Tufa Labs' tufazip squeezes 1 GB of Wikipedia into 105,388,353 bytes with a single transformer as its only predictor. Each run takes about 7.5 days on two B200 GPUs, and the compressor is tuned to that one file.

Glossy 3D Tufa Labs flea mascot perched on a glowing amber cube as a glass press squeezes a tall stack of printed pages, with GPU boards behind

At a glance

  • Tufa Labs says tufazip compressed the billion-byte enwik9 Wikipedia file to 105,388,353 bytes, decoder and weights included, and calls it the best benchmark entry that uses a neural network as its only predictor.
  • tufazip uses a 163-million-parameter transformer that learns while it compresses; the decoder repeats the same learning, so the weights learned during the run never need to be stored.
  • Tommy He found that a bigger replay window and a small 4K-word vocabulary helped, while markup tokens and larger models didn't, and a two-block 4.33 MB prior beat the online-only run by 0.59 MB.
  • Each run takes about 7.5 days on two NVIDIA B200 GPUs, and Mahoney's benchmark page describes the compressor as tuned to enwik9.
  • The benchmark table I read on 20 November lists 12 entries above tufazip, each between 88.98 MB and 97.62 MB; Tufa says they all mix a transformer with hand-built models.

Tufa Labs, a small independent AI research group in Zurich, says it has shrunk 1 GB of Wikipedia to 105.39 MB without losing a byte. Its compressor, tufazip, is the best entry on the Large Text Compression Benchmark that uses a neural network as its only predictor, according to the group’s announcement on X. Running it takes about 7.5 days on two NVIDIA B200 GPUs in each direction.

Tufa Labs posted the announcement on 19 November and credited the build to Tommy He. The code is on GitHub under the MIT licence. The full write-up on tufalabs.ai is dated 14 October, and Matt Mahoney’s benchmark page lists the first version from 7 September and the improved v2 from 30 September. This is the longer look: what the number measures, how the compressor works, what Tommy He tried that didn’t pay off, and what the result doesn’t show.

In plain words

A compressor is a guessing game. If a program can predict the next piece of text, it needs only a few bits to record each piece it guessed well and more bits for the surprises. tufazip hands the guessing to a transformer, the kind of neural network behind chatbots, and a second tool called an arithmetic coder turns those guesses into bits. The transformer starts with only a small trained head start and learns from the text as it goes. The decompressor repeats the same learning in the same order, ends up with the same guesses and rebuilds the file exactly.

That turns 1,000,000,000 bytes into 105,388,353, about 9.5 times smaller, with the decoder and the trained starting weights counted in. The price is time and hardware. Each direction takes about a week, and the code’s requirements list exactly two B200 GPUs.

Six stat tiles: 1,000,000,000 bytes in, 105,388,353 out, 9.5x smaller, 163 million parameters, about 7.5 days each way, two B200 GPUs
A billion bytes in, 105 million out, and a week of two-GPU compute each way.

What the benchmark measures

The Large Text Compression Benchmark is a test Mahoney has run for more than 20 years. Entries compress enwik9, a billion-byte file of Wikipedia text. Under the benchmark’s rules, the score is the compressed file plus the zipped decoder, including anything the decoder needs to rebuild the text. That rule matters later, because it makes pre-trained weights count against you.

Program Total size (bytes) What it is
gzip 1.3.5 -9 322,630,796 Everyday compressor
xz 5.2.1, preset 9e 197,368,568 Everyday compressor
NNCP v3.2 107,261,318 Fabrice Bellard’s transformer compressor
DNA-4 105,554,360 RWKV-7 language model trained on enwik9
tufazip v2 105,388,353 Tommy He, Tufa Labs
ex-girlfriend 2.0 88,983,124 Top listed entry, derived from cmix-neif
Total size on enwik9 from Mahoney’s benchmark page as I read it on 20 November. Smaller is better.

Tufa Labs’ write-up says tufazip took second place when it was submitted. The table I read on 20 November lists 12 entries above it, between 88.98 MB and 97.62 MB. Tufa’s X post says everything above tufazip mixes a transformer with hand-built models. The benchmark’s own notes agree for the two I checked. ex-girlfriend 2.0 is derived from cmix-neif, which Tufa’s write-up describes as combining a transformer, an online LSTM and statistical models. fractal-zip is derived from cmix-lex-transformer. The next entry below tufazip, DNA-4, is a compressor built on the RWKV-7 language model, and it’s 166,007 bytes bigger.

Bar chart of enwik9 total size: gzip 322.6 MB, xz 197.4 MB, NNCP v3.2 107.3 MB, tufazip v2 105.4 MB, top listed entry 89.0 MB
tufazip sits far below gzip and xz, 1.9 MB under NNCP and 16.4 MB behind the top entry.

How guesses become bits

Tufa’s write-up starts from one fact: predicting text and compressing it are the same job. A token is a chunk of text, usually a word or part of one. If the model gives the token that really comes next a probability of one in eight, writing it costs three bits. Double that probability and the cost drops by one bit. Training a language model to predict better is the same as shrinking the file.

Arithmetic coding does the writing. It starts with the range from zero to one and splits it into slices sized by the model’s probabilities. The slice for the token that actually occurs becomes the new range, and the process repeats. Likely tokens leave wide slices that take few bits to pin down, and surprises leave thin ones. The finished file is one very precise number between zero and one.

The model keeps learning while it works. In the online version, the encoder and decoder start from identical randomly initialised weights and repeat three steps for every token. Both predict the next token from the text they already know. The encoder writes the real token and the decoder recovers it. Then both train on that token. Because the decoder rebuilds the model as it rebuilds the text, the weights learned during the run never need to be stored. Tufa’s write-up adds one condition. The learning updates and the probabilities must be fully deterministic for the result to be lossless.

Three-step loop: predict the next token, encode or decode it, learn from it, repeated for every token, so learned weights are never sent
The encoder and decoder repeat the same three steps, so the learned weights never travel.

The model, and what moved the needle

The predictor is a 12-layer, decoder-only transformer with 163 million parameters, the adjustable numbers inside the network. It uses rotary position embeddings, SwiGLU feed-forward layers and RMSNorm, with a 4K-word vocabulary and a 192-token context. Tommy He says he started from Fabrice Bellard’s NNCP and improved the model and training recipe. Online learning alone got the file to 105.92 MB of compressed text, or 105.98 MB with the decoder, about 0.848 bits for every original byte.

The biggest lever he describes is replay. Every so often the model goes back over a window of text it has already decoded and trains on it again. That costs compute on both sides but needs no extra data in the file. With the same 163-million-parameter model, widening the replay window from 15 million to 35 million tokens cut the compressed text from 107.22 MB to 105.92 MB. That saved 1.30 MB, or 1.21 per cent, for about 40 per cent more estimated model operations.

The other tests produced mixed results. A smaller vocabulary helped. In a comparison with a five-million-token replay window, the 4K-word tokenizer beat the 8K and 16K ones. Dedicated XML and wiki-markup tokens made compression worse, even though the input is full of markup. Bigger models didn’t help either. On the full file, a 267-million-parameter model produced slightly larger output than a 188-million one. On the 100-million-byte enwik8, gains ran from 14 million to 77 million parameters, flattened at 163 million and reversed at 336 million. Tufa’s write-up hypothesises that a fixed document has an optimal model size.

Checklist: bigger replay window, small 4K vocabulary and a two-block prior helped; XML and wiki tokens and bigger models did not
Replay, a small vocabulary and a tiny prior helped; markup tokens and extra parameters didn’t.

The 4.33 MB head start

Online learning starts with no knowledge of English. So Tommy He also tried an offline prior: train a small model in advance, ship it inside the decoder and let it keep learning during the run. The benchmark counts those weights, so they have to save more text than they cost. Tufa tested full-precision models, low-bit models, LoRA adapters, which are small correction layers added to a frozen model, and compact layouts that reuse the same blocks. On enwik8 none of them beat the online baseline of 15.15 MB. The best compact layouts landed at 17.58 and 18.96 MB.

The full billion-byte file gives a prior ten times as much text to earn its cost back, and there the two-block layout won. Called C2, it trains two blocks and stores them once each, 4.33 MB in all. At run time they’re copied six times into 12 independent layers, which then learn on their own.

Two stored blocks A and B are copied six times into twelve layers, and each layer then learns on its own during compression
Only two trained blocks ship with the decoder; the other ten layers start as copies and learn on the job.

A six-layer prior predicted better but couldn’t pay for itself. Its compressed text was 99.17 MB, yet its decoder and weights cost 10.95 MB, so the total was 110.12 MB. C2 saved 4.97 MB of compressed text and added 4.38 MB of decoder, a net gain of 0.59 MB, or 0.56 per cent, over the online run. That gain is the gap between 105.98 and 105.39 MB.

Setup Compressed text (MB) Decoder and weights (MB) Total (MB)
Online only 105.924 0.055 105.979
Six-layer prior (L6) 99.173 10.945 110.118
Two-block prior (C2) 100.958 4.430 105.388
Full enwik9 runs from Tufa Labs’ write-up. The two-block prior is the one tufazip v2 ships.
Stacked bars: online only 105.98 MB, six-layer prior 110.12 MB, two-block prior 105.39 MB, split into compressed text and decoder weights
The six-layer prior predicts better but costs more in weights than it saves in text.

What it takes to run

The tufazip README lists Linux on x86-64, exactly two visible NVIDIA B200 GPUs, GCC, CUDA 13.0, cuDNN 9.20, Python 3.12.3 and the uv package manager as requirements. It publishes SHA-256 checksums for the 100,958,324-byte compressed file, the weights file and the decoded enwik9, so anyone with the hardware can check the round trip. The README gives about 631,548 seconds to compress and 645,748 seconds to decompress. That’s roughly 7.3 and 7.5 days. Mahoney’s page says the compression time doesn’t include the offline training of the prior.

What the number doesn’t show

Mahoney’s page describes tufazip as tuned to enwik9. Tufa’s own write-up calls compression on one fixed file a limited measure of intelligence and raises Goodhart’s law, the idea that a measure stops being useful once you optimise it hard enough. Tommy He writes that a smaller complete artifact is evidence of a better representation of the same corpus, but that it doesn’t establish greater general intelligence.

The write-up also points out that the score can’t tell where the knowledge came from. cmix v21 combines 2,077 predictors, and the benchmark counts their savings just as it counts a transformer’s. Tufa’s interest in a single neural predictor is in how much structure one general learning procedure can recover. Its write-up adds that it would want automated research loops to explain which findings carry over, not just shave bytes off the payload.

The code is at github.com/Tufalabs/tufazip, and the benchmark table changes as entries arrive. The top entry, ex-girlfriend 2.0, went in on 18 November at 88,983,124 bytes.

Glossary

Term Plain meaning
Lossless The decompressed file matches the original byte for byte.
Token A chunk of text, usually a word or part of one, that the model predicts one at a time.
Transformer The neural network design behind modern chatbots. It predicts the next token from the tokens before it.
Parameters The adjustable numbers inside a neural network. tufazip’s has 163 million.
Arithmetic coder The tool that turns a model’s probabilities into the bits of the compressed file.
Replay Training again on text the model has already seen. Here it’s text already decoded.
Offline prior A model trained in advance and shipped with the decoder as a head start.
Quantised Stored with fewer bits per number, so the weights take less space.
enwik9, enwik8 The billion-byte Wikipedia test file and its first 100 million bytes.
Payload The compressed text on its own, without the decoder.
Deterministic Gives exactly the same result every time from the same input. The decoder depends on it.
Ablation A test that changes one part of a system to see what it contributed.
Terms as used in Tufa Labs’ write-up.


Discover more from Sulat AI & Tech News

Subscribe to get the latest posts sent to your email.