Merve Noyan, who works on open source at Hugging Face, posted a benchmark on X on 18 November (UTC) in which Liquid AI’s d1 decision model answered a one-field question in 83 ms, against 298 ms for Liquid’s LFM3 model writing the answer as JSON. Noyan says the gap comes from two steps the language model does and the decision model skips: prefill and decode.
decision models are faster than their LLM equivalents in multipliers
— merve (@mervenoyann) November 18, 2026
because for LLM you do prefill + decode for the JSON structured output, whereas with decision models it only does a forward pass
benchmarked LFM3 against d1 @liquidai (excluding warmup) on Llama pic.twitter.com/4PBou8e0Y5
The post gives two numbers and a one-line reason, and it assumes you know the terms. This piece defines them in order: what a language model does with a prompt, why its two phases behave differently, what a decision model does instead, and what Noyan’s chart covers.
What a language model does with your prompt
A language model reads text as tokens, small pieces of words, and writes its answer the same way. It’s autoregressive, which Benjamin Merkel of TNG Technology Consulting defines in a Hugging Face post as every new token depending on all the tokens before it, the prompt plus whatever the model has already written. Writing an answer is therefore a chain of single steps.
Merkel splits the work into two phases. Prefill processes every token of the prompt in parallel and produces the first output token. Decode produces each later token, one at a time. While it reads the prompt, the model works out a set of key and value vectors for every token and keeps them in the KV cache, a store of notes on everything read so far. Each decode step then adds one new set of notes instead of redoing the old ones.
The two phases match the two speed numbers people quote. Time to first token is the prefill wait, and time per output token is the length of one decode step. TNG says interactive apps usually aim for a first token within three seconds and 100 to 300 ms for each token after that.

Why the two phases hit different limits
Prefill is compute-bound, meaning the chip’s arithmetic is the bottleneck, because all the prompt tokens are worked on at once. Decode is memory-bound. Each step does little arithmetic, and TNG says its speed is typically limited by how fast the GPU can load model weights and the KV cache from its memory.
That split has practical effects. TNG says it’s why commercial APIs charge input tokens at a much lower rate than output tokens. Serving engines batch many requests together to keep the GPU busy during decode, and TNG found that chunked prefill, which slices a long prompt into pieces so running decodes aren’t stalled, raised total token throughput by 50% in a standard vLLM set-up with evenly sized requests. Some companies now put the phases on different hardware: Together AI runs prefill and decode on separate clusters, and Nvidia and Rebellions plan GPUs for prefill and NPUs for decode.
What a decision model skips
A decision model is built for answers that come from a known list. Liquid’s documentation says it takes a situation and a set of questions and returns calibrated probabilities across a fixed set of outcomes “in a single call with zero generated tokens”. A language model asked for the same answer in JSON generates it token by token. After the prefill, the opening brace, the field name, the value and the closing brace each take a decode step.
Cloudflare describes the mechanism in its post on Clef, its own decision model. Clef “uses Qwen for a prefill-only pass, then scores the valid schema choices in parallel”, and Cloudflare says the decision step is non-autoregressive, so no text is generated token by token. That’s what Noyan means when she says a decision model “only does a forward pass”, a single run of the input through the model’s layers. Prefill is also a forward pass, so the difference is that a decision model stops there.
Fewer passes doesn’t mean proportionally less time. TNG says a prefill takes much longer than a single decode step, so skipping decode saves less than the count of skipped tokens suggests.

What Noyan’s chart covers
The chart in her post has two rows, both with short input. With one field to fill, d1 took 83 ms and LFM3 took 298 ms, so d1 was 3.6 times faster. With three fields, d1 took 201 ms and LFM3 took 546 ms, 2.7 times faster. Noyan left out warmup, the slow first requests while a model loads, and says the runs were “on Llama”. The post doesn’t give the hardware or the model sizes.

The gap narrows as questions are added. d1’s time rose 2.4 times between one field and three, while LFM3’s rose 1.8 times. Noyan adds in a reply that the d1 implementation she tested “does prefill multiple times per question, unlike clef”, so it will get faster on input with several options.
btw I caught that their implementation has a small quirk and it will get faster for multiple options (it does prefill multiple times per question, unlike clef)
— merve (@mervenoyann) November 18, 2026
so it will get even faster for the latter 🤝
Liquid’s LFM3 release post says d1’s questions are “read in one pass over the same input”, and the Clef model card says Clef returns a probability for every option of every question in a single forward pass. Liquid’s d1-2B model card quotes 5.7 ms per decision on an NVIDIA H100 for its own set-up, a figure Noyan’s chart doesn’t compare against.
When you still need a language model
Liquid’s decision model guide says to move a call to a decision model when the answer is one of a known set of options, such as classification, routing, yes/no checks and scoring. It says to keep the language model when the job is writing something new, such as drafts, summaries, code, chat and multi-step reasoning. Liquid also says d1-2B isn’t a chat model and that its answers get less reliable past about 64K tokens of input.
The terms in one place
| Term | In plain words |
|---|---|
| Token | A small piece of a word. Models read and write text in tokens. |
| Autoregressive | Each new token depends on every token before it. |
| Prefill | The first phase: the model reads the whole prompt at once and produces the first token. |
| Decode | The second phase: the model writes the rest of the answer one token at a time. |
| KV cache | Saved notes (key and value vectors) on every token so far, so each decode step adds one set instead of redoing them all. |
| Time to first token | The wait before the answer starts, set by prefill. |
| Time per output token | How long each later token takes, which is one decode step. |
| Compute-bound and memory-bound | Limited by the chip’s arithmetic (prefill) or by how fast it can read memory (decode). |
| Forward pass | One run of the input through the model’s layers. |
| Decision model | A model that returns a probability for each option in a fixed list instead of writing text. |
| Warmup | The slow first requests while a model loads, which Noyan’s numbers leave out. |
Noyan has a longer treatment of the same ground. Her local AI slide deck, posted on 6 October, covers prefill versus decode, mixture-of-experts versus dense models, quantisation and speculative decoding with llama.cpp, and she says it’s free to reuse with attribution.
Discover more from Sulat AI & Tech News
Subscribe to get the latest posts sent to your email.




