Cerebras says it has put split inference into production: AMD Instinct GPUs process the prompt, and Cerebras CS-3 wafer systems generate the response. Its blog post, dated 17 November, reports 5x the throughput per megawatt of a CS-3-only setup, measured on a shared gpt-oss-120b endpoint on 14 November, with no loss in generation speed.
Andrew Feldman, Cerebras’s co-founder and CEO, announced it on X on 21 November and said that, as far as Cerebras is aware, it is the first production-scale deployment combining GPU and non-GPU systems for inference. Nvidia, SambaNova, d-Matrix and Etched have each described the same division of labour this year, and Nvidia’s started with a licence deal for Groq’s chip technology in December 2025.
- What Cerebras launched
- Why inference splits in two
- A timeline of the split
- Who is paired with whom
- The money behind it
- How the headline numbers compare
- What none of it shows
- What happens next
- Sources
What Cerebras launched
Cerebras says early-access partners Amazon Web Services, Fireworks AI, Gimlet Labs and General Compute already have the setup for their own deployments. It plans to roll split inference out across hundreds of megawatts, in its own cloud and in partner sites, by the end of 2027.
The software has four parts, according to the blog post: a framework that routes requests between prefill and decode pools, a prefill engine built on open-source engines such as vLLM or SGLang, a Cerebras decode engine, and KV Portal, which moves the prompt’s saved state between the two over RDMA, a fast network transfer. Partners who want more control can take a decode-only SDK and bring their own framework and prefill engine.
What “tokens per megawatt” counts
A data centre buys power in megawatts, so the number of tokens a site can produce for each megawatt it draws sets how much it can sell. Cerebras’s 5x is measured on one shared endpoint and normalised to a CS-3-only baseline of 1x.
The idea is simple: use the best chip for each stage of inference.
Andrew Feldman, Cerebras CEO, on X, 21 November 2026
AMD and Cerebras had announced the pairing on 23 July at Advancing AI 2026. AMD’s Helios rack would handle prompts and large context windows, and the Cerebras wafer would handle token generation, with the companies expecting up to 5x more tokens per second per watt, modelled with a Kimi 2.6 1T model against a wafer-only setup.
AWS is the other prefill partner. On 13 March Cerebras said AWS would deploy Cerebras CS-3 systems in its data centres, offered through Bedrock, and that the two were building a version where Trainium computes the KV cache and sends it to the Cerebras wafer over Amazon’s EFA network, for 5x more high-speed token capacity in the same footprint.
Why inference splits in two
A request runs in two stages. In prefill, the model reads the whole prompt and builds its working notes, the KV cache. In decode, it writes the answer one token at a time, and each token means reading the model from memory again. Cerebras says prefill is compute-bound and needs little memory bandwidth, while decode is bandwidth-bound.
That favours chips that keep the model in fast on-chip memory called SRAM. Cerebras says one of its wafers holds tens of gigabytes of SRAM, with 350x the SRAM capacity and 5,400x the memory bandwidth of a typical GB300 GPU. Those are its figures. Nvidia’s Groq 3 LPU, the chip it licensed from Groq, carries 500 MB of SRAM and 150 TB/s of SRAM bandwidth per chip, and a 256-chip LPX rack holds 128 GB.
The trade-off is capacity, and Nvidia’s answer is to split the work. Its technical blog describes Dynamo software that sends prompts and the attention work to Rubin GPUs and the feed-forward layers to the LPUs.

- Prefill
- The first stage of a request, when the model reads the whole prompt and builds its working notes.
- Decode
- The second stage, when the model writes the response one token at a time.
- KV cache
- The saved key-value state from prefill that decode reads at every step.
- SRAM
- Fast memory built into the processor. It holds little and reads very quickly.
- HBM
- High-bandwidth memory stacked beside a GPU. It holds far more than SRAM and reads more slowly.
- Token
- A chunk of text, usually a word or part of one, that a model reads or writes.
A timeline of the split
Dates come from company releases, filings and posts unless an outlet is named.
| Date | What happened | Source |
|---|---|---|
| 24 Dec 2025 | Groq announces a non-exclusive licensing deal with Nvidia. Groq’s founder and CEO Jonathan Ross, president Sunny Madra and other staff join Nvidia.1 | Groq |
| 14 Jan 2026 | OpenAI and Cerebras announce 750 MW of low-latency compute, coming online in tranches through 2028. | OpenAI |
| 13 Mar | AWS says it will deploy Cerebras CS-3 systems through Bedrock. The two plan to pair Trainium for prompts with the Cerebras wafer for answers. | Cerebras |
| 16 Mar | At GTC, Nvidia unveils the Groq 3 LPX rack beside Vera Rubin and claims up to 35x throughput per megawatt, with availability in the second half. | Nvidia |
| 13 May | Cerebras prices its IPO at US$185 a share and raises US$5.55 billion. Trading starts on Nasdaq as CBRS on 14 May. | Cerebras, Reuters |
| 9 Jun | d-Matrix says its Corsair accelerator is in full production and will ship in volume to hyperscalers, neoclouds and frontier labs. | d-Matrix |
| 23 Jul | AMD and Cerebras announce a Helios plus wafer-scale pairing for split inference. | AMD |
| 6 Aug | AMD agrees to buy Taalas, a Toronto startup that hardwires a model into silicon. Terms aren’t disclosed. | AMD |
| 18 Aug | Etched raises US$700 million at a US$21 billion valuation, led by Jane Street, and says it shipped its first rack to Jane Street in July. | Etched |
| 24 Aug | Nvidia says Groq 3 LPX is in full production. Nebius is the first AI cloud to adopt it. | Nvidia |
| 10 Sep | Positron raises US$875 million at a US$5 billion valuation. | Positron |
| 12 Nov | Cerebras reports record third-quarter revenue of US$210.8 million. | Cerebras |
| 15 Nov | SambaNova and Intel demonstrate an air-cooled rack at SC26: four Intel D-Series GPUs for prompts and a SambaRack SN50 for answers. | Unite.AI |
| 17 Nov | Cerebras publishes its split-inference blog post. AMD completes its Taalas acquisition. | Cerebras, AMD |
| 21 Nov | Feldman posts the news on X. | X |


Who is paired with whom

| Company | Decode chip | Prefill partner | Status | Source |
|---|---|---|---|---|
| Nvidia | Groq 3 LPX, 256 LPUs a rack | Vera Rubin NVL72 GPUs | In full production from 24 August; Nebius first to adopt | Nvidia |
| Cerebras | Wafer-scale engine in CS-3 and CS-4 systems | AMD Instinct GPUs; AWS Trainium | Split inference in production on its cloud with AMD; AWS in early access | Cerebras |
| SambaNova | SN50 RDU, 16 a rack | Intel D-Series GPUs | Demonstrated at SC26 on 15 November | Unite.AI |
| d-Matrix | Corsair | GPUs, unnamed | Corsair in full production since 9 June | d-Matrix |
| AMD | Taalas hardwired chips, to be added to its roadmap | Instinct GPUs in Helios racks | Taalas deal closed 17 November | AMD |
| Etched | Its own chips | Its own chips; Etched says its racks handle prefill and decode | First rack delivered to Jane Street | Etched |
NVIDIA today announced that NVIDIA Groq 3 LPX, the interactive AI inference accelerator, is now in full production.
Nvidia news release, 24 August 2026
Hardwired and air-cooled alternatives
Taalas builds the other extreme. Its chips cast one model’s weights and dataflow into silicon, so a part runs the model it was built for and nothing else. Its HC1 test chip served Meta’s Llama 3.1 8B at close to 17,000 tokens a second, a figure SiliconANGLE reports.
SambaNova’s vice president of core product, Anton McGonnell, told SiliconANGLE’s theCUBE that its racks stay air-cooled, that newer GPU racks can demand 120 to 250 kilowatts each, and that power and cooling equipment has 18 to 24 months of lead time. SambaNova’s demo rack needs about 20 kW, per its own figures. McGonnell’s comments are on SiliconANGLE.
The hyperscalers sit alongside
Google, Amazon and Microsoft build their own chips and appear here mainly as customers and hosts. Anthropic said on 6 April that it signed a deal with Google and Broadcom for multiple gigawatts of next-generation TPU capacity from 2027, and that it trains and runs Claude on Trainium, TPUs and Nvidia GPUs.
The money behind it
US$26.1 billion
Cerebras’s remaining performance obligations at 30 September
US$210.8 million
Cerebras’s GAAP revenue in the third quarter, up 55% on a year earlier
US$5.55 billion
raised in Cerebras’s May IPO
US$21 billion
Etched’s valuation after its August round
Cerebras’s third-quarter release put GAAP cloud revenue at US$141.9 million, up 265%, and GAAP gross margin at 20%. It guided core revenue of US$310 million to US$330 million for the fourth quarter and US$940 million to US$960 million for 2026, and Feldman said Cerebras expects to triple revenue in 2027. The release lists dependence on a limited number of significant customers among its risks.
The same release says Cerebras has 700 MW of data-centre capacity under contract for delivery by the end of 2027, that Oracle will offer Cerebras in its cloud in 2027, and that it signed a deal with an unnamed top-three hyperscaler. The CS-5, built on the WSE-5 chip that Cerebras says is back from the fab, is targeted for early summer 2027 at up to 10,000 tokens a second per user.
Etched says it has more than US$1 billion in customer contracts and has named one customer, Jane Street. Positron says its Atlas racks are deployed at Oracle Cloud Infrastructure, 50-plus racks. SambaNova was valued at US$11 billion in July, SiliconANGLE reports.
How the headline numbers compare
Cerebras’s 5x is against its own CS-3-only setup on one model. Nvidia’s 35x is a projection against its previous rack, for models of a trillion parameters or more. AMD’s 5x is a July model against a wafer-only setup, using a different model again. The table lists who measured what.
| Claim | Who measured | Baseline | Model and setting | Source |
|---|---|---|---|---|
| 5x tokens per MW | Cerebras, measured | Its CS-3-only setup | gpt-oss-120b, shared endpoint, 14 Nov | Cerebras |
| Up to 35x tokens per MW | Nvidia, projected | GB200 NVL72 | Trillion-parameter models | Nvidia |
| Up to 5x tokens per second per watt | AMD and Cerebras, modelled | Wafer-only setup | Kimi 2.6 1T, July | AMD |
| Up to 2.5x throughput | Nvidia, MLPerf v6.1 preview | GB300 NVL72 | DeepSeek-R1 | Nvidia |
| 21,333 tokens a second per GPU | AMD, its own MLPerf Offline setup | Published Vera Rubin NVL72 results | DeepSeek-R1 | AMD |
| 3,400 tokens a second, one user | Artificial Analysis, per Nvidia | None given | Gemma 4 31B, 100K context | Nvidia |

What none of it shows
- Cerebras’s blog post doesn’t give prompt lengths, load or a price per token for the 5x result, and its footer says speed comparisons against GPU systems “may vary depending on workload, configuration, date and models being tested”.2
- Nvidia’s 24 August release says Nebius “plans to bring” LPX to its Token Factory. The release doesn’t name a customer running on one.
- Etched has named one customer. Nvidia hasn’t disclosed what it paid Groq, and AMD hasn’t disclosed what it paid for Taalas.
- No independent benchmark compares a Cerebras split setup with a Nvidia LPX rack. The Artificial Analysis test of LPX used one model, and Nvidia published it.
- Feldman’s claim to be first applies to Cerebras’s own phrasing, “production-scale deployment combining GPU and non-GPU systems”. Nvidia announced LPX as in full production on 24 August.
What happens next
- Cerebras speaks at a Barclays conference on 9 December. It targets the CS-5 for early summer 2027 and split inference across hundreds of megawatts by the end of 2027.
- Nebius plans to add LPX to Token Factory, and Groq says it will deploy LPX itself with Dell.
- AMD closed the Taalas deal on 17 November and hasn’t said when a product will follow.
- Anthropic’s new Google and Broadcom TPU capacity starts coming online in 2027.
Sources
- Cerebras launches disaggregated inference, Cerebras blog, 17 November 2026
- Cerebras third-quarter 2026 results, 12 November 2026
- Andrew Feldman on X, 21 November 2026
- AMD and Cerebras announcement, 23 July 2026
- Cerebras is coming to AWS, 13 March 2026
- Nvidia Vera Rubin platform, 16 March 2026, and Groq 3 LPX in full production, 24 August 2026
- Inside NVIDIA Groq 3 LPX, Nvidia technical blog, 16 March 2026
- Groq and Nvidia licensing agreement, 24 December 2025
- AMD completes Taalas acquisition, 17 November 2026
- Etched funding and first delivery, 18 August 2026
- Unite.AI on SambaNova and Intel, 16 November 2026
- CNBC reported a price of about US$20 billion on 24 December 2025. Groq’s announcement disclosed no price. ↩︎
- The Cerebras blog post carries this note in its footer. ↩︎
Discover more from Sulat AI & Tech News
Subscribe to get the latest posts sent to your email.






