Sulat.com / Writing on AISunday, November 22, 2026
Business & Industry/22 November 2026

Nvidia, Cerebras, SambaNova and d-Matrix all split inference between two kinds of chip

Cerebras says GPUs now read the prompt while its wafer chips write the answer, with 5x the throughput per megawatt. Nvidia, SambaNova and d-Matrix describe the same split, each against its own baseline.

Glossy 3D wafer-scale chip with the Cerebras logo beside two GPU cards carrying the NVIDIA and AMD logos, joined by streams of light

At a glance

  • Cerebras says AMD Instinct GPUs now read the prompt while its CS-3 wafer systems write the answer, and that the setup delivers 5x the throughput per megawatt of a CS-3-only configuration on one shared endpoint.
  • Nvidia licensed Groq's chip technology in December 2025 and says its Groq 3 LPX rack, which pairs with Vera Rubin GPUs, has been in full production since 24 August.
  • SambaNova pairs its SN50 with Intel GPUs, d-Matrix pairs Corsair with GPUs, AMD agreed to buy Taalas, and Etched and Positron raised US$1.6 billion between them in August and September.
  • Each performance figure uses a different baseline, model and measurer, so the claims from Cerebras, Nvidia and AMD can't be compared directly.

Cerebras says it has put split inference into production: AMD Instinct GPUs process the prompt, and Cerebras CS-3 wafer systems generate the response. Its blog post, dated 17 November, reports 5x the throughput per megawatt of a CS-3-only setup, measured on a shared gpt-oss-120b endpoint on 14 November, with no loss in generation speed.

Andrew Feldman, Cerebras’s co-founder and CEO, announced it on X on 21 November and said that, as far as Cerebras is aware, it is the first production-scale deployment combining GPU and non-GPU systems for inference. Nvidia, SambaNova, d-Matrix and Etched have each described the same division of labour this year, and Nvidia’s started with a licence deal for Groq’s chip technology in December 2025.

Andrew Feldman, Cerebras CEO, on X, 21 November 2026: Cerebras launches disaggregated inference with AMD.

What Cerebras launched

Cerebras says early-access partners Amazon Web Services, Fireworks AI, Gimlet Labs and General Compute already have the setup for their own deployments. It plans to roll split inference out across hundreds of megawatts, in its own cloud and in partner sites, by the end of 2027.

The software has four parts, according to the blog post: a framework that routes requests between prefill and decode pools, a prefill engine built on open-source engines such as vLLM or SGLang, a Cerebras decode engine, and KV Portal, which moves the prompt’s saved state between the two over RDMA, a fast network transfer. Partners who want more control can take a decode-only SDK and bring their own framework and prefill engine.

What “tokens per megawatt” counts

A data centre buys power in megawatts, so the number of tokens a site can produce for each megawatt it draws sets how much it can sell. Cerebras’s 5x is measured on one shared endpoint and normalised to a CS-3-only baseline of 1x.

The idea is simple: use the best chip for each stage of inference.

Andrew Feldman, Cerebras CEO, on X, 21 November 2026

AMD and Cerebras had announced the pairing on 23 July at Advancing AI 2026. AMD’s Helios rack would handle prompts and large context windows, and the Cerebras wafer would handle token generation, with the companies expecting up to 5x more tokens per second per watt, modelled with a Kimi 2.6 1T model against a wafer-only setup.

AWS is the other prefill partner. On 13 March Cerebras said AWS would deploy Cerebras CS-3 systems in its data centres, offered through Bedrock, and that the two were building a version where Trainium computes the KV cache and sends it to the Cerebras wafer over Amazon’s EFA network, for 5x more high-speed token capacity in the same footprint.

Why inference splits in two

A request runs in two stages. In prefill, the model reads the whole prompt and builds its working notes, the KV cache. In decode, it writes the answer one token at a time, and each token means reading the model from memory again. Cerebras says prefill is compute-bound and needs little memory bandwidth, while decode is bandwidth-bound.

That favours chips that keep the model in fast on-chip memory called SRAM. Cerebras says one of its wafers holds tens of gigabytes of SRAM, with 350x the SRAM capacity and 5,400x the memory bandwidth of a typical GB300 GPU. Those are its figures. Nvidia’s Groq 3 LPU, the chip it licensed from Groq, carries 500 MB of SRAM and 150 TB/s of SRAM bandwidth per chip, and a 256-chip LPX rack holds 128 GB.

The trade-off is capacity, and Nvidia’s answer is to split the work. Its technical blog describes Dynamo software that sends prompts and the attention work to Rubin GPUs and the feed-forward layers to the LPUs.

Flowchart: prefill reads the prompt on GPUs, hands the KV cache over a network, then decode writes the answer on fast-memory chips
Prompts go to GPUs; answers come from chips built to read memory fast.
Prefill
The first stage of a request, when the model reads the whole prompt and builds its working notes.
Decode
The second stage, when the model writes the response one token at a time.
KV cache
The saved key-value state from prefill that decode reads at every step.
SRAM
Fast memory built into the processor. It holds little and reads very quickly.
HBM
High-bandwidth memory stacked beside a GPU. It holds far more than SRAM and reads more slowly.
Token
A chunk of text, usually a word or part of one, that a model reads or writes.
Earlier on Sulat, 20 November: prefill and decode explained with Merve Noyan’s decision-model benchmark.

A timeline of the split

Dates come from company releases, filings and posts unless an outlet is named.

DateWhat happenedSource
24 Dec 2025Groq announces a non-exclusive licensing deal with Nvidia. Groq’s founder and CEO Jonathan Ross, president Sunny Madra and other staff join Nvidia.1Groq
14 Jan 2026OpenAI and Cerebras announce 750 MW of low-latency compute, coming online in tranches through 2028.OpenAI
13 MarAWS says it will deploy Cerebras CS-3 systems through Bedrock. The two plan to pair Trainium for prompts with the Cerebras wafer for answers.Cerebras
16 MarAt GTC, Nvidia unveils the Groq 3 LPX rack beside Vera Rubin and claims up to 35x throughput per megawatt, with availability in the second half.Nvidia
13 MayCerebras prices its IPO at US$185 a share and raises US$5.55 billion. Trading starts on Nasdaq as CBRS on 14 May.Cerebras, Reuters
9 Jund-Matrix says its Corsair accelerator is in full production and will ship in volume to hyperscalers, neoclouds and frontier labs.d-Matrix
23 JulAMD and Cerebras announce a Helios plus wafer-scale pairing for split inference.AMD
6 AugAMD agrees to buy Taalas, a Toronto startup that hardwires a model into silicon. Terms aren’t disclosed.AMD
18 AugEtched raises US$700 million at a US$21 billion valuation, led by Jane Street, and says it shipped its first rack to Jane Street in July.Etched
24 AugNvidia says Groq 3 LPX is in full production. Nebius is the first AI cloud to adopt it.Nvidia
10 SepPositron raises US$875 million at a US$5 billion valuation.Positron
12 NovCerebras reports record third-quarter revenue of US$210.8 million.Cerebras
15 NovSambaNova and Intel demonstrate an air-cooled rack at SC26: four Intel D-Series GPUs for prompts and a SambaRack SN50 for answers.Unite.AI
17 NovCerebras publishes its split-inference blog post. AMD completes its Taalas acquisition.Cerebras, AMD
21 NovFeldman posts the news on X.X
Split inference, 24 December 2025 to 21 November 2026. Sources are the announcing company unless an outlet is named.

Who is paired with whom

Map pairing prefill GPUs with decode chips: AMD and Trainium to Cerebras, Rubin to Groq 3 LPX, Intel to SambaNova, GPUs to d-Matrix
Every pairing passes the prompt’s saved state from a GPU to a decode chip.
CompanyDecode chipPrefill partnerStatusSource
NvidiaGroq 3 LPX, 256 LPUs a rackVera Rubin NVL72 GPUsIn full production from 24 August; Nebius first to adoptNvidia
CerebrasWafer-scale engine in CS-3 and CS-4 systemsAMD Instinct GPUs; AWS TrainiumSplit inference in production on its cloud with AMD; AWS in early accessCerebras
SambaNovaSN50 RDU, 16 a rackIntel D-Series GPUsDemonstrated at SC26 on 15 NovemberUnite.AI
d-MatrixCorsairGPUs, unnamedCorsair in full production since 9 Juned-Matrix
AMDTaalas hardwired chips, to be added to its roadmapInstinct GPUs in Helios racksTaalas deal closed 17 NovemberAMD
EtchedIts own chipsIts own chips; Etched says its racks handle prefill and decodeFirst rack delivered to Jane StreetEtched
Split-inference pairings announced by 21 November 2026. Each company’s own description.

NVIDIA today announced that NVIDIA Groq 3 LPX, the interactive AI inference accelerator, is now in full production.

Nvidia news release, 24 August 2026

Hardwired and air-cooled alternatives

Taalas builds the other extreme. Its chips cast one model’s weights and dataflow into silicon, so a part runs the model it was built for and nothing else. Its HC1 test chip served Meta’s Llama 3.1 8B at close to 17,000 tokens a second, a figure SiliconANGLE reports.

SambaNova’s vice president of core product, Anton McGonnell, told SiliconANGLE’s theCUBE that its racks stay air-cooled, that newer GPU racks can demand 120 to 250 kilowatts each, and that power and cooling equipment has 18 to 24 months of lead time. SambaNova’s demo rack needs about 20 kW, per its own figures. McGonnell’s comments are on SiliconANGLE.

The hyperscalers sit alongside

Google, Amazon and Microsoft build their own chips and appear here mainly as customers and hosts. Anthropic said on 6 April that it signed a deal with Google and Broadcom for multiple gigawatts of next-generation TPU capacity from 2027, and that it trains and runs Claude on Trainium, TPUs and Nvidia GPUs.


The money behind it

US$26.1 billion

Cerebras’s remaining performance obligations at 30 September

US$210.8 million

Cerebras’s GAAP revenue in the third quarter, up 55% on a year earlier

US$5.55 billion

raised in Cerebras’s May IPO

US$21 billion

Etched’s valuation after its August round

Cerebras’s third-quarter release put GAAP cloud revenue at US$141.9 million, up 265%, and GAAP gross margin at 20%. It guided core revenue of US$310 million to US$330 million for the fourth quarter and US$940 million to US$960 million for 2026, and Feldman said Cerebras expects to triple revenue in 2027. The release lists dependence on a limited number of significant customers among its risks.

The same release says Cerebras has 700 MW of data-centre capacity under contract for delivery by the end of 2027, that Oracle will offer Cerebras in its cloud in 2027, and that it signed a deal with an unnamed top-three hyperscaler. The CS-5, built on the WSE-5 chip that Cerebras says is back from the fab, is targeted for early summer 2027 at up to 10,000 tokens a second per user.

Etched says it has more than US$1 billion in customer contracts and has named one customer, Jane Street. Positron says its Atlas racks are deployed at Oracle Cloud Infrastructure, 50-plus racks. SambaNova was valued at US$11 billion in July, SiliconANGLE reports.

Earlier on Sulat, 19 November: Cerebras says it serves Kimi K3.1 Pro at up to 2,000 tokens per second.

How the headline numbers compare

Cerebras’s 5x is against its own CS-3-only setup on one model. Nvidia’s 35x is a projection against its previous rack, for models of a trillion parameters or more. AMD’s 5x is a July model against a wafer-only setup, using a different model again. The table lists who measured what.

ClaimWho measuredBaselineModel and settingSource
5x tokens per MWCerebras, measuredIts CS-3-only setupgpt-oss-120b, shared endpoint, 14 NovCerebras
Up to 35x tokens per MWNvidia, projectedGB200 NVL72Trillion-parameter modelsNvidia
Up to 5x tokens per second per wattAMD and Cerebras, modelledWafer-only setupKimi 2.6 1T, JulyAMD
Up to 2.5x throughputNvidia, MLPerf v6.1 previewGB300 NVL72DeepSeek-R1Nvidia
21,333 tokens a second per GPUAMD, its own MLPerf Offline setupPublished Vera Rubin NVL72 resultsDeepSeek-R1AMD
3,400 tokens a second, one userArtificial Analysis, per NvidiaNone givenGemma 4 31B, 100K contextNvidia
Headline performance claims in this story. Different baselines and models, so the rows don’t compare with each other.
Four stat cards: Cerebras 5x tokens per megawatt, Nvidia up to 35x, Cerebras US$26.1 billion contracted revenue, Etched US$21 billion value
Each figure uses a different baseline, so none compares with another.

What none of it shows

  • Cerebras’s blog post doesn’t give prompt lengths, load or a price per token for the 5x result, and its footer says speed comparisons against GPU systems “may vary depending on workload, configuration, date and models being tested”.2
  • Nvidia’s 24 August release says Nebius “plans to bring” LPX to its Token Factory. The release doesn’t name a customer running on one.
  • Etched has named one customer. Nvidia hasn’t disclosed what it paid Groq, and AMD hasn’t disclosed what it paid for Taalas.
  • No independent benchmark compares a Cerebras split setup with a Nvidia LPX rack. The Artificial Analysis test of LPX used one model, and Nvidia published it.
  • Feldman’s claim to be first applies to Cerebras’s own phrasing, “production-scale deployment combining GPU and non-GPU systems”. Nvidia announced LPX as in full production on 24 August.

What happens next

  • Cerebras speaks at a Barclays conference on 9 December. It targets the CS-5 for early summer 2027 and split inference across hundreds of megawatts by the end of 2027.
  • Nebius plans to add LPX to Token Factory, and Groq says it will deploy LPX itself with Dell.
  • AMD closed the Taalas deal on 17 November and hasn’t said when a product will follow.
  • Anthropic’s new Google and Broadcom TPU capacity starts coming online in 2027.

Sources

  1. CNBC reported a price of about US$20 billion on 24 December 2025. Groq’s announcement disclosed no price. ↩︎
  2. The Cerebras blog post carries this note in its footer. ↩︎

Discover more from Sulat AI & Tech News

Subscribe to get the latest posts sent to your email.