AMD previews MI300X with 192GB of memory

AMD introduced its Instinct MI300X accelerator on 13 June, pairing up to 192GB of HBM3 memory with its CDNA 3 architecture. The company says a large language model such as Falcon-40 can fit on one accelerator. Its event announcement also describes a platform containing eight MI300X devices.

AMD’s Santa Clara headquarters, photographed on 21 June 2020. Cropped. Photo: Coolcaesar / Wikimedia Commons, CC BY-SA 4.0. This crop is licensed under CC BY-SA 4.0. Company context, not the MI300X chip or June 2023 event.

The hardware is at different stages: MI300A, the combined CPU-and-GPU accelerator for scientific computing and AI, is already sampling to customers. MI300X sampling for key customers is planned for the third quarter. June’s presentation therefore gives developers specifications and a timetable, rather than general availability.

Software has its own rollout

A same-day Hugging Face announcement adds an important qualification to the hardware pitch. The partnership will begin with Instinct MI2xx and MI3xx GPUs, followed by Radeon Navi3x. It also covers inference on Ryzen and EPYC CPUs and work on the Alveo V70 accelerator.

Hugging Face plans to test models across PyTorch, TensorFlow and ONNX Runtime, but warns that not every model will support both training and inference on every framework and processor. Integrating ROCm into its libraries and developing a dedicated Optimum library are part of the work ahead.

Read together, these announcements identify separate checks for a deployment: whether the model fits in memory, whether the chosen software combination works, and whether the hardware is available. The partnership is a development plan. It isn’t a blanket compatibility guarantee.

SDxCentral’s event report corroborates the split sampling schedules and reports 128GB of memory and 24 Zen 4 CPU cores for MI300A. Those details help distinguish the two products: the MI300 name covers more than one hardware configuration.