AMD Details The MI300X & MI300A, Announces ROCm 6.0 Software

At AMD's AI event today the company provided more details on their Instinct MI300 series for their very exciting data center APU and CDNA3 discrete GPU accelerator. ROCm 6.0 was also announced for advancing AMD's AI software capabilities.

With the AMD Instinct MI300X accelerator they are aiming it directly as being an excellent alternative to NVIDIA for AI accelerations.

Besides having more than twice the amount of HBM3 memory as the H100 SXM, the AMD Instinct MI300X at least for its peak theoretical potential should outpace the NVIDIA competition. The total board power for the MI300X is 750 Watts. Up to eight MI300X accelerators can be combined into a single server for offering 1.5TB of HBM3 memory and potentially 1.3x more compute power than the NVIDIA H100 HGX.

The numbers provided by AMD appear in very good standing against NVIDIA's H100 competition. Unfortunately no independent testing in advance of today's announcement.

What I'm personally most intrigued by is the Instinct MI300A as this APU accelerator for AI and HPC. The MI300A with Zen 4 CPU cores, AMD CDNA3 graphics, and 128GB of HBM3 unified memory provides a lot of interesting potential in the data center as an alternative to Xeon Max for that Sapphire Rapids CPU with HBM2e memory and ahead of Intel's Falcon Shores APU coming in 2025.

The MI300A has 24 Zen 4 CPU cores, 256MB AMD infinity Cache, eight HBM3 stacks for around 5.3 TB/s of memory bandwidth, and 228 AMD CDNA3 compute units.

AMD's benchmarks show great HPC and AI performance for ROCm workloads able to leverage the incredible memory bandwidth and combination of the CNDA3 compute units and Zen 4 CPU cores.