Intel Announces Gaudi 3 AI Accelerator, Intel Xeon 6 Brand

Intel is using its Vision 2024 conference in Arizona today to announce the Gaudi 3 AI accelerator. With Gaudi 3 comes some rather bold AI claims from Intel: 50% on average better inference and 40% on average better power efficiency than the NVIDIA H100. All while costing "a fraction" of the NVIDIA H100. Gaudi 3 sounds quite promising and will be interesting to see how its adopted in the marketplace. In addition, Intel also is disclosing the new Xeon 6 branding for their upcoming server processors formerly codenamed Sierra Forest and Granite Rapids.

Intel announced the Gaudi 3 AI accelerator as the next iteration of the Habana Labs Gaudi line and successor to Gaudi 2. Gaudi 3 will be available this year and comes ahead of the much anticipated Intel Falcon Shores.

Gaudi 3 is a 5nm design and features 2x the FP8 compute, 4x the BF16 compute, 2x the network bandwidth, and 1.5x the memory bandwidth over the already very capable Gaudi 2 AI accelerator.

Gaudi 3 features 64 5th gen tensor processor cores, 8 matrix math engines, 128GB of HBM with 3.7 TB/s bandwidth, 96MB sRAM, 24 x 200 GbE, and PCIe 5.0 x16 connectivity.

At least for Intel's own projections, they expect Gaudi 3 to sharply outperform the NVIDIA H100 across various large language models.

Like AMD and unlike NVIDIA, Intel's Gaudi software stack is open-source software and a much more open ecosystem at large compared to the walled CUDA gardens.

Gaudi 3 will be available as an OAM-compliant HL-325L accelerator card, an HLB-325 universal baseboard, and as the HL-338 PCIe CEM add-in card.

Gaudi 3 air-cooled and liquid-cooled parts are sampling in H1'2024 while the volume production availability is expected in H2'2024.

So far I haven't yet seen any public patches working on bringing up Gaudi 3 support within the Habana Labs accelerator kernel driver for Linux, but presumably that will be starting very soon to get the good open-source upstream support in place by the time of the volume ramp.

In addition to Gaudi 3, Intel is announcing today the Intel Xeon 6 Processors. Intel Xeon 6 processors are intended for data centers, cloud, and edge. Intel Xeon 6 processors with E-cores will launch this quarter (Sierra Forest) while Intel Xeon 6 with performance cores (Granite Rapids) will launch "soon after the E-core processors." With the Intel Xeon 6 Sierra Forest CPUs they are promoting today a 2.4x performance per Watt improvement and 2.7x better rack density than 2nd Gen Xeon processors. For Intel Xeon 6 Granite Rapids CPUs they are today promoting the MXFP4 data format

Gaudi 3 is sounding like a nice upgrade for the Gaudi product line ahead of Intel Falcon Shores that integrates their next-gen Xe GPU for AI paired with Intel Gaudi IP on the same chip for delivering a oneAPI power house. The hardware is great and will continue to be backed by Intel's open-source software support. Over on the Intel Xeon 6 side we're certainly excited to see Sierra Forest and Granite Rapids in the lab for Linux testing and performance benchmarking.