Intel Announces Gaudi 3 AI Accelerator, Intel Xeon 6 Brand

Written by Michael Larabel in Processors on 9 April 2024 at 11:35 AM EDT. Page 1 of 1. 8 Comments.

Intel is using its Vision 2024 conference in Arizona today to announce the Gaudi 3 AI accelerator. With Gaudi 3 comes some rather bold AI claims from Intel: 50% on average better inference and 40% on average better power efficiency than the NVIDIA H100. All while costing "a fraction" of the NVIDIA H100. Gaudi 3 sounds quite promising and will be interesting to see how its adopted in the marketplace. In addition, Intel also is disclosing the new Xeon 6 branding for their upcoming server processors formerly codenamed Sierra Forest and Granite Rapids.

Intel Gaudi 3 advantage

Intel announced the Gaudi 3 AI accelerator as the next iteration of the Habana Labs Gaudi line and successor to Gaudi 2. Gaudi 3 will be available this year and comes ahead of the much anticipated Intel Falcon Shores.

Intel Gaudi 3 AI accelerator

Gaudi 3 is a 5nm design and features 2x the FP8 compute, 4x the BF16 compute, 2x the network bandwidth, and 1.5x the memory bandwidth over the already very capable Gaudi 2 AI accelerator.

Intel Gaudi 3 details

Gaudi 3 features 64 5th gen tensor processor cores, 8 matrix math engines, 128GB of HBM with 3.7 TB/s bandwidth, 96MB sRAM, 24 x 200 GbE, and PCIe 5.0 x16 connectivity.

Intel Gaudi 3 vs. NVIDIA H100

At least for Intel's own projections, they expect Gaudi 3 to sharply outperform the NVIDIA H100 across various large language models.

Intel Gaudi 3 with open-source software

Like AMD and unlike NVIDIA, Intel's Gaudi software stack is open-source software and a much more open ecosystem at large compared to the walled CUDA gardens.

Intel Gaudi 3 baseboard

Gaudi 3 will be available as an OAM-compliant HL-325L accelerator card, an HLB-325 universal baseboard, and as the HL-338 PCIe CEM add-in card.

Intel Gaudi 3 details

Gaudi 3 air-cooled and liquid-cooled parts are sampling in H1'2024 while the volume production availability is expected in H2'2024.

Intel Gaudi 3 announcement

So far I haven't yet seen any public patches working on bringing up Gaudi 3 support within the Habana Labs accelerator kernel driver for Linux, but presumably that will be starting very soon to get the good open-source upstream support in place by the time of the volume ramp.

Intel Xeon 6 brand

In addition to Gaudi 3, Intel is announcing today the Intel Xeon 6 Processors. Intel Xeon 6 processors are intended for data centers, cloud, and edge. Intel Xeon 6 processors with E-cores will launch this quarter (Sierra Forest) while Intel Xeon 6 with performance cores (Granite Rapids) will launch "soon after the E-core processors." With the Intel Xeon 6 Sierra Forest CPUs they are promoting today a 2.4x performance per Watt improvement and 2.7x better rack density than 2nd Gen Xeon processors. For Intel Xeon 6 Granite Rapids CPUs they are today promoting the MXFP4 data format

Intel Xeon 6 is Granite Rapids and Sierra Forest

Gaudi 3 is sounding like a nice upgrade for the Gaudi product line ahead of Intel Falcon Shores that integrates their next-gen Xe GPU for AI paired with Intel Gaudi IP on the same chip for delivering a oneAPI power house. The hardware is great and will continue to be backed by Intel's open-source software support. Over on the Intel Xeon 6 side we're certainly excited to see Sierra Forest and Granite Rapids in the lab for Linux testing and performance benchmarking.

If you enjoyed this article consider joining Phoronix Premium to view this site ad-free, multi-page articles on a single page, and other benefits. PayPal or Stripe tips are also graciously accepted. Thanks for your support.

Related Articles
About The Author
Michael Larabel

Michael Larabel is the principal author of and founded the site in 2004 with a focus on enriching the Linux hardware experience. Michael has written more than 20,000 articles covering the state of Linux hardware support, Linux performance, graphics drivers, and other topics. Michael is also the lead developer of the Phoronix Test Suite, Phoromatic, and automated benchmarking software. He can be followed via Twitter, LinkedIn, or contacted via