Intel Xeon 6780E / Xeon 6766E 144-Core Performance Benchmarks

With Intel's launch today of the Xeon 6700E series processors formerly codenamed Sierra Forest I am now able to share benchmarks of these new E-core server processors. Here is an initial look at the Intel Sierra Forest Linux performance using the Xeon 6766E and Xeon 6780E 144-core server processors in both single and dual socket configurations compared to prior Intel Xeon processors and the AMD EPYC competition.

See the Intel Xeon 6 / 6700E series overview for more background information on these new Xeon 6 E-core (Sierra Forest) processors now shipping. For the launch day testing of the Xeon 6700E series, Intel kindly provided their new Xeon 6700 reference server as well as the Xeon 6766E and Xeon 6780E SKUs. Both are 144 core processors but the 6766E is a 250 Watt part while the 6780E as the flagship of the 6700E series is rated for a 330 Watt TDP. The Xeon 6766E with its reduced power budget has a 1.9GHz base clock compared to 2.2GHz with the 6780E while the 6766E has a 2.7GHz turbo maximum clock and the 6780E has a 3.0GHz turbo.

Both the Xeon 6766E and Xeon 6780E have a 108MB L3 cache, DDR5-6400 eight channel memory support, DSA / IAA / QAT / DLB accelerators, Intel TDX support, four UPI links, and 88 PCIe 5.0 Express lanes.

The closest competition to the Xeon 6766E / 6780E processors is the AMD EPYC 9754 that is 128 cores (256 threads) using Zen 4C. The EPYC 9754 has a 360 Watt TDP, 2.25GHz base clock, 3.1GHz boost clock, 256MB L3 cache, and 128 PCIe 5.0 lanes. The Bergamo/Genoa(X) processors support 12 channel memory but at DDR5-4800 speeds. Zen 4C does support AVX-512 and other ISA features in common with Zen 4 as well as having SMT support. Intel lacks SMT/HT but has 12% more physical cores than the EPYC 9754.

Ampere Computing's AmpereOne promises 144, 160, and 192 core SKUs but there have been no review samples there nor any other widespread availability of AmpereOne servers yet or even public availability in the cloud with the likes of Oracle Cloud still being in private preview. The lack of availability and independent reviews still baffles me as we've been talking about AmpereOne for two years now and it was public knowledge that Sierra Forest would arrive by the middle of 2024 that only increases competition in the cloud native space so surprising without any AmpereOne push prior to today's Intel launch. So given no "real" availability yet on AmpereOne, the Intel Xeon 6700E series for the moment has the core count advantage that is nice to see with Intel for the past number of years having trailed AMD EPYC in core counts. For those wondering how Intel's Xeon 6700E processors compare to existing Ampere Altra processors for up to 128 ARMv8 cores, I will be running some benchmarks there in power and performance-per-Watt in the next week or two. But given the maturity of the Ampere Altra platform, the results shouldn't be particularly surprising.

The Intel Xeon 6700E reference server used for this 6766E/6780E testing is a Quanta Cloud QuantaGrid D55Q-2U that is equipped with 16 x 32GB DDR5-6400 memory. It is quite similar overall to the Intel reference server platform from Sapphire Rapids but updated for handling Xeon 6.

This Intel reference server has been working out well but with the major notable exception of not being able to POST with PCIe 5.0 NVMe storage attached... If any PCIe 5.0 SSD is installed, the server didn't POST and was a known issue to Intel. They are working through with a firmware update to hopefully address the issue. The server POSTs fine without any storage or using PCIe 5.0 NVMe or latter. By the time this Intel Xeon 6 reference server arrived, I had already been re-testing existing AMD/Intel processors using a PCIe 5.0 Kioxia NVMe SSD. In order to use the same drive for this Sierra Forest benchmarking, I had to install the drive after the system powered on and thus had to install the OS to a PCIe 4.0 drive and then carry out the benchmarks from the desired PCIe 5.0 drive that was hot-plugged.

One exciting aspect of the Intel reference server is that they've switched over to OpenBMC! OpenBMC is now in use for that Linux-based BMC software stack rather than the conventional proprietary BMCs. Great seeing Intel's reference servers now hop on the OpenBMC bandwagon.

All of my Intel Xeon 6766E/6780E testing so far has been done using Ubuntu 24.04 LTS. Testing was done both with the default Linux 6.8 based kernel as well as a Linux 6.9 based kernel (the benchmarks are on Linux 6.9 in when starting the CPU re-testing in April, I wasn't aware initially how well Sierra Forest would work on 6.8 relative to the very latest 6.8 upstream kernel code at this time). Intel has the Sierra Forest support good-to-go at launch for the Linux kernel and related components, the GCC and LLVM/Clang compiler support being upstreamed long ago, etc. No complaints in terms of the Linux/open-source support for the Intel Xeon 6 processors and platform support.