Cloudflare Teases Next-Gen Server Design, Benefits Going From 1U To 2U Servers

Two years ago Cloudflare rolled out their "Gen 11" server fleet built around AMD EPYC Milan processors and on Friday the company began talking about their forthcoming "Gen 12" server designs that will soon be rolling out across their data centers for powering this widely-used web infrastructure.

Cloudflare's blog is often home to a lot of interesting technical insight from this leading web company. While Cloudflare has long leveraged 1U server designs, with their Gen12 servers they are moving to a 2U design due to the increased power/thermal requirements of the latest-generation server processors from AMD and Intel.

Cloudflare hasn't yet publicly said whether they are using 4th Gen Intel Xeon Scalable or AMD Zen 4 (Genoa/Genoa-X or Bergamo), but in Friday's blog post they noted that with their Cloudflare services they have noted scaling up to 128 cores / 256 threads. They also noted a CPU TDP sweet spot at around 340 Watts. With scaling up to 128 cores / 256 threads, it wouldn't be at all surprising if they went with an AMD EPYC "Bergamo" design that features 128 cores / 256 threads today with the likes of the EPYC 9754.

Cloudflare shows off their Gen 12 server design.

Due to the higher CPU TDPs, they found benefit going from 1U to 2U form factor as in their racks they have more space available than racks. In going for a 2U layout with their Gen 12 servers they are moving to larger fans, larger heatsinks to help in heat dissipation, and enjoying less air impedance, and room for more PCIe-attached accelerators / GPUs / add-on cards.

They further noted in their blog post on their 2U benefits:

Thermal simulations provided by Cloudflare vendors showed that 4x 60mm fans or 4x 80mm fans at less than 40 Watt per fan is sufficient to cool the system. That is a theoretical savings of at least 150 Watt compared to 8x 40mm fans in a 1U design, which would result in significant Operational Expenditure (OPEX) savings and a boost to TCO improvement. Switching to a 2U form factor also gives us the benefit of fully utilizing our rack power budget and our rack space, and provides ample room for the addition of PCIe attached accelerators / GPUs, including dual-slot form factor options.
...
It might seem counter-intuitive, but our observations indicate that growing the server chassis, and utilizing more space per node actually increases rack density and improves overall TCO benefit over previous generation deployments, since it allows for a better thermal design. We are very happy with the result of this technical readiness investigation, and are actively working on validating our Gen 12 Compute servers and launching them into production soon.

It will be interesting to see ultimately how they spec'ed out their Gen 12 servers and other technical details to be published soon. At least from the details published so far, it's likely they went for AMD's Bergamo while their follow-up posts should shed better detail.