Intel Linux Kernel Optimizations Show Huge Benefit For High Core Count Servers

[phoronix]

Phoronix: Intel Linux Kernel Optimizations Show Huge Benefit For High Core Count Servers

Earlier this month I wrote about Intel engineers working on more big optimizations to the Linux kernel with a focus on enhancing the kernel's performance at high core counts. The numbers shared then were very promising and since then I've had more time looking at the performance impact of Intel's stellar software optimization work and its impact on real-world workloads. Here is a look at how Intel's pending kernel optimization patches are a huge deal for today's high core count servers.

https://www.phoronix.com/review/intel-scalability-optimizations

[guspitts]

Interesting results. The dip in performance with 120 threads is a bit concerning though. The best performance is obtained when using 1/4 of the hardware in the machine? That's not good. Do we know if it is a problem with Intel's processor or the kernel?

[arjan_intel]

Interesting results. The dip in performance with 120 threads is a bit concerning though. The best performance is obtained when using 1/4 of the hardware in the machine? That's not good. Do we know if it is a problem with Intel's processor or the kernel?

this is how scaling issues show up... you get lock or cacheline contention... and the contention tanks overall performance -- this is not something for a specific cpu; that is a generic thing that happens everywhere. What can be per cpu is where this "knee" in the graph is... but one you hit mass contention the game is mostly over.
That is, until the code is changed to avoid the contention. then the knee moves much to the right (to the next bottleneck if any)

the tanking happens because all the cores contending for the cache line, are taking up resources (cache line bounces, snoops etc etc) away from the task that does do the work while not making forward progress themselves.

[kylew77]

Would like to see DragonFlyBSD thrown in since it claims to be better than other kernels in high SMT situations. See if it dips as much as Linux does at high core counts. I must say this was shocking as a discovery. Never dreamed 60 cores would preform better than 120 cores or 240 threads!

[CochainComplex]

Amdahls law

[Mark Rose]

MySQL/Maria/Percona should perform much than those MariaDB results with tuning. Take a look at how Percona tuned it for benchmarking high core count VMs:

https://www.percona.com/blog/comparing-amd-epyc-performance-with-intel-xeon-in-gcp/

innodb_thread_concurrency is particularly important.

[Mark Rose]

I hope Intel sends you one of those 960 core Sapphire Rapids boxes. Maybe ask for one. They'd dominate AMD on the benchmarks for the next few years ;-)

[Volta]

Would like to see DragonFlyBSD thrown in since it claims to be better than other kernels in high SMT situations. See if it dips as much as Linux does at high core counts. I must say this was shocking as a discovery. Never dreamed 60 cores would preform better than 120 cores or 240 threads!

Would be interesting to see. DragonFlyBSD only supports up to 128 CPU cores and 256 hardware threads while Linux supports up to 8192 CPUs (/threads?). However, it doesn't mean it will be slower.

[Mark Rose]

Would be interesting to see. DragonFlyBSD only supports up to 128 CPU cores and 256 hardware threads while Linux supports up to 8192 CPUs (/threads?). However, it doesn't mean it will be slower.

8192 threads. The NR_CPUS thing comes from before SMT and multi-core. What we call a CPU has drifted while to Linux it's still a single execution unit.

[coder]

I hope Intel sends you one of those 960 core Sapphire Rapids boxes. Maybe ask for one. They'd dominate AMD on the benchmarks for the next few years ;-)

I'm pretty sure that's 960 thread, but 480 core. AFAIK, Intel Xeon only scales up to 8 sockets, for the most scalable models. I'm aware of servers with like 32 CPUs, but anything beyond 8 would need to have some extra glue that will surely come at a cost, latency-wise. Benchmarks like these would suffer grievously.

I imagine a 480-core Sapphire Rapids box must have a 30 Amp plug.