AMD + Older Intel CPUs To See Much Faster AES-NI XTS Crypto Performance On Linux 5.12

[caligula]

Note that `cryptsetup benchmark` benchmarks the crypto algorithms via AF_ALG, the userspace interface to the kernel crypto API. As such it is benchmarking not just the crypto itself, but also all the overhead of making system calls to write/read the data to/from sockets. That extra system call overhead doesn't exist when the algorithms are used in the kernel (e.g. for dm-crypt or fscrypt). So the results of `cryptsetup benchmark` aren't very meaningful when the algorithms are already very fast, e.g. AES-NI accelerated AES-XTS. For example the results could appear bad if system calls are slow on a particular system, yet the actual in-kernel crypto could be just as fast as a system on which system calls are faster.

Interesting. I wonder how large the overhead could be on modern systems, e.g. latest gen Atom/APU or Ryzen 5900X?

[f0rmat]

I am using a SuperMicro H11SSL-NC, which has excellent peripheral expansion capabilities in the ATX MB format.

I use on that motherboard 2 GPU 2-slot cards (used for compute applications), one 10 Gb/s Ethernet NIC and I use the on-board SAS controller to connect an LTO tape driver, on which I backup all my data.

Thanks. I love SuperMicro motherboards. They may not always have the latest and greatest, but they are rock solid and indestructible. Generally their QA/QC is among the best in the business - hell, I think that they ran QA/QC with fire and electricity for at least ten years before they started using either.

[AdrianBc]

Thanks. What motherboard are you running your Epyc 7401P on?

I am using a SuperMicro H11SSL-NC, which has excellent peripheral expansion capabilities in the ATX MB format.

I use on that motherboard 2 GPU 2-slot cards (used for compute applications), one 10 Gb/s Ethernet NIC and I use the on-board SAS controller to connect an LTO tape driver, on which I backup all my data.

[f0rmat]

Since many years ago, I am using a cluster of 6 servers, for running various computer-aided design programs and for testing various programs that are intended to run over a network. (I design electronic hardware and also software.)

I usually upgrade one of the servers each year. Many years ago, some of the servers used AMD Opterons, but since 2009, after Intel Nehalem, after several upgrades they eventually used only Intel Xeons.

However, during the last 3 years, I have chosen for the upgrades AMD CPUs, so now only half of the servers are using older Intel Xeons (a Skylake Xeon, a Kaby Lake Xeon and a Coffee Lake Xeon), while the 3 newer servers use a 24-core Epyc 7401P, a Ryzen 7 3700X and a Ryzen 9 5900X.

Both Ryzens use ASUS workstations motherboards (Pro WS X570-ACE), which have good ECC memory support, together with ECC memory.

The servers are interconnected by 10 Gb/s Ethernet links and all the communication between themselves is encrypted with AES in counter mode, so I easily notice after an upgrade if the encryption speed is improved or not.


The newer Intel Tiger Lake CPUs have also improved a lot the speed of AES in comparison with the older Skylake derivatives and I am not sure which is faster at AES, between Intel Tiger Lake (or the future Rocket Lake) and AMD Zen 3.

However all the Skylake derivatives, up to Comet Lake, are slower at the AES instructions than AMD Zen.

Thanks. What motherboard are you running your Epyc 7401P on?

[AdrianBc]

Nice collection in your toy box. Care to share?

Since many years ago, I am using a cluster of 6 servers, for running various computer-aided design programs and for testing various programs that are intended to run over a network. (I design electronic hardware and also software.)

I usually upgrade one of the servers each year. Many years ago, some of the servers used AMD Opterons, but since 2009, after Intel Nehalem, after several upgrades they eventually used only Intel Xeons.

However, during the last 3 years, I have chosen for the upgrades AMD CPUs, so now only half of the servers are using older Intel Xeons (a Skylake Xeon, a Kaby Lake Xeon and a Coffee Lake Xeon), while the 3 newer servers use a 24-core Epyc 7401P, a Ryzen 7 3700X and a Ryzen 9 5900X.

Both Ryzens use ASUS workstations motherboards (Pro WS X570-ACE), which have good ECC memory support, together with ECC memory.

The servers are interconnected by 10 Gb/s Ethernet links and all the communication between themselves is encrypted with AES in counter mode, so I easily notice after an upgrade if the encryption speed is improved or not.


The newer Intel Tiger Lake CPUs have also improved a lot the speed of AES in comparison with the older Skylake derivatives and I am not sure which is faster at AES, between Intel Tiger Lake (or the future Rocket Lake) and AMD Zen 3.

However all the Skylake derivatives, up to Comet Lake, are slower at the AES instructions than AMD Zen.

[ebiggers]

Note that `cryptsetup benchmark` benchmarks the crypto algorithms via AF_ALG, the userspace interface to the kernel crypto API. As such it is benchmarking not just the crypto itself, but also all the overhead of making system calls to write/read the data to/from sockets. That extra system call overhead doesn't exist when the algorithms are used in the kernel (e.g. for dm-crypt or fscrypt). So the results of `cryptsetup benchmark` aren't very meaningful when the algorithms are already very fast, e.g. AES-NI accelerated AES-XTS. For example the results could appear bad if system calls are slow on a particular system, yet the actual in-kernel crypto could be just as fast as a system on which system calls are faster.

[numacross]

Still way faster than AMD EPYC 7702 64-Core .

And way more expensive too

[f0rmat]

I own several Intel Xeons and a couple of Ryzens (3700X & 5900X), and the AMD CPUs are clearly faster at AES.

Nice collection in your toy box. Care to share?

[AdrianBc]

2 x Intel Xeon Platinum 8280 < Intel Core i3-10100

Damn those expensive, top of the line Xeons are very slow indeed

Benchmarking fixed function hardware (and that's what AES-NI is) is quite complex, but it looks like AMD's implementation is indeed slower in general.

I do not know what is wrong in those benchmarks, but it certainly is not caused by AES implementation. Maybe it is something specific to XTS.

In all the benchmarks that I have ever seen, through several generations of Intel and AMD CPUs, Ryzens/EPYCs were faster at AES than Intel CPUs.

I own several Intel Xeons and a couple of Ryzens (3700X & 5900X), and the AMD CPUs are clearly faster at AES.

[oleid]

Still way faster than AMD EPYC 7702 64-Core .

These are really desperate times for Intel fan boys. The only thing they have left is this fixed function hardware.

Personally, I prefer non-hardware crypto. Anything, which can be audited by third parties.