The Performance Hit For A Xeon-Backed Ubuntu Linux VM With L1TF / Foreshadow Patches

Last week L1 Terminal Fault (a.k.a. L1TF and Foreshadow) was made public as the latest set of speculative execution vulnerabilities affecting Intel processors. This Meltdown-like issue was met by same-day Linux kernel patches for mitigating the problem and does introduce another performance penalty but in this case is at least only limited to virtual machines. Last week I posted some initial L1TF-mitigated KVM-based VM benchmark results using a Core i7 CPU but the results for sharing today are using a much more powerful dual Xeon server.

Red Hat does a great job as always explaining this security vulnerability.

For getting a better idea of the performance impact of mitigating L1TF/Foreshadow vulnerabilities I tested the Ubuntu patched kernel in a variety of configurations. First was the unmitigated Ubuntu 18.04 kernel, then Ubuntu 18.04 with the default out-of-the-box mitigation on the host and guest kernels, then having the host booted with the kernel parameter to force an L1D cache flush on every VMENTER rather than the default behavior of the conditional flushing, and then again when booting with l1tf=full for the full mitigation, which in the process also disables SMT/HT support.

The system used for this round of testing was a Tyan 1U server with two Intel Xeon Gold 6138 processors yielding a combined total of 40 cores / 80 threads, 128GB of RAM, and a Samsung 850 EVO SATA 3.0 SSD for storage. The KVM guest was the only VM running on the system during testing and was allowed access to 80% of the system's cores/threads (64 threads), 48GB of RAM, and 160GB of storage. Ubuntu 18.04 LTS was running on both the guest and host. The other default Spectre/Meltdown mitigations were present in their default behavior during testing including KPTI, __user pointer sanitization, and full generic IBPB IBRS_FW.

Via the Phoronix Test Suite a wide range of benchmarks were carried out from the Ubuntu Kernel-based Virtual Machine for comparing the performance impact introduced by the default L1 Terminal Fault / Foreshadow mitigation, then with the "always" flushing configuration, and finally with the "full" L1TF protection that disables Hyper Threading.

Update: To note, no microcode changes/updates were made to the systems under test for this article, just testing/comparing the kernel patches.