Cloudflare Improving Linux Disk Encryption Performance - Doubling The Throughput

Cloudflare employs Linux disk encryption on their servers and with some optimizations have made it at least two times faster throughput while also lowering the latency.

Cloudflare began exploring Linux disk encryption performance when finding it wasn't performing as well as they would like. Cloudflare engineers dug into the Linux kernel source tree and worked to avoid extra queuing and asynchronous behavior.

With a patch that adds a new flag to dm-crypt to not offload crypto operations and handle everything inline was one of the steps. The other essential change was implementing a Crypto API AES-XTS synchronous driver that uses AES-NI by default while falling back to generic AES if needed. Those patches are currently available via Cloudflare's GitHub.

Cloudflare engineers found with the changes they could more than double their throughput while the latency was about cut and half. This is helping Cloudflare already in production with their encrypted Linux servers but they still are exploring more areas to improve upon.

This post shows how an architecture review can double the performance of a system. Also we reconfirmed that modern cryptography is not expensive and there is usually no excuse not to protect your data.

We are going to submit this work for inclusion in the main kernel source tree, but most likely not in its current form. Although the results look encouraging we have to remember that Linux is a highly portable operating system: it runs on powerful servers as well as small resource constrained IoT devices and on many other CPU architectures as well. The current version of the patches just optimises disk encryption for a particular workload on a particular architecture, but Linux needs a solution which runs smoothly everywhere.

More details on this work via the Cloudflare Blog.