Cloudflare Ditches Nginx For In-House, Rust-Written Pingora

[stormcrow]

Did... anyone here actually read the original post? Because it's very clear that the performance improvements have nothing to do with Rust, at all.

> This is not because we run code faster. Even our old service could handle requests in the sub-millisecond range.

Oh, huh. So their code isn't faster? Then why is it using 1/3rd the resources, both CPU and RAM?

> The savings come from our new architecture which can share connections across all threads. This means a better connection reuse ratio, which spends less time on TCP and TLS handshakes. Across all customers, Pingora makes only a third as many new connections per second compared to the old service.

(Emphasis mine)

The programming language has nothing to do with it. Their proxy makes 1/3rd the connections, and thus uses 1/3rd the resources. It's a direct correlation. Nginx could be modified to see the same exact win, but it'd be nontrivial, which is exactly why CloudFlare says they didn't do it.

This isn't 'Oh wow, Rust is so much faster!', it's 'Oh wow, doing less work is faster!'

Edit:

Actually, they do mention that rewriting into Rust [was one factor that] improved performance - but not from C, from Lua. That's wildly different.

I did wonder if anyone would actually pick up on the fact they aren't really comparing nginx to Pingora. That's an apples to oranges comparison. They're comparing an off the shelf solution to a custom solution designed to handle their specific use case.

I would have been surprised if they didn't have a considerable performance and resource advantage given the circumstances. That also doesn't mean that anyone else will necessarily see any benefit to Pingora, either, without having something about their circumstances being at least somewhat similar to Cloudflare's.

Figured I'd add: Rewriting in Rust at least gives the benefit of a certain amount of memory safety which takes several failure cases off the table, so that is a net positive even if performance had turned out to have been similar in either case.

[cl333r]

Good evening earthlings,
Did I understand it correctly that the main reason it runs much faster is because it uses threads instead of processes?

[Chugworth]

Web servers like Nginx and Apache are designed to handle a very wide range of web hosting scenarios. It makes sense that custom-written solution for a specific use case would perform better.

[darkonix]

Good evening earthlings,
Did I understand it correctly that the main reason it runs much faster is because it uses threads instead of processes?

It appears so, the new design using threads is not depending on Rust. The language is not the cause of the new design. That's a common misconception. Rust is safer, not faster nor slower than C.

[stormcrow]

It is obviously both. No matter how good your code is, if you have say very performance sensitive requirements, some languages are going to be entirely ill suited for the job. Even if you can theoretically code the best performing code in assembly, very few people if any are going to be fluently writing multi threaded assembly code, so a language that allows you to easily express that at a higher level with safety guarantees and without a runtime cost would be better overall because it reduces programmer cost. Languages do absolutely matter.

Languages shape how humans see the world. Computer languages do indeed shape how we create machines in a computer. The less expressive a language is, the more constrained it will be in how one creates a machine - and the less likely you are to reach weird states. The more expressive a language is and the more 'open ended' a language is the less constraints you place on a machine and more likely you are to reach weird states. That's the entire premise of language theory. Yes, the language absolutely matters beyond the sheer economics of programmer salaries.

[piotrj3]

Did... anyone here actually read the original post? Because it's very clear that the performance improvements have nothing to do with Rust, at all.

> This is not because we run code faster. Even our old service could handle requests in the sub-millisecond range.

Oh, huh. So their code isn't faster? Then why is it using 1/3rd the resources, both CPU and RAM?

> The savings come from our new architecture which can share connections across all threads. This means a better connection reuse ratio, which spends less time on TCP and TLS handshakes. Across all customers, Pingora makes only a third as many new connections per second compared to the old service.

(Emphasis mine)

The programming language has nothing to do with it. Their proxy makes 1/3rd the connections, and thus uses 1/3rd the resources. It's a direct correlation. Nginx could be modified to see the same exact win, but it'd be nontrivial, which is exactly why CloudFlare says they didn't do it.

This isn't 'Oh wow, Rust is so much faster!', it's 'Oh wow, doing less work is faster!'

Edit:

Actually, they do mention that rewriting into Rust [was one factor that] improved performance - but not from C, from Lua. That's wildly different.

Rust actually makes multithreading much easier. So to certain point that architecture that shares connection among all threads is possible in big part thanks to Rust as doing that in C would be hella hard to do safe.

[RahulSundaram]

Yes, the language absolutely matters beyond the sheer economics of programmer salaries.

To be clear, when I said 'cost', I wasn't just referring to salaries. I was using it a far more abstract way. Things like error rate, cognitive load etc are 'costs' that one must deal with. Otherwise, we are in agreement.

[Developer12]

Did... anyone here actually read the original post? Because it's very clear that the performance improvements have nothing to do with Rust, at all.

> This is not because we run code faster. Even our old service could handle requests in the sub-millisecond range.

Oh, huh. So their code isn't faster? Then why is it using 1/3rd the resources, both CPU and RAM?

> The savings come from our new architecture which can share connections across all threads. This means a better connection reuse ratio, which spends less time on TCP and TLS handshakes. Across all customers, Pingora makes only a third as many new connections per second compared to the old service.

(Emphasis mine)

The programming language has nothing to do with it. Their proxy makes 1/3rd the connections, and thus uses 1/3rd the resources. It's a direct correlation. Nginx could be modified to see the same exact win, but it'd be nontrivial, which is exactly why CloudFlare says they didn't do it.

This isn't 'Oh wow, Rust is so much faster!', it's 'Oh wow, doing less work is faster!'

Edit:

Actually, they do mention that rewriting into Rust [was one factor that] improved performance - but not from C, from Lua. That's wildly different.

That's only a partial truth.

Why can they share connections like that? Because they switched from a multi-process to a multi-thread model. Why can they do that? Doesn't it loose tons of isolation? It does, but having written it in rust gives them the confidence they need that data won't get leaked from one thread to another.

Improved memory safety guarantees can have REAL performance benefits.

[jrch2k8]

That's only a partial truth.

Why can they share connections like that? Because they switched from a multi-process to a multi-thread model. Why can they do that? Doesn't it loose tons of isolation? It does, but having written it in rust gives them the confidence they need that data won't get leaked from one thread to another.

Improved memory safety guarantees can have REAL performance benefits.

This is more "We don't wanna deal with modifying this huge general use codebase for our ultra specific purpose, let write something new optimized for our needs" more than a Rust specific thing.

Sure Rust/Go/C++/whatever you like best can provide some benefits as a language but the big performance meat is from the design changes and how well it adjust to your hardware / use case. Hell they can prolly had used vala and still be faster than nginx for their specific use case.

[GreenReaper]

This reminds me of the work Wikimedia has done to ease updating of Wikidata - essentially rewriting how they update their graph database so as to reduce the load on it.

Outside of Wikimedia, relatively few installations of Wikibase are going to need this solution because they don't have the same massive amount of updating traffic, and they probably don't want the extra setup and dependencies involved in using it.

Personally I have been largely happy with nginx, despite its idiosyncrasies. But I'm not running a CDN on CloudFlare's scale, doing all that it does. I suspect most of our CPU usage attributed to it is likely to be in TLS, and we have limited upstream connections.