Cloudflare Ditches Nginx For In-House, Rust-Written Pingora

[pixelherodev]

Did... anyone here actually read the original post? Because it's very clear that the performance improvements have nothing to do with Rust, at all.

> This is not because we run code faster. Even our old service could handle requests in the sub-millisecond range.

Oh, huh. So their code isn't faster? Then why is it using 1/3rd the resources, both CPU and RAM?

> The savings come from our new architecture which can share connections across all threads. This means a better connection reuse ratio, which spends less time on TCP and TLS handshakes. Across all customers, Pingora makes only a third as many new connections per second compared to the old service.

(Emphasis mine)

The programming language has nothing to do with it. Their proxy makes 1/3rd the connections, and thus uses 1/3rd the resources. It's a direct correlation. Nginx could be modified to see the same exact win, but it'd be nontrivial, which is exactly why CloudFlare says they didn't do it.

This isn't 'Oh wow, Rust is so much faster!', it's 'Oh wow, doing less work is faster!'

Edit:

Actually, they do mention that rewriting into Rust [was one factor that] improved performance - but not from C, from Lua. That's wildly different.

[CommunityMember]

Of course, if I were Cloudflare I'd almost hesitate to open source since it could give them a huge service and cost savings advantage and kick their major competitors in the nuts.

My guess (and it is just a guess based on their past of open source activities) is they will open source it after they have reviewed the code for possible patentable ideas (and filed as appropriate), as having such patents, or obtaining related patents via a DPA is often a necessary defense against NPEs.

[pixelherodev]

I was also sitting here wondering if this kind of implementation in Rust could happen for io_uring? Meaning, can Rust do for io_uring (hi Jens!) what it did for Pingora or is it apples and oranges?

No, it can't - the bottleneck for nginx was that it was creating a massive amount of connections, each of which requires an expensive [at scale] TLS handshake.

There's a wild misconception here, that the programming language is the key factor to performance: it's not. It's a factor, but not even close to the most important: an O(1) algorithm running in three layers of interpreters will outperfom an O(N^100) algorithm written in C or Rust in 99.9% of cases.

What matters is the code, not the language it's written in.

[CommunityMember]

Losing Cloudflare is kind of a big deal

Cloudflare, and Dropbox before them, dropping use of nginx is mostly an issue of promotional marketing ("nginx is used by ...."), as I think both were using the open source version of the product (so no real monetary loss for F5 (the parent company)). One must also consider that Cloudflare and Dropbox are at a completely different scale than perhaps 99.95% of any other current or potential user of nginx, so nginx still has a pretty good story to tell for the rest of us.

[cj.wijtmans]

Cloudfare is the most annoying piece of network stack to having an open and secure internet.

[onlyLinuxLuvUBack]

Not bad, 70 % less resources is a real deal in this business. They probably got back the development money for this project after one month.

Let's see what our dear Rust haters come up with this time.

cool kids use python :P

[shmerl]

Looking forward to them open sourcing that stack, especially HTTP library.

[Quackdoc]

Did... anyone here actually read the original post? Because it's very clear that the performance improvements have nothing to do with Rust, at all.

> This is not because we run code faster. Even our old service could handle requests in the sub-millisecond range.

Oh, huh. So their code isn't faster? Then why is it using 1/3rd the resources, both CPU and RAM?

> The savings come from our new architecture which can share connections across all threads. This means a better connection reuse ratio, which spends less time on TCP and TLS handshakes. Across all customers, Pingora makes only a third as many new connections per second compared to the old service.

(Emphasis mine)

The programming language has nothing to do with it. Their proxy makes 1/3rd the connections, and thus uses 1/3rd the resources. It's a direct correlation. Nginx could be modified to see the same exact win, but it'd be nontrivial, which is exactly why CloudFlare says they didn't do it.

This isn't 'Oh wow, Rust is so much faster!', it's 'Oh wow, doing less work is faster!'

Edit:

Actually, they do mention that rewriting into Rust [was one factor that] improved performance - but not from C, from Lua. That's wildly different.

We chose Rust as the language of the project because it can do what C can do in a memory safe way without compromising performance.

I would say doing it in a memory safe way is critical, so the preformance improvement there is heavily implied

[RahulSundaram]

What matters is the code, not the language it's written in.

It is obviously both. No matter how good your code is, if you have say very performance sensitive requirements, some languages are going to be entirely ill suited for the job. Even if you can theoretically code the best performing code in assembly, very few people if any are going to be fluently writing multi threaded assembly code, so a language that allows you to easily express that at a higher level with safety guarantees and without a runtime cost would be better overall because it reduces programmer cost. Languages do absolutely matter.

[ssokolow]

It's a common misconception that rewriting things in Rust makes them faster because Rust is a fast language. Rewriting them in Rust makes them faster because it makes you, the developer, willing to choose an architecture or set of optimizations that you wouldn't want to debug and maintain in C or C++.

Hell, Node.JS can be faster than Rust if you hammer away at optimizing it and overlook some tunable (eg. a buffer size) that it gives a more favourable default to than Rust. That's why NPM rewrote one of their components and found it was about the same speed but kept the Rust rewrite because it wasn't a constant fight to keep it from throwing exceptions in weird edge cases that became common at the scale they were running it.