Linux 6.8 Network Optimizations Can Boost TCP Performance For Many Concurrent Connections By ~40%

[Volta]

Thanks for the summary table coder. The kernel engineer did go above and beyond what was expected to conclude which stack was better (not that it would have prevented FreeBSD people to think it better than Linux, or for that matter people with dead set ideas).
A fresh look would be nice to see if there was any progress on both OSes (there should have been) and more specifically how much.

What needs to be taken into account I assume Ubuntu will be used with its powersafe governor. It's nothing like you would use on the server. I don't know if FreeBSD has any power management, but I bet it's always running in the fastest mode.

[Volta]

Empirically this is actually not a good reason for something being popular, i.e. just because its popular doesn't mean its fast. If you look into the history of Linux, especially in the late 90's/2000's era you will see that none of the reasons why Linux took off had to do with performance, and its definitely an objective fact that for example at that time Linux's network stack was demonstrably much slower than FreeBSD.

Nope. You're just repeating FreeBSD propaganda that they were spreading years ago. I remember benchmark coming from bsd guys from around 2000. The only problem on Linux side was malloc library. When it was replaced it performed better than FreeBSD. Performance is the main reason why Linux took off. That's why FreeBSD could only dream about HPC and many other workloads. It was only server OS until Linux nearly entirely replaced it. But here it is, Linux vs BSD TCP performance benchmark from 2003:

http://bulk.fefe.de/scalability/

The clear winner in the graph is Linux 2.6. OpenBSD does not scale at all, and even panics under high load. NetBSD scales O(n), which is respectable for the grandfather of all the BSDs, but it is not a winning performance. Linux 2.4 shows that there is work to be done; I give it the third place. FreeBSD looks like it would scale O(1) if I could create more processes with it, but as long as I can't confirm it, I can only give it the second place.

Conclusion: Linux 2.6 is the clear winner, scaling O(1) in every respect. The clear loser is OpenBSD; I have never seen bad performance of this magnitude. Even Windows would probably outperform OpenBSD. NetBSD performance leaves a lot to be desired as well. This mmap graph is the only part of the whole benchmark suite where FreeBSD did not perform top notch. If the FreeBSD people fix this one dark spot, they will share the top space with Linux 2.6.

Linux 2.6 scales O(1) in all benchmarks. Words fail me on how impressive this is. If you are using Linux 2.4 right now, switch to Linux 2.6 now!

FreeBSD 5.1 has very impressive performance and scalability. I foolishly assumed all BSDs to play in the same league performance-wise, because they all share a lot of code and can incorporate each other's code freely. I was wrong. FreeBSD has by far the best performance of the BSDs and it comes close to Linux 2.6. If you run another BSD on x86, you should switch to FreeBSD

This one is funny and it shows were windows always was:

Conclusion: Linux 2.6 is the clear winner, scaling O(1) in every respect. The clear loser is OpenBSD; I have never seen bad performance of this magnitude. Even Windows would probably outperform OpenBSD

sophisticles

eat that fanboy.

[coder]

Thanks for posting those excerpts.

sophisticles

eat that fanboy.

We need to accept that we're probably dealing with a toxic narcissist. He can't accept that he's wrong, because he has a deep, psychological need to be right. Since he's got his self-worth intertwined with his self-righteousness, accepting he's wrong creates a deep pain in his psyche and threatens to invalidate his existence. That marshals his very survival instincts to the cause and turns up his powers of self-deception to 11.

Either correct his misstatements or ignore him, but don't expect him to agree with you. The only chance of wining him over, in an argument, is if you can spin your position so that he can claim it actually aligns with what he was saying (or doesn't disprove it), and he was right all along. Most of the time, his positions are so extreme that would be impossible to do while also being factually accurate.

[sophisticles]

https://azure.microsoft.com/en-us/products/virtual-machines/linux

More than 60 percent of customer cores in Azure run Linux workloads. Choose from popular Linux distributions including Red Hat, SUSE, Ubuntu, CentOS, Debian, and CoreOS. Use preconfigured solutions from Oracle and other open-source virtual machine (VM)-compatible providers and find Azure-optimized Linux VM images from publishers of your choice. Get the cloud's only co-located technical support from Azure, Red Hat, and SUSE with just one ticket. Choose the underlying processors_AMD, Arm64, or Intel_that best meet your requirements.

You: That's a well-known fact... even Microsoft Azure has more Linux usage than all of the other operating systems combined.

Me: Pretend that it's not that well known and show me some actual data.

You: (Paraphrasing here) See above link and quote.

Me: I assume you don't realize that you above doesn't support your claim.

Technically Linux refers to the kernel not the operating system. What you have shown is that Linux based operating systems combined account for greater than 60% of customer cores on Azure.

That 60% however is the combination of Red Hat OS, Ubuntu OS, Debian OS, and so on.

But more importantly you are making the same mistake that most Linux advocates make when evangelizing for Linux. Let's assume you found out that Red Hat 7 was by far the most popular choice for Azure, AWS and a bunch of others, would you suddenly stop using whatever distro you are currently using and switch to RH7?

Why or why not?

[sophisticles]

Anyway, I care a lot more about data than anecdotes. If you have statistically significant reliability data to share, feel free to.

There is this:

https://papers.freebsd.org/2019/fosdem/looney-netflix_and_freebsd/

https://itsfoss.com/netflix-freebsd-cdn/

Normally I don't like this type of "proof" because it falls into the "appeal to authority" class of evidence but since i do have a background in video editing and I used to consult for a company that produced music videos and since Netflix is the type of business where their servers will get hammered with lots of traffic and they absolutely need reliable, fast performance, I do think it's significant that they chose BSD over any other OS for the content delivery system.


Further, the PS3, PS4, and PS5 use a BSD based OS:

https://www.playstation.com/en-us/oss/ps4/freebsd-kernel/

https://en.wikipedia.org/wiki/PlayStation_3_system_software

https://en.wikipedia.org/wiki/PlayStation_4_system_software

As I have said numerous times, money is where the rubber meets the road.

When cash is on the line that is when you find out what someone actually believes.

I have said for years that if MS takes a page out of Apple's playbook and decides to release a BSD based Windows, Linux use would dry up within a year.

[janos666]

Off-topic but I have been trying to to set up RDMA between my Linux homeserver and Windows workstation for ~10 years. KSMBD was promising but according to my knowledge it still won't handle linking Windows email address based accounts to Linux users.

[Volta]

There is this:

[...]netflix[...]

Netflix, you say? There is this:

https://lkml.org/lkml/2016/4/23/194

I tested 1 and 2 node NUMA, as that is typical for my employer (Netflix, and our tens of thousands of Linux instances in the AWS/EC2 cloud), even though I wasn't expecting any difference on 1 node. I've used synthetic workloads so far.

As I have said numerous times, money is where the rubber meets the road.

You mean Google, Meta and dozens of other running on Linux? Ever heard of SAP? You won't see BSD there.

I have said for years that if MS takes a page out of Apple's playbook and decides to release a BSD based Windows, Linux use would dry up within a year.

What you're saying never makes sense, so keep it up. Linux killed BSD just like it killed Slowlaris. Apple and Microsoft are next. Tell me why there's no Windows phone anymore and why Microsoft moved to Linux in its cloud? Rhetoric question of course. You have no basic clue about operating systems. There's no way to turn any of BSD's into something serious as Linux. It's out of the game on most of the markets. BSD's had a chance in 90's, but Linux wiped them out.

[Volta]

Thanks for posting those excerpts.

We need to accept that we're probably dealing with a toxic narcissist. He can't accept that he's wrong, because he has a deep, psychological need to be right. Since he's got his self-worth intertwined with his self-righteousness, accepting he's wrong creates a deep pain in his psyche and threatens to invalidate his existence. That marshals his very survival instincts to the cause and turns up his powers of self-deception to 11.

Either correct his misstatements or ignore him, but don't expect him to agree with you. The only chance of wining him over, in an argument, is if you can spin your position so that he can claim it actually aligns with what he was saying (or doesn't disprove it), and he was right all along. Most of the time, his positions are so extreme that would be impossible to do while also being factually accurate.

No problem. You're also doing great job. This one reminds me slowlaris troll kebabbert who lost his job, because his company replaced slowlaris servers with Linux. He used to be little smarter, though. Sophimindless is too primitive. However, maybe he regressed. We can easily ignore him, but he's giving us opportunity to show how miserable non Linux operating systems are. It's also great way to remind old Microsoft FUD and FreeBSD propaganda. There was plenty of it in the 2000s.

[mdedetrich]

Nope. You're just repeating FreeBSD propaganda that they were spreading years ago. I remember benchmark coming from bsd guys from around 2000. The only problem on Linux side was malloc library. When it was replaced it performed better than FreeBSD. Performance is the main reason why Linux took off. That's why FreeBSD could only dream about HPC and many other workloads. It was only server OS until Linux nearly entirely replaced it. But here it is, Linux vs BSD TCP performance benchmark from 2003:

Wrong again, it didn't have anything to do with malloc. FreeBSD has better TCP algorithms along with taking advantage of things like NUMA, see

• https://2019.eurobsdcon.org/slides/NUMA%20Optimizations%20in%20the%20FreeBSD%20Networ k%20Stack%20-%20Drew%20Gallatin.pdf as an example
• https://netflixtechblog.com/serving-...e-cdb51dda3b99
• https://people.freebsd.org/~scottl/N...n-20130515.pdf

Also most cases where people complain about FreeBSD being slower than Linux, its usually not FreeBSD's fault but rather the hardware they are using is not well supported and in some cases using a different TCP algorithm solves the problem (see https://forums.freebsd.org/threads/s...0/#post-398453)

Like I am not sure people realize, but FreeBSD used to run systems like WhatsApp, probably one of the largest non trivial setups you can think of and the only reason they moved to Linux is because they were forced to do some from management after Facebook acquired them.

[Volta]

Wrong again, it didn't have anything to do with malloc. FreeBSD has better TCP algorithms along with taking advantage of things like NUMA, see

Nope, it was malloc. It seems we have another one who ignores reality. As was already shown FreeBSD sucks in comparison to Linux in networking. Don't even mention NUMA, because FreeBSD is light years behind Linux in this case. What are you pasting here? Some slides from 2019? How does it compare to 2003? Maybe I shouldn't ask.. Let's check when OS's got support for NUMA:

FreeBSD 9 - 2012
Windows 7 - 2009
Linux 2.5 - ............................ 2002! Voila!