Linux 6.8 Network Optimizations Can Boost TCP Performance For Many Concurrent Connections By ~40%

[coder]

say that Youtube and all other major websites use Linux, what exactly would that prove?

My point was that when people are using both FreeBSD and Linux to do comparable things, I think we really can't conclude one is necessarily more suitable than the other, without further data.

Am I asking for too much ...

What's asking too much is to expect us to believe a thing without showing good supporting data. That's all I want. I don't care about your or mdedetrich's unicorn examples of Netflix and WhatsApp. For all I know, those companies' founders or core technical team were just BSD zealots who drank the cool aid and simply put enough effort into making it work for them.

[mdedetrich]

What's asking too much is to expect us to believe a thing without showing good supporting data. That's all I want. I don't care about your or mdedetrich's unicorn examples of Netflix and WhatsApp. For all I know, those companies' founders or core technical team were just BSD zealots who drank the cool aid and simply put enough effort into making it work for them.

You are asking for something thats not possible to provide because it would break NDA. The patches that allow FreeBSD to push 700+ gb/s encrypted TLS traffic from a single dual socket AMD rome machine are not public yet because its being still being prototyped. When it will be done it will be pushed upstream, just like every other change Netflix has done in the past.

The kind of smoking gun you are trying to provide cannot be provided, so stop trying to ask for it.

Also sorry to break it to you, but the benchmarks you post are so synthetic that they are pointless. In fact I would argue they are worse then pointless, they are deceptive because you are giving them an air to authenticity that they don't deserve. The issue with most synthetic benchmarks (and this is something I deal with as part of my day job because I maintain a project that deals with streaming/clustering, see Pekko) is that they almost never replicate what is actually run in production. This is exactly what is described from https://it-notes.dragas.net/2022/01/...ux-to-freebsd/

Whatsapp servers to Linux and move them to their datacenters. Regarding the real system performance (i.e. disregarding benchmarks, useful only up to a certain point), FreeBSD shines, especially under high load conditions. Where Linux starts to gasp (ex: waiting for I/O) with 100% CPU, FreeBSD has lower processor load and room for more stuff. In the real world (of my servers and load types), I sometimes experienced severe system slowdowns due to high I/O, even if the data to be processed was not read/write dependent. On FreeBSD this does not happen, and if something is blocking, it blocks THAT operation, not the rest of the system. When performing backups or other important operations this factor becomes extremely important to ensure proper (and stable) system performance.

The point here is that none of the benchmarks you posted actually replicate that scenario. And replicating those scenarios are really hard, often the only way to do it is to run your actual app in production with production payload while keeping everything constant aside from the OS. The thing is since these kinds of scenarios are not public you don't see it.

So please, stop posting useless benchmarks to try and provide credence to your posts. You want evidence that FreeBSD is better? Then run some existing non trivial baseload http app with TLS application under insane load under both FreeBSD and Linux and see the difference for yourself and not some stupid useless benchmark.

[coder]

You are asking for something thats not possible to provide because it would break NDA.

Not at all. If your claim is that FreeBSD has a better network stack, then Netflix' in-house patches shouldn't be needed to demonstrate that simple fact.

The kind of smoking gun you are trying to provide cannot be provided, so stop trying to ask for it.

Honestly, the flat earth community has better arguments than what you're giving us. You make a factual claim but dismiss the data we provide and counter with none of your own.

the benchmarks you post are so synthetic that they are pointless.

Then provide better ones.

The issue with most synthetic benchmarks (and this is something I deal with as part of my day job because I maintain a project that deals with streaming/clustering, see Pekko) is that they almost never replicate what is actually run in production.

They don't have to exactly match, in order to have predictive power. Do you not see the irony of arguing against benchmarks on a site that heavily features benchmarking and even an article that used benchmarks to quantify a performance optimization in Linux' network stack?

This is exactly what is described from https://it-notes.dragas.net/2022/01/...ux-to-freebsd/

This is the same article you linked before, and the only supporting evidence they provide of their networking performance claims links back to a 2014-era article. This smells exactly like some FreeBSD zealots that fell prey to confirmation-bias. If that 2014 article is the most relevant thing they could find, you should be worried.

The part you quoted about one I/O operation blocking another suggests the author doesn't understand how journalling filesystems work. Lots of people have been successful in tuning Linux to perform and scale incredibly well. I have no idea how competent the author and his team are, which is a good example of why we can't simply accept an anecdote - much less, one so greatly lacking in specifics.

So please, stop posting useless benchmarks to try and provide credence to your posts. You want evidence that FreeBSD is better? Then run some existing non trivial baseload http app with TLS application under insane load under both FreeBSD and Linux and see the difference for yourself and not some stupid useless benchmark.

I won't apologize for being evidence-based. If you reject synthetic benchmarks, then find application-level benchmarks you believe are better. But, you cannot argue against something with nothing. If you're going to make claims, you out to be able to back them up with data. If you cannot, then you're simply acting on faith.

You are asking us to believe your claims on the basis of faith. You are the religious zealot, in this conversation, and you're acting as if we're the unreasonable ones.

[Gipr]

Wow, I thought you were more reasonable than sophisticles, but now I'm starting to wonder.

It's not the first time that he's told something like:

Do you realize how primitive and obvious is what you are doing here by repeating this time after time? You are trying beforehand and generally depreciate any opponent's argument by transferring dialogue into a plane where they will give you rational and thoughtful points while you are free to give them nonsense and insults, because it is "unsafe space".

Been reading your inexhaustible run of comments under every rust topic for several years and it looked like a violent and ruthless manipulative behavior (people call it trolling nowadays) but nope, it is neither fun nor entertaining so much so, that it looks like something closer to a psychopathic condition.

Maybe you want to ease your condition?

[mdedetrich]

Not at all. If your claim is that FreeBSD has a better network stack, then Netflix' in-house patches shouldn't be needed to demonstrate that simple fact.

Your twisting this around, it was objectively better when previous past patches were upstreamed. When I was talking about patches in the articles I linked regarding +700 gbit/sec, I am literally talking about the most recent ones (i.e. within last year).

Honestly, the flat earth community has better arguments than what you're giving us. You make a factual claim but dismiss the data we provide and counter with none of your own.

Your also doing the same in reverse, your just using data that isn't even entirely relevant which is imho worse.

Then provide better ones.

Did you even read what was stated?

They don't have to exactly match, in order to have predictive power. Do you not see the irony of arguing against benchmarks on a site that heavily features benchmarking and even an article that used benchmarks to quantify a performance optimization in Linux' network stack?

I am sorry, but I have dealt with so much "benchmarketing" bullshit in the past I tbh don't care what the benchmarks show either way if they are not entirely relevant. I have learnt the hard way that aside from extremely simplistic scenario's which very rarely correspond to reality almost all of the time the benchmarks don't show the full picture.

The point I was making is the areas that FreeBSD shines is in areas where its not easy to show benchmarks. Its akin to the whole "latency vs throughput" angle, that is a system that is optimized to prioritise to latency over a wide distribution will often lose in basic simplistic benchmarks that only measure throughput.

This is the same article you linked before, and the only supporting evidence they provide of their networking performance claims links back to a 2014-era article. This smells exactly like some FreeBSD zealots that fell prey to confirmation-bias. If that 2014 article is the most relevant thing they could find, you should be worried.

That article is from 2022 Jan

The part you quoted about one I/O operation blocking another suggests the author doesn't understand how journalling filesystems work. Lots of people have been successful in tuning Linux to perform and scale incredibly well. I have no idea how competent the author and his team are, which is a good example of why we can't simply accept an anecdote - much less, one so greatly lacking in specifics.

You should re-read that paragraph, it has nothing to do with ignoring blocking calls (which is what I think you are talking about otherwise I don't know why journaling is relevant here????) but rather async IO. AIO in FreeBSD is kernel based (it uses an in kernel thread to prevent context switching) which makes it a lot more performant as it doesn't have to spawn a user mode thread in order to run operations on that thread as Linux does. This is probably solved in Linux with IO_Uring, but that is an extremely recent invention in Linux kernel (talking 1 to 2 years). FreeBSD has had this for over like a decade?

So yes thanks for inadvertadely pointing this out, this is another reason why FreeBSD has historically been better its because Linux has has utter shit support for asychnronous IO for userland applications up until very recently (IO_uring).

I won't apologize for being evidence-based. If you reject synthetic benchmarks, then find application-level benchmarks you believe are better. But, you cannot argue against something with nothing. If you're going to make claims, you out to be able to back them up with data. If you cannot, then you're simply acting on faith.

If you provide evidence that is not entirely relevant to the point being made and because of that its misleading, that is even worse than not posting relevant evidence at all. Your giving a false sense of security.

You are asking us to believe your claims on the basis of faith. You are the religious zealot, in this conversation, and you're acting as if we're the unreasonable ones.

Your not being unreasonable instead your putting yourself on a pedestal by providing evidence that on the surface looks compelling enough to convince innocent bystanders but is actually not refuting or support the claims being made either way.

I am being quite clear that in order to demonstrate these advantages, you yourself need do your own performance based testing on your own application and calculate the results yourself. If your picking technologies based on some simplistic benchmarks posted on some forum and leaving it there without doing proper benchmarking on your own applications and creating conclusions out of that then I am sorry but your an idjit.

I am saying myself I did this type of testing a decade ago and I indeed could easily replicate FreeBSD performing better than Linux (even with heavy tuning of both although FreeBSD didn't need it) when under extreme load. I can't replicate or publisize those results since I am not allowed because it was internally within a company benchmarking the companies own software (and also because I don't work there now). People can take that at face value or ignore it, I really don't care.

It's not the first time that he's told something like:

Do you realize how primitive and obvious is what you are doing here by repeating this time after time? You are trying beforehand and generally depreciate any opponent's argument by transferring dialogue into a plane where they will give you rational and thoughtful points while you are free to give them nonsense and insults, because it is "unsafe space".

Been reading your inexhaustible run of comments under every rust topic for several years and it looked like a violent and ruthless manipulative behavior (people call it trolling nowadays) but nope, it is neither fun nor entertaining so much so, that it looks like something closer to a psychopathic condition.

Maybe you want to ease your condition?

Thats nice, can we leave it to the adults now who are contributing something of use in the forums rather than finger pointing red scare style?

[coder]

Your twisting this around, it was objectively better when previous past patches were upstreamed. When I was talking about patches in the articles I linked regarding +700 gbit/sec, I am literally talking about the most recent ones (i.e. within last year).

Okay, so you're saying it's not better until Netflix' latest patches get upstreamed? That might be okay for you...

Your also doing the same in reverse, your just using data that isn't even entirely relevant which is imho worse.

Again, you claim it's not relevant (the rest of us disagree), but provide nothing better? That leads us back to "accepting based on faith", which might work for you but not me.

Did you even read what was stated?

Yup. Read all your posts, so far. I don't reply to something without reading it & trying to check the links.

I am sorry, but I have dealt with so much "benchmarketing" bullshit in the past I tbh don't care what the benchmarks show either way if they are not entirely relevant.

In order to reach a comparative assessment, you must have a defined workload that's run on both systems. If you say it has to be an application, then so bet it. However, you cannot make claims of comparative performance without supporting data. The only way to even reach such conclusions without data is based on faith.

That article is from 2022 Jan

Again, check what they reference, to support their performance claims. In that section of the article, the only source they reference is from 2014 and they provide no hard data of their own.

rather async IO. AIO in FreeBSD is kernel based (it uses an in kernel thread to prevent context switching) which makes it a lot more performant

So, let's see some benchmarks of FreeBSD AIO vs. io_uring.

This is probably solved in Linux with IO_Uring, but that is an extremely recent invention in Linux kernel (talking 1 to 2 years). FreeBSD has had this for over like a decade?

If it has it, then it has it. You're willing to compare patches you claim Netflix hasn't even upstreamed, yet it's somehow unfair to consider io_uring which, by the way, was officially released almost 5 years ago? Please, be consistent!

https://www.phoronix.com/review/linux-51-features

If you provide evidence that is not entirely relevant to the point being made and because of that its misleading, that is even worse than not posting relevant evidence at all. Your giving a false sense of security.

The solution to bad data is better data. From my perspective, the data we've provided seems relevant. You're the one saying it's not, so show us something better. As long as your argument isn't for us to accept facts based on faith...

your putting yourself on a pedestal by providing evidence

Fuck pedestals. I don't care about that shit. I care about data. You cannot reach a factual conclusion without data. As you've apparently reached such a conclusion, show us the data. If you cannot show us the data, then you're operating on hearsay or faith and that's just not good enough for me.

in order to demonstrate these advantages, you yourself need do your own performance based testing on your own application and calculate the results yourself.

So, you're claiming nobody in the entire world has ever benchmarked a server application on both Linux and FreeBSD and published the results? I can't accept that. Either you're not looking or you're just not liking the results you found when you did.

I did this type of testing a decade ago

I don't care about 2014 shit. This is 2024. I'm not contesting whether FreeBSD ever had an advantage over Linux, although that's not an uninteresting question.

[aviallon]

You: That's a well-known fact... even Microsoft Azure has more Linux usage than all of the other operating systems combined.

Me: Pretend that it's not that well known and show me some actual data.

You: (Paraphrasing here) See above link and quote.

Me: I assume you don't realize that you above doesn't support your claim.

Technically Linux refers to the kernel not the operating system. What you have shown is that Linux based operating systems combined account for greater than 60% of customer cores on Azure.

That 60% however is the combination of Red Hat OS, Ubuntu OS, Debian OS, and so on.

But more importantly you are making the same mistake that most Linux advocates make when evangelizing for Linux. Let's assume you found out that Red Hat 7 was by far the most popular choice for Azure, AWS and a bunch of others, would you suddenly stop using whatever distro you are currently using and switch to RH7?

Why or why not?

What?