Linux 5.17 To Boast A Big TCP Performance Optimization

[phoronix]

Phoronix: Linux 5.17 To Boast A Big TCP Performance Optimization

While the Linux 5.16 merge window just ended and that kernel won't be out until the tail end of the calendar year, already for Linux 5.17 new material is beginning to accumulate in the respective subsystem development trees... One set of changes merged this morning from Google can provide a sizable performance win around TCP performance in the datacenter...

https://www.phoronix.com/scan.php?page=news_item&px=Linux-5.17-TCP-Optimization

[mikus]

This is pretty significant that normal userland can start using effectively bandwidth at these higher 100gig interface rates without needed special zero-copy/pf_ring architecture involvement with root, proprietary drivers, applications to make use of it, etc. Netflix publishes some good docs on how they get ~350Gbps out of 4x 100gbe nics on freebsd with kTLS, curious if/when one can see this sort of bandwidth use in linux.

[Laughing1]

Nice.

[Grinness]

This is an improvement of about 12% across the 2 cases -- this is good, but the 82 to 95 look bigger only because base is bigger ...

[Volta]

Everyone benefits. Not just proprietary company like Netflix.

[Space Heater]

Great! Hopefully this means that one day Linux will be on par with BSDs in network performance.

Care to explain this comment? FreeBSD is likely better for Netflix's specific use case with sendfile and kTLS, but I don't see any clear evidence that Linux has inferior overall network performance to FreeBSD, let alone that Linux has worse network performance compared to NetBSD, OpenBSD, or DragonflyBSD.

[Nth_man]

Everyone benefits. Not just proprietary company like Netflix.

GPL vs BSD?

[perpetually high]

Sah-weet... and it patches cleanly against 5.16-rc1, compiling it now. Doesn't patch cleanly against 5.15.2, however.

edit: our good friend sirlucjan has added the 5.15 ported patches for the TCP optimizations. (and it's now included in my kernel build script also (if you know, you know)

[microcode]

This is pretty significant that normal userland can start using effectively bandwidth at these higher 100gig interface rates without needed special zero-copy/pf_ring architecture involvement with root, proprietary drivers, applications to make use of it, etc. Netflix publishes some good docs on how they get ~350Gbps out of 4x 100gbe nics on freebsd with kTLS, curious if/when one can see this sort of bandwidth use in linux.

It depends on what exactly you're trying to do. If it's just one giant bulk TCP stream per core, this is true; if it's thousands or tens of thousands of connections (even serving stuff straight out of memory) then you still benefit from io_uring and other new techniques/ABIs.

[Volta]

Great! Hopefully this means that one day Linux will be on par with BSDs in network performance.

Linux beats BSD in network performance by large margin. It seems you're smoking too much.

https://matteocroce.medium.com/linux-and-freebsd-networking-cbadcdb15ddd

The first thing that struck me, is that FreeBSD packet rate was substantially the same with one or 8 CPU. I investigated a bit, and I’ve found it to be a known issue: bridging under FreeBSD is known to be slow because the if_bridge driver is pratically monothread due to excessive locking