10GbE Linux Networking Performance Between CentOS, Fedora, Clear Linux & Debian

[Michael]

There are many online. Broadcom have theirs, Intel have theirs, Mellanox have theirs ... They're mostly the same, tunning your tcp stack settings and congestion algorithms based on lan or wan scenarios, pinning nic interrupt processing threads to specific cores, enlarging the nic queue lengths, making sure that offloads are enabled etc. Mellanox even has a script that does all of that for you.

Happen to have a link to the Mellanox script? (Haven't looked yet for that script, but I did pick up two Mellanox adapters the other day...) so might do a tuned vs. stock benchmark comparison from that perspective. Thanks.

[pegasus]

Check https://community.mellanox.com/s/art...mlnx-tune-tool

It's part of their driver package that you can find at http://www.mellanox.com/page/product...duct_family=27

[claytyler]

Congrats for expanding into new benchmarking territory, but there are new dragons here. These numbers all seem way too low. I regularly max out 100Gbit on old ivy bridge storage nodes running centos 6 and that's with less than 30min spent on tuning them. 10Gbit today can be maxed out with a single core ...

I'm curious about what you did to tune them

[claytyler]

The point of testing Fedora, I suspect, is to get a newer kernel. Also, Fedora is essentially RHEL/CentOS-next. CentOS is used because it's practically identical to RHEL. As a starting point, this was a good mix of distros.

I think it'd be better to test CentOS with a newer kernel rather than test Fedora.
I run CentOS servers with newer kernels (4.18 currently).
While I play with "CentOS-next" I wouldn't put it on a public facing server and I haven't really heard of anyone doing it.

[claytyler]

Deploy... do you even know what Haiku is? The nighties are usually pretty stable which is why nobody bother'd to do a Beta release for 7 years.

No, I don't actually.
This is the first I've ever heard of it.
I'll take a look.
Thanks for mentioning it.

[cb88]

No, I don't actually.
This is the first I've ever heard of it.
I'll take a look.
Thanks for mentioning it.

That's a bit supprising as it has been around awhile. Anyway it is an Open Source MIT licensed reimplementation of BeOS.... That aims to match and in many ways exceed what BeOS was by R1. It's a desktop operating system first and foremost. And trys to make good defaults rather than lots of options, though it isn't lacking of those for where they are needed.

[gilboa]

Testing out the new stuff, maybe.
Actually running it, not unless it's something not critical and you like to like on the edge...

We build our appliance around Fedora as its a great base when you need latest-and-greatest kernel and libraries.
We continue maintaining specific packages (E.g. OpenSSL, kernel) once a certain Fedora release is EOL (E.g. we still maintain Fedora 25).

BTW, we are not alone, at least one of the biggest competitors (a multi-billion company) uses heavily modified Fedora for their appliance.

- Gilboa

[claytyler]

We build our appliance around Fedora as its a great base when you need latest-and-greatest kernel and libraries.
We continue maintaining specific packages (E.g. OpenSSL, kernel) once a certain Fedora release is EOL (E.g. we still maintain Fedora 25).

BTW, we are not alone, at least one of the biggest competitors (a multi-billion company) uses heavily modified Fedora for their appliance.

- Gilboa

Nice to know!
Thanks!!

[gilboa]

Nice to know!
Thanks!!

You're very welcome.

- Gilboa