Netflix Continues Experiencing Great Performance In Using FreeBSD For Their CDN

[F.Ultra]

Here are some benchmarks made by an ex-Red Hat employee. FreeBSD was performing quite poorly in some of the scenarios. It was even freezing under high load which seems to be in contrary to Haxor claims in another thread. It was pointed out VirtIO drivers which were used for a few of the tests could be a culprit. However, on a physical interface results didn't drastically change. That's why I found those benchmarks quite interesting. It also seems FreeBSD have higher syscall overhead. If someone has similar up to date benchmarks it will be nice to share them.

https://medium.com/@matteocroce/linu...g-cbadcdb15ddd

Thanks, that was interesting to read. Do however remember that Netflix runs a highly patched FreeBSD kernel (e.g they perform the TLS encryption inside the network stock) but that one is kinda hard to benchmark since only Netflix have access to it since not all patches have been moved upstream yet (and some might never do).

[bnolsen]

Careful... (that's not why)

No but on our cdn centos systems we get systemd crashes about 10-20 a day. Handling the traffic is a piece of cake. Linux handles the load just fine, cpu usage is under 20% utilization (sandy and ivy dual xeon systems), 100GB or more ends up being limited by intel pci express lanes when nvme x4 drives are in play. The next generation stuff should be able to handle 2x100GB and a bank of nvme cache drives. We'll see how good amd's pcie bus is...

Linux does have zero copy api packts. I don't know if nginx uses this feature but ATS does not.

openssl 1.1.1 makes tls handling almost free cpu wise, it is a serious upgrade.

[aht0]

Thanks, that was interesting to read. Do however remember that Netflix runs a highly patched FreeBSD kernel (e.g they perform the TLS encryption inside the network stock) but that one is kinda hard to benchmark since only Netflix have access to it since not all patches have been moved upstream yet (and some might never do).

I remember seeing pointed out to you that they use standard BSD stack, just running FreeBSD from -CURRENT branch (atm FreeBSD-13) and making use of Mellanox hardware. Yeah, the latter is not your average desktop NIC but it's not supposed to be. It's enterprise use-case after all. Hardware offloading in itself is not something special, there are ton of more common network adapters doing heavy hardware offloading as well. Intel NIC's probably the best known. Why do you suppose these NIC's need passive heat sinks for cooling?

There might be some or even more patches to the kernel - I won't know enough to argue you here, compared to public FreeBSD code but that's pretty standard approach in industry. Unless they want to have the butterfly effect of accumulating patches on top of every new release, they are smart to contribute back as much as possible and keep back as little as possible.

[stefantalpalaru]

I don't understand. Why BSD is faster than Linux in CDN scenario?

Because the core FreeBSD contributor hired by Netflix modified the kernel for the kind of network traffic that he was optimising for.

If they would have hired a Linux core contributor, you'd be reading now about Netflix's "love affair" with Linux.

[Weasel]

Thanks for showing just how clueless you anti systemd trolls are (if you truly are serious with that statement).

You should thank hreindl he turned me and a lot of others into anti-systemd trolls.

[aht0]

You should thank hreindl he turned me and a lot of others into anti-systemd trolls.

Be joyous instead. That montrosity will end up killing itself eventually. I at some point realized it and now the thought of it keeps me cool.

I noticed my OpenSuse has systemd-bash-completion nowadays. Google search listed bunch of issues related to it, like "horribly slow". What's next? systemd-media-player? systemd-ass-wiper?

So, if Poettering, Kay & Co keep working on systemd over the coming years.. Heh, we can expect systemd take over more and more of the GNU/Linux components and effectively sabotage itself.

https://www.theregister.co.uk/2014/04/05/torvalds_sievers_dust_up/

[pal666]

everyone else Continues Experiencing Great Performance In Using linux For Their CDN

[pal666]

I don't understand. Why BSD is faster than Linux in CDN scenario?

i don't understand why do you think that bsd is faster? linux isn't mentioned in the presentation. they use freebsd because facebook uses php.

[pal666]

Not really. Sony uses BSD for the PS4. Apple uses BSD as a base for a lot of their stuff. They make some of the most popular devices in the world.

no, they don't. android is used by most of devices in the world. apple and sony use bsd because bsd allows them to steal code and they like to steal code.

[pal666]

balance of stability and features, a strong development community and staff expertise

well, linux is better on all counts. truth is the one making decision knew freebsd, they are sticking to it since then(if you are willing to believe "we aren't afraid of gpl")