Benchmarks: FreeBSD 13 vs. NetBSD 9.2 vs. OpenBSD 7 vs. DragonFlyBSD 6 vs. Linux

[JackLilhammers]

Classical
Regardless of which operating system is faster for a particular workload, your sources are old.
5-6 years, 9 in the case of nginx, are a long time in computer science.
You can't expect to be taken seriously if you try to prove your point with information that old

[kraileth]

Because your examples and claims doesn't make any sense. KVM eats bhyve for breakfast:

https://www.reddit.com/r/homelab/com...ce_comparison/

Now that as a general statement is plain wrong. If your name didn't pop up first when thinking "user on Phoronix to be deliberately about as ignorant regarding *BSD as possible", I'd suggest that you try to run both in production for a while.

You won't believe it, but there are people who do - and the results might not be to your liking either. The majority of the VMs that the company I work for runs is on Proxmox/KVM. The reason for this is plain and simple: It's good enough and very simple to administer (i.e. you can hire basically every admin and he or she is pretty likely to be able to work with it). Customers who have special requirements, though, may in fact be better served with Bhyve. Administration certainly is a bit more involved (there's work being done to eventually have a nice web UI, too, but that's volunteer work of a single person compared to how much paid work goes into the same thing on the Linux side). But depending on your use case it may or may not be worth it.

Ah ok so that clears it up: you are a BSD fanboy. Go look at some more serious data and then there will be more to talk about.

I see, user Classical pointing out that there in fact are good reasons to choose BSD over Linux in various scenarios make him a "fanboy". Well, by sticking this label to everybody who's interested in a balanced view of the topic rather than trying to "prove" the ultimate superiority of the one single beatifying OS, you're basically extending the group of BSD fanboys to include all reasonable people. Job well done!

Like all BSD zealots you also make the mistake to believe that Docker is somehow the Linux equivalent of FreeBSD jails.

Another nice one; I've met a couple of die-hards in the BSD camp and I've got to say: What sets them apart from their much more numerous counterparts on the Linux side of things, they all knew their stuff. One thing penguin fanatics always seem to forget is that the "enemy" they picked is about guaranteed to know Linux as well while they - with very few exceptions - are completely ignorant of what they hate. So: No. When BSD people are referring to Docker in trying to make a comparison to jails, it's because they are simplifying things in the hope of helping the other person understand and as a starting point for a discussion.

It isn't, Linux's jails are LXD and I'm not aware of any current security issues with it.

Sorry, but that's not correct. Unless that's a typo and you actually meant LXC, you're referring to a management tool that leverages LXC to do it's thing. The latter is a containerization technology that provides some features that could be compared to FreeBSD's jails. Since in contrast to Linux FreeBSD is an operating system, it includes tools to manage its jails. While it has evolved over time, there are multiple jail managers available from ports or packages that are more convenient to use and provide easier access to some advanced features. LXD or Docker can be compared to those.

At any rate yes it does tend to have more frequent security advisories than jails largely for a simple reason: unlike jails it is actually used in the real world and not just on hobby servers.

While there really was no reason to stress that you have no idea of what you talking about you just had to, didn't you? Jails have been implemented to solve a real-world problem and have been used in production in large scale. Sun saw them, understood their value and built their own technology in the form of Solaris zones. Again those were rapidly adapted.

It's just the penguins who mostly didn't understand what this might be good for. There were people who did things like OpenVZ, but the Linux community at large didn't care. It's always kind of cute when the kiddies that haven't been around before the hype started think that "they" invented containers with Docker and on their high horse ask if this BSD thing could do something like that, too! What would you tell tem?

Like any widely used tech, Docker has security vulnerabilities and there are much more reports of them than for something barely anyone uses in prod. You will find much less security disclosures about Minix than about Linux. By itself that doesn't imply that Minix is necessarily more secure, just that it's less used and less scrutinised.

I certainly hope you don't want to imply that Linux Containers are more secure because there are more people who have looked at them. The problem here is: They weren't designed for security! The latter was an afterthought - and things like that are always recipe for a nightmare.

Only when running FreeBSD as a paravirtualised guest. In cases that actually matter, like running other OSes, the picture is very different (https://forums.freebsd.org/threads/b...rements.66977/).

Do you care to explain why you dug out a forum post about FreeBSD 11.x instead of 13? This line of releases was the first where Windows support for Bhyve landed and only barely. I'm lucky enough to only work with Open Source operating systems, so I don't have any first-hand experience with how well Windows virtualization works. But from what you heard in the last few years, things improved quite a bit.

Oh, and here's one for you: Our customer with the highest individual number of Bhyve VMs uses those primarily to host telephone appliances (a high two-digit count at this point). Those serve everything from a simple doctor's office with a single receptionist to call centers with several hundred employees. The appliance is Linux-based (used to be CentOS 6, new version is CentOS 8 and got them in trouble thanks to IBM's coup). He got a dedicated Proxmox setup from us and while he liked the web UI, there were problems with the machines all the time.

Eventually he switched over to Bhyve as an experiment and this works well for a couple of years now. So well in fact, that he was spared of some of the worst fallout that hit many others and that even the vendor took a look at it earlier this year.

And that's only one success story. Bhyve is a great piece of tech. I wasn't too surprised when I heard that the illumos people had invested the time to port it over despite having had KVM support for ages. Not only choosing Bhyve over KVM after a careful evaluation but doing so despite they already had KVM working and battle-tested is something that speaks for itself, I think. (Which is not to say that Bhyve is by any means perfact or the right solution in every case. DragonFly BSD for example chose NetBSD's NVMM over porting Bhyve and they certainly have their reasons for it, too.)

The VM startup time is also not very relevant, what matters in a hypervisor is throughput and scalability. bhyve is not even in the same game as KVM.

This is a generalization that is plain wrong. While it's of course possible to use virtualization for scaling and also in cases where high throughput is necessary, there are also cases where you need to minimize downtime. Do you remember the amount of praise that systemd received for massively cutting down boot times? There were people who argued that this was less important with servers - but in the end nobody enjoys waiting longer than necessary. And if a hypervisor can speed up the time it takes until the system is ready to serve, that's a plus.

Yes, furthermore BSD developer explains the results:

Err... Ok, did you even try to understand what the developer stated there or did you just rush to get the next best quote that seemed to support your story? He says that there's "a lot of work going on in HEAD" regarding e.g. NUMA support. This indeed was a thing - a couple of years ago. So I looked up if I could find your source. It's 4 (!) years old now. It's hilarious that you either don't have a clue what 4 years of work mean in the IT or decided not to care because... it's those damn BaStarDs who still don't understand the superiority of Linux, right?

And most of NginX is running on Linux just like Apache, so what's your point? No, I haven't, because Linux performance is superior to FreeBSD:

https://matteocroce.medium.com/linux...g-cbadcdb15ddd

Wrong again. That's about networking performance (and there's quite a bit that would have to be said about that article, so let's at this point assume that it's both correct and complete). Strange answer to somebody who talked how Nginx was performing differently on both platforms.

Even netflix storage runs on Linux

Riiight. That's why they invest in UFS improvements among other things, I guess? But maybe you have some newer information than I have and Netflix has since switched to Linux (theoretically possible since they are running a heterogeneous landscape including Linux machines, anyway)? Mine's from late February this year, where I remember them talking about it at a conference. Looked it up, see here:

https://www.usenix.org/sites/default...des_looney.pdf

Where's yours?

[Classical]

Regardless of which operating system is faster for a particular workload, your sources are old.
5-6 years, 9 in the case of nginx, are a long in computer science.
You can't expect to be taken seriously if you try to prove your point with information that old

Indeed, the page comparing Epoll to Kqueu is already dated. The youtube video comparing NginX performance on FreeBSD and CentOS is from March 2020. This gives me the impression that the old article still applies. There are also clues to the fact that FreeBSD still has the fastest network stack in 2021. In addition to the recent article about the network (UDP & TCP) performance comparison of Ubuntu and FreeBSD there is also this recent article for example: https://people.freebsd.org/~gallatin/talks/euro2021.pdf With regard to the comparison of the audio stack of FreeBSD and Linux, that info also seems to be still current: https://meka.rs/blog/2021/10/12/freebsd-audio/

Finally, I would like to share several anecdotes. I have an Intel i3 dual core that is almost 10 years old. I also have a rather weak gpu and my ram is only 4GB at 1600 MHz DDR3. The best part of my PC is probably the EVO 850 500GB SSD. I updated my oldest sister's 1 year old MacBook Pro to Monterey two weeks ago. And then I was able to work with macOS for a few hours. What I noticed was that most basic apps opened (significantly) slower than on my FreeBSD desktop. Although the MacBook is many times more expensive than my FreeBSD desktop. In the end, using light Xfce apps on FreeBSD can get you very far in terms of daily performance.

Before using FreeBSD on my desktop, I used several popular Linux systems on the same hardware for years. So I can make a good comparison in terms of system stability and performance. GIMP opens on FreeBSD in 4 seconds, and eg the game 0 A.D. opens in 2 or 3 seconds. This has never been so fast on the Linux systems. And finally, there's also my experience with Dota 2 on FreeBSD. I can make reliable statements about it. In terms of game speed when browsing menus and fps, and in terms of connection stability with the steam servers, FreeBSD was more stable and faster in my tests than both Linux and windows10 had ever been.

I've seen many ignorants spreading this misinformation and no matter how many times its corrected they just keep coming back.
First of all, most of AI is run on domain specific hardware that don't accept general purpose programming languages, which means that there's a library that compiles special code optimized for that hardware. Second, Python is only a front end to lower level libraries and doesn't actually do any heavy lifting by itself, it's all done in complied c/c++ library. Python being slow or fast have marginal effect on performance.

Even TensorFlow's code is 25% python. Do you think none of this 25% does heavy lifting?
Python is the dominant language in Machine Learning and AI, see for example how much Python is used in these types of projects: https://github.com/commaai/openpilot

And DevOps has the same problems: https://github.com/ansible/ansible

In fact, lately I see more and more large projects that are close to 100% Python code, both in AI, Data Science and DevOps projects. In fact, I have a strong impression that computer science students get almost all their lessons in Python and often aren't very good at any other programming language.

In this way, you nihilate about two decades of hardware innovations in the field of energy efficiency.

[grahamperrin]

Zstandard compression on FreeBSD

As has been discussed already, the Zstd tests for this comparison were the system/zstd test profile making use of the zstd binary shipped by the OS itself, not compiled by PTS.

Thanks.

For what it's worth, I gained superior results with FreeBSD 14.0-CURRENT on vastly inferior hardware; linked from the foot of the opening post at <https://forums.freebsd.org/threads/83311/>. tl;dr 27.4 compared to 20.3 from the article.

(27.4 is still astonishingly slow compared to the other OSes; see below.)

… reported by mjg@…

Better to not give a person's e-mail address. Instead, maybe:

<https://github.com/mjguzik> | <https://people.freebsd.org/~mjg/>

on feeebsd-current@, …

From <https://lists.freebsd.org/archives/f...er/001183.html> (not using his address @freebsd.org), with added emphasis:

… the low number is computed from supposed total time spent on CPU. Compiling by hand gives c11 primitives to do it, while using the FreeBSD source tree lands with c90 which end up giving bogus results.

A hack which I can't be bothered to productize pasted below.

I can't easily repeat the test with patched zstd on the same box, but on another one this goes from supposed ~3.3MB/s to 70.2MB/s, …

[Volta]

I wonder what FreeBSD does to make it create threads so much faster than anything else here, are their spectre mitigations applied in a different manner perhaps? Just speculating since those mitigations took a hard hit for context switches.

Also strange how it changed behaviour for zstd compression when the level was decreased.

It's strange, because Linux always shined when comes to thread creation performance. I noticed the lowlatency kernel has lower result than generic. I would expect something opposite, but I did tests without system load.

[Volta]

I didn't change rhetoric. I nuanced which performance I was talking about. Start and stop times are definitely in the 'performance' segment. And KVM is not competitive in this area. In terms of memory usage, the reddit link you provided shows that KVM uses more than twice as much memory as bhyve for the same task.

Startup times and performance are two different segments and bhyve is not competitive in term of performance. Even if KVM uses more memory for single VM it will use much less memory for many VM's. It's because Linux' KSM.

If you had read my link from redbyte, you would have understood that read+write is the most realistic scenario for most databases. Linux is indeed faster in a 'read only' scenario, but this is less important because most databases are configured in read+write mode. The phoronix link you provide is indeed about this 'read only' mode. But FreeBSD is 2x faster than Linux in PostgreSQL in read+write mode.

The link you provided is useless, because the one that made the benchmark had no clue about configuring Linux for this workload. Even BSD developer debunked your 'benchmark', but you ignored this. The same applies to nginx.

FreeBSD's network stack has always been faster than Linux's. Facebook has tried to make Linux more competitive in this area, but their efforts have had little effect. Even phoronix has admitted that FreeBSD has the fastest network stack: https://www.phoronix.com/scan.php?pa...sd-linux&num=3

Not a chance. Linux eats it for breakfast as shown in the link I provided. Furthermore, here are some very old benchmarks when Linux was young, but it didn't stop it from making BSD's to look like a toys:

http://bulk.fefe.de/scalability/

Linux 2.6 scales O(1) in all benchmarks. Words fail me on how impressive this is. If you are using Linux 2.4 right now, switch to Linux 2.6 now!

FreeBSD 5.1 has very impressive performance and scalability. I foolishly assumed all BSDs to play in the same league performance-wise, because they all share a lot of code and can incorporate each other's code freely. I was wrong. FreeBSD has by far the best performance of the BSDs and it comes close to Linux 2.6. If you run another BSD on x86, you should switch to FreeBSD!

******************

FreeBSD is faster than Linux in the two most popular web servers and thus is the best operating system for the cloud (FreeBSD also has a more stable implementation of ZFS). In the case of NginX (the best web server out there), FreeBSD is 4 to 5 times faster.

Your data is very deprecated and debunked by the benchmark I showed before. Furthermore, CentOS isn't the latest and greatest Linux distribution. When comes to NginX (which was not so optimized for Linux in the past) its performance can be boosted even up to nine times. It's thanks to thread pools. And when comes to default NginX (thus your meaningless benchmark) configuration:

By default, multi-threading is disabled, it should be enabled with the --with-threads configuration parameter. Currently, multi-threading is compatible only with the epoll, kqueue, and eventport methods. Multi-threaded sending of files is only supported on Linux.

******************

Since FreeBSD 1) is usually slightly more secure than Linux 2) has the highest performance in NginX and 3) has the most stable ZFS implementation, we can conclude that FreeBSD is unquestionably a higher-up operating system for the cloud compared to Linux.

FreeBSD doesn't come close to Linux when comes to security and networking performance. Linux own the cloud and where's FreeBSD? Nowhere. FreeBSD doesn't scale above two sockets, so it's a toy OS. I don't care about ZFS.

FreeBSD also has important advantages blah, blah, blah.

Oh, legacy OSS crap again. Wake me up when FreeBSD catches up in graphic stack and DE support. FreeBSD has no single advantage on desktop.

If you have any further questions about these topics, you can always ask me for a more in-depth explanation. I think it's also helpful to explain why Linux has been more popular than FreeBSD for years. Popularity of a topic usually has little or nothing to do with how good something is. For example, think of McDonald. It is the most popular fast food chain, but a survey found that their products have the lowest quality of literally all fast food chains tested.

Linux is better in every single case (including community). Linux is Linux while FreeBSD is Linux wannabe: 'stealing' graphic stack, desktop environments, file systems (this one from slowlaris), it's even emulating Linux, because it can't exist without Linux. You have no clue and I don't care about trollish opinions.

[Volta]

So far no links to PR brochures as "technical guides" and random out of context Torvalds quotes as "evidence", so not so sure.

It's slowlaris troll without a doubt. Out of contest articles and even ignoring BSD developer opinion about useless benchmark.

[JackLilhammers]

Indeed, the page comparing Epoll to Kqueu is already dated. The youtube video comparing NginX performance on FreeBSD and CentOS is from March 2020. This gives me the impression that the old article still applies. There are also clues to the fact that FreeBSD still has the fastest network stack in 2021. In addition to the recent article about the network (UDP & TCP) performance comparison of Ubuntu and FreeBSD there is also this recent article for example: https://people.freebsd.org/~gallatin/talks/euro2021.pdf With regard to the comparison of the audio stack of FreeBSD and Linux, that info also seems to be still current: https://meka.rs/blog/2021/10/12/freebsd-audio/

Again regardless of which system is faster/safer/bettere for which workload, I'd argue that the comparison from May 2020 is hardly relevant.
No indication of the test conditions. No data. IOW the test is not verifiable nor repeatable.

The other 2 links have no comparisons.
I'm happy that Netflix runs part of its infrastructure on FreeBSD, but that's just a statement. Linux is not even mentioned in those slides.
As I'm happy for that guy that can do anything on FreeBSD. Apart from ZFS better support, anything said about Linux refers to 2015-2016 and should be checked before saying it still applies.

BTW, ZFS was also one of the main reasons for using bhyve in the other video you posted.

Finally, I would like to share several anecdotes.

I'm sorry, but anecdotes are almost worthless.

Even TensorFlow's code is 25% python. Do you think none of this 25% does heavy lifting?
Python is the dominant language in Machine Learning and AI, see for example how much Python is used in these types of projects: https://github.com/commaai/openpilot

And DevOps has the same problems: https://github.com/ansible/ansible

In fact, lately I see more and more large projects that are close to 100% Python code, both in AI, Data Science and DevOps projects. In fact, I have a strong impression that computer science students get almost all their lessons in Python and often aren't very good at any other programming language.

In this way, you nihilate about two decades of hardware innovations in the field of energy efficiency.

This last paragraph is so wrong that I have a strong impression that you don't know what you're talking about.

Percentage of python code in a project means nothing.
BTW there are tools to make pythonic code do the heavy lifting without leaving performance on the table.
Maybe the next time take a look at

• Cython
• Numba
• Nuitka

Last but not least, students don't have to know many programming languages, but must be eager to learn a new one when they need to

[Classical]

Not a chance. Linux eats it for breakfast as shown in the link I provided. Furthermore, here are some very old benchmarks when Linux was young, but it didn't stop it from making BSD's to look like a toys.

Many of the older benchmarks prove that 10 years ago Linux was mostly slower than FreeBSD in almost all domains. What I read recently is that Linux has generally not become faster than 10 years ago, in a phoronix article I read that. What do you think this means? You say it's sad that FreeBSD has to emulate Linux for certain things. But is it not sad that FreeBSD is often faster even though it emulates Linux, eg for gaming in wine this used to be the case.

And many Python developers complain that it is difficult to let the code run fast. Maybe its not a good programming language for AI ?

[F.Ultra]

It's strange, because Linux always shined when comes to thread creation performance. I noticed the lowlatency kernel has lower result than generic. I would expect something opposite, but I did tests without system load.

Yes Linux have always been able to even start processes faster than Windows and Solaris could start threads. But I don't know if anyone have compared it with FreeBSD before. This should in particular be quite slowed down by the spectre mitigations which is why I wonder if FreeBSD is doing something different there or if FreeBSD simply does way less things when launching threads (whatever that now might be).