Benchmarks Of FreeBSD 8.1 RC2
Phoronix: Benchmarks Of FreeBSD 8.1 RC2
FreeBSD 8.1 is slated to be released this month as the first significant update to FreeBSD since the rollout of the 8.0 release last November. With the second release candidate of FreeBSD 8.1 having just been made available a few days back, we have conducted a set of tests comparing the performance of FreeBSD 8.1 RC2 versus FreeBSD 8.0 and an Ubuntu 10.10 development snapshot.
UFS. Oh man, Michael, the BSD folks will never let you up over this...
GCC on FreeBSD 8.1
Some questions if you could please clarify:
1) Why did you used the GCC 4.3.4 version for the FreeBSD 8.1 benchmarks...?
2) ...vs GCC 4.2.1 for the FreeBSD 8.0 benchmarks?
2) Is GCC 4.2.1 not the default compiler on FreeBSD 8.1?
3) Was the kernel built using that GCC version...?
4) ...or just the user-space programs?
5) Did UFS have soft-updates enabled?
6) Are you going to compare the "development" version of FreeBSD (aka -current) as you compare against the development version of Ubuntu?
7) Did you do any single threaded IO tests?
8) Could you use a single GCC version for all tests (i.e. to compare kernel performance)?
Thank you for the comparison. I found it an interesting read.
P.S. the latest geom_sched work should greatly improve FreeBSD's threaded IO performance. It is in FreeBSD 8.1 but not enabled by default. See http://svnweb.freebsd.org/viewvc/bas...ME?view=markup for details.
Cue BSD aplogists in 3...2...1...
I predict the flow will go like this.
BSD: Hey, no fair, it was a RC2, we had eleventy billion debug options turned on.
Linux: Not our fault. Note that 8.0 had nearly the same numbers and was still beaten by Linux is almost every test.
BSD: That's our old version, 8.1 is 100x faster without debugging on and is more secure, your benchmark doesn't show how secure BSD is.
Ubuntu has debugging turned on. The main thing which makes a difference is probably a different GCC used in both systems.
First I would like to point out that FreeBSD stable branches (i.e. 8) has debugging options "turned off". The RC should provide the same speed as the release (unless an important bug was identified).
Also, I am not aware of any benchmarks that can show how secure a system is. I do not think there is any doubt that OpenBSD is the leaders in security.
As a quick (and inaccurate) comparison of security I counted 13 security advisories from Novell for Linux  and 6 advisories from and for FreeBSD  (of which only 1 directly affected the kernel). This does not give an indication of root vulnerabilities though. I could not find a webpage where Linux published their security advisories.
Regarding the benchmarks, I would like to know is how much of the difference is due to compiler or kernel performance. This is important since each OS used a different compiler.
Some questions regarding the benchmarks:
1) Why was the base compiler (GCC 4.2.1) not used for the FreeBSD 8.1 tests?
2) Was the kernel compiled with GCC 4.3.4 on FreeBSD 8.1 (or just the user-space programs)?
3) Was soft-updates active on UFS?
4) Is it possible to use the same compiler across the board?
5) Is it possible to use the same compiler for all the kernels?
6) Have you done single threaded IO tests?
7) Any plans to test the "development" branch of FreeBSD (aka -current) as the development snapshot of Ubunutu was included?
8) Are there any plans to test ZFS?
FreeBSD is developing support for non-default compilers to be used. It should be fairly easy to compile it with GCC 4.4 (as used by Ubuntu).
A recent development in FreeBSD is GEOM_SCHED that should improve the performance of threaded IO. It is included in FreeBSD 8.1 but is not active by default. See http://svnweb.freebsd.org/viewvc/bas...ME?view=markup for further details.
Thank you for the benchmark, it was an interesting read.
Wait a minute. So GCC is still the system compiler for FreeBSD. What happened to all the big plans of eradicating all GPL code from the FreeBSD base system?
Originally Posted by phoronix
Well, unlike Linux which likes to make sweeping core API / ABI changes between minor releases FreeBSD tries to err on the side of stability and although 8.0 -> 8.1 will see some software version changes they're mostly bug fix changes with some additions / improvements, but nothing as sweeping as going from GCC to llvm/clang to build the core system. It's in the works and should be out with the 9.0 release.
Originally Posted by monraaf
More on topic of the article it would be interesting to find out how much of that performance is due to the file systems used. UFS is stable and rock solid, but performance has never been its forte. It's seems a lot of the CPU / memory bound tests it's pretty equal and only once the filesystem gets hit heavily does FreeBSD falter.
Personally, as an overall package FreeBSD is still a better system and it's eco-system is very developer friendly with solid documentation and a well defined set of "core" libraries.
and with 'Secure' you mean 'local root exploits are not a security problem and will not be patched'
Originally Posted by locovaca
and with 'better' you mean more open security holes, less performance, missing features and of course the 'it is better because we say so' philosophy.
Originally Posted by rhavenn