FreeBSD 8.0 vs. Ubuntu 9.10 Benchmarks

[kraftman]

Disk I/O is a classical example of BSD v. Linux difference of stability v. performance: BSD attempts to always write meta-data before writing file data, while Linux would happily keep dirty meta-data in cache.

I think it's not like this, but you've got to look somewhere at lkml. Ext4 didn't run in the "fastest" mode here. Btw. what's the point in writing meta-data first when power failure will occur?

[mtippett]

Dear Phoronix,

Please stop comparing desktop operating systems with server operating systems whose defaults include debugging, performance monitoring, fault tracing, and conservative compiler optimisations.

For desktop OS comparisons, please consider rather benching PC-BSD in future. It would be more informative, and if PC-BSD also under performs, the PC-BSD project is more likely to improve the situation and benefit from the tests.

FreeBSD users and developers don't care that the system is slower than Ubuntu as a desktop out of the box. In fact, many of us could have told you the same without your tests.

The ongoing religous debate that alwas goes on is "*BSD to Linux, which is faster?". For most users out there who have heard of this "BSD-thing", their first BSD they find is FreeBSD. Unless they know of Open/Net/PC/Free-BSD, that is about as deep as most people will go. As a consequence, although simplistic, it represents the answer to the question. It really does point to a marketing or fragmentation issue in the BSD marketspace.

Can I suggest that you

1) look over www.phoronix-test-suite.com,
2) construct test suites based on those tests that provide value for a PC-BSD vs Linux, FreeBSD vs Linux, etc comparison.
3) Do at least one run of each of those suites and upload it to global.phoronix-test-suite.com
4) communicate that run and it's results here.

That way you are providing the basis for improved testing.

I would suggest that you do compare and contrast PC-BSD/Linux/FreeBSD to ensure that your hypothesis is supported.

(Note that the order list of tasks is not intended to be belittling).

Regards,

Matthew

[kraftman]

Uhm, that's 2006, and FreeBSD 6.x. You clearly don't know much about FreeBSD.
FreeBSD went through huge changes in the design in 5.x, which made it actually slower than 4.x, AFAIR. But, since then it was improved constantly, with each release.

Just a note, Linux improves much faster.

[kraftman]

yea but you know, the free bsd people spreading der crap.

The painful truth. Even MS stopped doing this, but at freebsd.org you can read a lot of such bull.

[kraftman]

A regression is simple to define.

A regression is a change in behaviour that occurs between a configuration under test and a secondary related configuration under test. The regression can be positive or negative, but it is the change in behaviour that is important.

If there is a document somewhere about what has been turned on or off in development version and what will make it to the final production version, then I agree that it can be communicated. If one doesn't exist, then it is a fair comparison since removing it is a black art that isn't communicated to anyone.

Regards,

Matthew

They benchmarked distro using Ext3 in writeback mode and distro using Ext4 in ordered (or some other). Distro using Ext3 and writeback was faster in *SQL and distro using Ext4 and ordered was slower.

[risner]

blabla, and there they are, the FreeBSD apologists. Always claiming FreeBSD is faster.

Few are claiming FreeBSD is faster. Most of the comments I see here claim the test was poorly done (and fundamentally unfair.)

GCC 4.4 includes support for processor features like SSSE3, SSE4.1, SSE4.2, and more that are not present in GCC 4.2 (which is used by FreeBSD.)
Due to licensing issues, the newer GCC can't be used in the base OS.
The performer of this benchmark didn't say they compiled everything with GCC 4.4 for a apples to apples test.

The documentation for compiling things with GCC 4.4 are here:

http://www.freebsd.org/doc/en/articles/custom-gcc/article.html

If you want an honest test where it is proved Ubuntu is faster, you should perform a fair test. If you wish to "just show FreeBSD slower" that can be done by running the benchmark in an unfair way. Like was done by this test.

[mtippett]

They benchmarked distro using Ext3 in writeback mode and distro using Ext4 in ordered (or some other). Distro using Ext3 and writeback was faster in *SQL and distro using Ext4 and ordered was slower.

Yes that is clear. One thing that keeps come up again and again is that there is an assumption that most users will have sufficient knowledge or care enough to switch between filesystems or tweak between options.


In the same way, most Linux-heads abstract XP vs Vista vs Win7 as "fast", "dog-slow", "mostly better", they don't go down into that level of details. For a Linux head they would look at the top level "is it faster" and go with XP or wait for Win7 if they _had_ to use a Microsoft OS.

This is no different. If someone is looking at writing an application using something like SQLite, they generally either won't have complete access to the system (hosted server or end-user system). Consequently their view of supporting different OSes will be based on a high level

"yeah, Ubuntu-9.04 is faster, I'd recommend you my app on this",

rather than

"yeah, if you reconfigure the filesystem by modifying the kernel options with this string and changing your fstab entries, you will get good performance with Ubuntu 8-10."

If you have ever supported software, you know that you will *always* go for the first comment and push people away from 8-10. It is all about domain's of expertise and variables that you can control. Most people simply won't get down to modifying those options on a system. Their view outside _their_ domain of expertise is mostly a black box.

The key driver for Michael's performance reviews and comparisons is to drive awareness of system differences. It is bare and simple facts. It is the upstream maintainers who are making decisions on behalf of users. If they have done their jobs right, they are balancing the experience of the end user, the safety of the data and so on. Some users tune, the majority will not and inherit the maintainers view of how things should work. There are many cases of Michael's testing having beneficial results ultimately for the community when the numbers are groked and a correct decision made.

My standard response here is that if there is an accessible collection of directions for changing the configuration then you may have a strong argument for changing the testing. People who live and breathe filesystems assume that everybody knows how to tune the filesystem, but those who don't need to have their hands held very tightly.

Do you know of any accessible guides for tuning filesystems?

Matt

[kraftman]

Do you know of any accessible guides for tuning filesystems?

For newbies or for you? I don't know any. I don't even do it myself, because defaults are enough for me.

@Risner

GCC 4.4 includes support for processor features like SSSE3, SSE4.1, SSE4.2, and more that are not present in GCC 4.2 (which is used by FreeBSD.)

Aren't some flags have to be specified to use such features? And does opteron supports all of them? :>

[mtippett]

For newbies or for you? I don't know any. I don't even do it myself, because defaults are enough for me.

Accessible was used carefully . For FS tuning, I consider my self as newbie (with a deeper technical understanding).

The point is that

"defaults + accessible tuning guide" = argument for "fairer" comparison

where as

"defaults + tuning as a black art" ~~ "defaults"

for the majority of users.

I don't completely buy into "fairer" because "fairness" is a function of level of expertise, effort in configuration, particularly intent for the results . That is why Phoronix for the most part relies on the upstream's "Best Effort" of what they believe is production worthy.

(To be fair, I updated my reply slightly between our exchanges .

[risner]

Aren't some flags have to be specified to use such features? And does opteron supports all of them? :>

Yes -O is used to enable them (they are enabled by default if you use the default of O2) and yes AMD support those features:

http://www.amd.com/us-en/Processors/ProductInformation/0,,30_118_8796_8799,00.html?redir=SWAB01