FreeBSD 8.0 vs. Ubuntu 9.10 Benchmarks

[krazy]

wrong and wrong.

Ubuntu has debug turned on

Ubuntu was also compared to a RELEASE. So the comparism is valid. Even if you take 8 RC out of the picture Ubuntu still kicked 7.2 RELEASE

differing GCC versions make any kind of "ubuntu > freebsd" conclusion from this test impossible anyway.

that wasn't the point of the test though, which was comparing "out-of-the-box" experience.

[risner]

It is the maintainers who are making the tuning decisions, consequently the benchmark approach of "out of the box" is still valid and still valuable.

I appreciate your point (it is FreeBSD's fault they benchmark slow by leaving debugging defaults on), but I guess I don't know what to do about it. To me, it is very well documented how to build your own system kernel, how to compile the system with optimize options, how to build the system with newer GCC.

It may be that you feel those actions are not "in your face" enough that you found them?

The FreeBSD handbook goes through these steps, but for your convenience I will restate a few here:

1) Build a new kernel, the GENERIC kenrel isn't intended to be used in production systems. A new kernel will remove/disable debugging, improve security (unused kernel features), require less memory (unneeded drivers), and more.

2) Rebuild world with your CPU type to take advantage of optimization based on CPU type.

3) Rebuild ports/world with GCC 4.3 or 4.4 to take advantage of SSE3/SSE4 CPU features prohibited due to restrictive licensing imposed by GCC. GCC starting with 4.2.2 is under GPL3, which is an incompatible license with BSD license. So they can not legally ship GCC 4.2.2 or up (which is why you see 7.2 and 8.0 using GCC 4.2.1 compiler.)

Each of those actions (#1 and #2 are assumed all users will perform, #3 is optional) will significantly improve performance. If you have trouble performing these options, I can help. PM me.

So, there were a bunch of messages (including from myself) about this, without actually checking it. I just installed FreeBSD 8.0RC1 in Qemu, and AFAICT the debugging options aren't enabled.
Therefore, I don't expect big changes in the RELEASE, unless the benchmarking will be done by using a newer GCC version (8.0 has version 4.2.1, although a newer version can be used from ports).

8.0 RC1 (and 7.2) ship with the kernel compiled with debugging:
/usr/src/sys/i386/conf% grep DEBU GENERIC
makeoptions DEBUG=-g # Build kernel with gdb(1) debug symbols
options IEEE80211_DEBUG # enable debug msgs

[Zhick]

Rebuild ports/world with GCC 4.3 or 4.4 to take advantage of SSE3/SSE4 CPU features prohibited due to restrictive licensing imposed by GCC. GCC starting with 4.2.2 is under GPL3, which is an incompatible license with BSD license. So they can not legally ship GCC 4.2.2 or up (which is why you see 7.2 and 8.0 using GCC 4.2.1 compiler.)

I didn't do any research on this, so sorry if this question is stupid, but how is GPLv3 anymore incompatible with the BSD-license than GPLv2? The GPL has always been more restrictive and thus BSD-incompatible, but that didn't stop them from shipping GPLed software before now, has it?

[risner]

I didn't do any research on this, so sorry if this question is stupid, but how is GPLv3 anymore incompatible with the BSD-license than GPLv2? The GPL has always been more restrictive and thus BSD-incompatible, but that didn't stop them from shipping GPLed software before now, has it?

I am not a licensing expert so I can only comment on the situation as I remember it when the decision to stop at GCC 4.2.1 was made by the FreeBSD developers. At the time it was said to be a legal reason and not a "political" or emotional reason.

What I do know is that the BSD license is significantly more permissive than the GPL license, particularly the GPLv3 license. There are a number of licenses incompatible with GPLv3 including Apache 1-2.0, Mozilla, Original BSD, XFree86 1.1 and GPLv3 (if not for the usual "code can be upgraded to future GPL licenses" line in most GPLv2 versions.)

Did a little googling, I think this is the problem:

http://www.fsf.org/licensing/licenses/gcc-exception-faq.html

Where they now (with GCC 4.2.2) consider OUTPUT of the GCC compiler GPLv3 licensed material. That would prevent someone from using GCC 4.2.2 and up to compile anything and release the compiled application without source code for the whole system (because Target Code is now forced to be GPLv3 code) being released.

[gongoputch]

Of course you are aware that 8.0 doesn't have the debugging symbols out of the kernel even, let alone libc, malloc, etc. yet. Why not set up a more real world benchmark like a MySQL or Postgres ACID test, or a tpc benchmark. Zipping files will generate a mostly sequential read and/or write, not really where filesystems tend to fall down. Did the compile tests use concurrency?
Nice graphs tho.
If you wanted to talk to some seasoned BSD admins about setting up benchmarks, try me in ##freebsd on freenode.

[nanonyme]

Did a little googling, I think this is the problem:

http://www.fsf.org/licensing/licenses/gcc-exception-faq.html

Where they now (with GCC 4.2.2) consider OUTPUT of the GCC compiler GPLv3 licensed material. That would prevent someone from using GCC 4.2.2 and up to compile anything and release the compiled application without source code for the whole system (because Target Code is now forced to be GPLv3 code) being released.

I dare you quote the part of the page that actually in your opinion says that.
Important parts of the page: "Our fundamental policy has not changed; the new license is meant to permit all the uses of GCC that were permitted before." They are merely afraid of loopholes in licenses that would allow someone to use "GCC's internal, low-level compilation data structures" and being allowed to integrate those components in some other compiler under a different (GPL-incompatible) license.

[risner]

I dare you quote the part of the page that actually in your opinion says that.

I can't help that FreeBSD's attorneys advised against switching to GPLv3 due to the wording of that page and/or the license. Are you a lawyer? Can you indemdify users of GCC 4.2.2 based on their statement that nothing has changed, when the other parts of the page directly suggests otherwise?

Basically, you shot the messenger. I didn't make the decision, I just posted about it. This whole debacle isn't a FreeBSD specific problem. Every single BSD with the exception of DragonFlyBSD have opted to not upgrade past 4.2.1 GCC.

[kraftman]

Can't verify the debug options myself.

What I heard is that some debug things were turned on for the BSD kernel and not the Ubuntu kernel. If that's wrong, so BSD and Ubuntu had similar debug options turned on, then I suppose the benchmark is okay from that perspective.

To be honest, all these comments that look like they're coming from 2-year olds is that destroys this thread.

Anyway. Can't wait for the huge OS/distro comparison.

I just checked Ubuntu and there are some debugging option turned on, but probably very light

[Penti]

Sounds like a political issue that only they have to me.

[risner]

I just checked Ubuntu and there are some debugging option turned on, but probably very light

Curious, which options? Does it have "gcc -g" options?

The FreeBSD (all release and release candidate GENERIC kernels) have full debugging symbols (and code) "gcc -g" enabled.