Five Years Of Linux Kernel Benchmarks: 2.6.12 Through 2.6.37

[TemplarGR]

Almost all old kernels would exist in virtual machines in corporate environments these days. The expensive Linux application that you bought or got built 10 years ago that is only supporting and working on Red Hat 7.1 won't be running on the same hardware. That original hardware would have expired a few years after deployment. Most corporate environments then will move those installations to a VM.

Particularly for the older kernels, it's closer to a real production scenario then most would believe.

What you say may be true, but doesn't justify the methodology.

1)As i said, most kernel optimizations are already there, since host's kernel is 2.6.35. It's the host's kernel that is managing the metal, not the guest's. These tests do not show for example the difference of having a host with 2.6.12 kernel and having a host with 2.6.35 kernel.

2)The userland is too old(since from what i read you used Fedora 4 with all the kernels). For example the graphic stack should be vastly improved. Libraries are old etc.

3) While VM's are important for businesses. and what you have tested is indeed a usual use case senario, this senario doesn't correctly discribe kernel's evolution in general. What this article says is that it tries to measure kernel's performance through time, in *general*. It should be on real hardware, and if possible using appropriate userspace software for each kernel. I recognize this is difficult and time consuming, but it is the right way to do it.

[Guest]

and if possible using appropriate userspace software for each kernel

No. Not in _kernel_ test.

If you want to test kernel, you should not change userspace at all.

It's difficult, because old kernels doesn't build on modern systems - you even may not be able to boot 2.6.27 (still supported) on latest distros.

[igf1]

Interesting that also, Intel corp, has confirmed that Linux is getting slower and slower. Maybe this is because Linux is getting more and more bloated?

http://www.theregister.co.uk/2009/09/22/linus_torvalds_linux_bloated_huge/

"Citing an internal Intel study that tracked kernel releases, Bottomley said Linux performance had dropped about two per centage points at every release, for a cumulative drop of about 12 per cent over the last ten releases. "Is this a problem?" he asked.

"We're getting bloated and huge. Yes, it's a problem," said Linux Torvalds."



Maybe Linux should focus one release on bug fixes instead of introducing new functionality all the time? Just like Apple did. One of the recent big Mac OX S releases was devoted to only bug fixes and slimming down OS X. Which paid off.

I conducted a study which concluded the same thing, .24 - .30 was a downward spiral, that, apparently has been addressed to some degree judging by the data here. However, "bloat" is an artifact of non-modular development. The goal is keep the kernel "pure" and allow anyone to bloat their distribution as they see fit.

[V!NCENT]

So what? It was benchmarket in a thread... That's hella better than doing it on real hardware where performance may be hindered by driver implementations...

A thread runs consistant. Again... the problem? Multithreading? Errr... boohoo!

[Guest]

So what? It was benchmarket in a thread... That's hella better than doing it on real hardware where performance may be hindered by driver implementations...

A thread runs consistant. Again... the problem? Multithreading? Errr... boohoo!

In your test you are benchmarking:
- hardware
- host kernel space
- host user space
- virtualization platform
- guest kernel space
- guest user space

I don't give a flying ship about results. You can do what you want in your free time. Just don't call these benchmarks meaningful

[V!NCENT]

You test the kernel for global regressions and you not the performance of HW configuration X.

Or was this not about regression testing? Care to explain why my post was meaningless? Because if it's not then yours is.

[Guest]

You test the kernel for global regressions and you not the performance of HW configuration X.

Or was this not about regression testing? Care to explain why my post was meaningless? Because if it's not then yours is.

Just read about kernel benchmarks methodologies.

[V!NCENT]

Just read about kernel benchmarks methodologies.

I genuinly wanted to do that, but I could not find one that could also work on watches, TiVo, TomTom, 386, Android and elevators. Must be because they can't possibly do virtualization even though they are all running the Linux kernel...

[TemplarGR]

No. Not in _kernel_ test.

If you want to test kernel, you should not change userspace at all.

It's difficult, because old kernels doesn't build on modern systems - you even may not be able to boot 2.6.27 (still supported) on latest distros.

A large part of kernel functionality is accessed using userspace libraries. If those libraries are old, some features cannot be used or you lose some improvements of later versions.

[TemplarGR]

So what? It was benchmarket in a thread... That's hella better than doing it on real hardware where performance may be hindered by driver implementations...

A thread runs consistant. Again... the problem? Multithreading? Errr... boohoo!

Again you do not seem to know what you are talking about(as usual).

If that was the case, why not just use Windows as a host and Linux only as guest? After all, most driver implementations (for pc systems) are more complete and/or fast on Windows...

In reality, when you benchmark a virtual machine, you largely benchamark the virtual machine implementation and its host performance. That is why in most of these tests performance is similar for all versions of the kernel. Just have a look at the graphs again...

What is funny is that from what i recall, older kernel benchmarks here on Phoronix revealed larger variations on a nearer range of kernel versions, when performed on native hardware...

I propose adding a page on this article, testing a native 2005 era machine using Fedora Core 4 with its stock kernel, and later compiling a recent kernel. I believe it will prove my point...