Announcement

Collapse
No announcement yet.

Linux 2.6.24 Through Linux 2.6.33 Benchmarks

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Linux 2.6.24 Through Linux 2.6.33 Benchmarks

    Phoronix: Linux 2.6.24 Through Linux 2.6.33 Benchmarks

    At Phoronix we have been benchmarking the Linux kernel on a daily basis using Phoromatic Tracker, a sub-component of Phoromatic and the Phoronix Test Suite. We launched our first system in the Linux kernel testing farm just prior to the Linux 2.6.33 kernel development cycle and found a number of notable regressions during the past three months. Now with the Linux 2.6.34 kernel development cycle getting into swing, we have added an additional two systems to our daily kernel benchmarking farm. One of the systems is an Atom Z520 system but what makes it more interesting is that the system is using a Btrfs file-system and then the second new system added to the kernel tracker is a 64-bit setup. However, to provide a historical look at the Linux kernel performance, we have ran some fresh benchmarks going back to the Linux 2.6.24 kernel and ending with the recently released Linux 2.6.33 kernel.

    Phoronix, Linux Hardware Reviews, Linux hardware benchmarks, Linux server benchmarks, Linux benchmarking, Desktop Linux, Linux performance, Open Source graphics, Linux How To, Ubuntu benchmarks, Ubuntu hardware, Phoronix Test Suite

  • #2
    I can't believe such big regressions like apache where left untouched and unresolved during so much release. I don't understand either why the PostgreSQL performance regression passed through the rc phase... Either you just prove to the worl the kernel development model is flawed, either their is a flaw in the tests you ran. In both case, some analysis and explanation would have been nice along the results.

    Comment


    • #3
      Originally posted by Xheyther View Post
      I can't believe such big regressions like apache where left untouched and unresolved during so much release. I don't understand either why the PostgreSQL performance regression passed through the rc phase... Either you just prove to the worl the kernel development model is flawed, either their is a flaw in the tests you ran. In both case, some analysis and explanation would have been nice along the results.
      Usually those tests are meaningless. If there's intentional change in the kernel it's described here like regression. In example some file system has different mode set as default in newer kernel and this can make big impact on some benchmarks - PostgreSQL etc. Apache benchmark is meaningless, because it's not done properly.

      Comment


      • #4
        Originally posted by kraftman View Post
        Apache benchmark is meaningless, because it's not done properly.


        Be careful not to run ab on the same machine as you run apache, otherwise
        the numerous apache processes can limit ab's throughput. This is the same
        reason as why I educate people so that they don't run a single-process
        proxy in front of a multi-process/multi-thread web server. Apparently
        it's not obvious to everyone.

        Comment


        • #5
          I like this article a lot!! testing from 24-33, When I first read the Phoromatic Tracker i was wondering if that could be done. Phoromatic Tracker is a superb tool! well done.

          I always thought that there will be regression on performance from 24 to 33, because kernel is implementing more APIs, graphics code, graphics focus... but this article shows me that i was worng.

          Comment


          • #6
            I was very excited about PTS at the beginning.
            Now it produces meaningless numbers. your conclusions based on the PTS results could be wrong.
            Like with HD test. It's results leading to very wrong assumption. 20$ CPU can not play HD movie with high bit rate.
            I can't find testing file systems with different settings producing something useful.
            You can use some tests to evaluate some components , but i can't see the point to post this 24-33 tests. they are at least misleading.

            Comment


            • #7
              Originally posted by Jimbo View Post
              I always thought that there will be regression on performance from 24 to 33, because kernel is implementing more APIs, graphics code, graphics focus... but this article shows me that i was worng.
              This article mainly shows that tests are often meaningless.

              Comment


              • #8
                Real-world feedback

                It would be nice to hear heavy users/developers of PostgreSQL to chime in on these numbers. There must be businesses around to which performance of an order of magnitude less would mean disaster.

                OR is this just another case of Phoronix relying on the "default" Ext3 settings for each kernel and not bothering to compare the same settings across kernels to generate FUD and page hits?

                Comment


                • #9
                  I'm interested in the special commit that made PostSQL run so much faster and slower again. And I also miss the feature you described a few weeks ago, that you can see the deviation of every single run if you move the mouse over the graph.

                  All in all interesting I think, thank you

                  Comment


                  • #10
                    Skip to the next response to that thread..

                    I turned on apache, and played with ab a bit, and yup, ab is a hog, so
                    any fairness hurts it a badly. Ergo, running ab on the same box as
                    apache suffers with CFS when NEW_FAIR_SLEEPERS are turned on. Issuing
                    ab bandwidth to match it's 1:N pig nature brings throughput right back.




                    Remember that you can't test anything, and testing in the obvious path will usually result in flat lines - since they represent the 95% path.

                    As indicated above, what has been identified is that in some scenarios CFS completely tanks. The ab is just a tool to make this visible.

                    As per usual, if there is any benchmark which you believes provides a suitable equivalent scenario but is more "correct", please tell us.

                    Comment

                    Working...
                    X