Announcement

Collapse
No announcement yet.

Autonomously Finding Performance Regressions In The Linux Kernel

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • #11
    Wow, this is amazing work. I wasn't paying much attention to PTS thinking big deal, so it is cool, but is it really important? But now, HELL YEAH!

    Just the other day I was remembering something I read and thinking, while Windows and Mac has proper regression tests, who does proper regression tests in Linux? Well, step up to the plate PTS, at least for performance regressions. Yaaay!! PTS FTW.

    Comment


    • #12
      Very cool. I'm not working on any projects that would benefit from this directly, but I can see using this to help when I want to report bugs regarding performance regressions in mesa (or other projects).

      Comment


      • #13
        Originally posted by mirza View Post
        Wait, this means that if I want to have consistent data on ext4 I must look at 5x slower PostgreSQL and other write-intensive applications? And proposed solution is either obscure mount switch with possibility of occasional file corruption or really _slow_ PostgreSQL. WTF?
        What are you talking about? Afaik slower mode - ordered is used science 2.6.31. In 2.6.32 they're replacing some I/O related parts and maybe that's why there's such slowdown. Just wait for stable release. You've got to be kidding if you consider Ext3 is faster - afaik in some previous benchmarks Ext3 was using writeback mode (with possibility of occasional file corruption) while Ext4 was using ordered mode (really_slow_PostgreSQL, but I wonder if this is a meaningful test).

        Comment


        • #14
          XFS rocks!

          Comment


          • #15
            Originally posted by bugmenot View Post
            Just the other day I was remembering something I read and thinking, while Windows and Mac has proper regression tests, who does proper regression tests in Linux? Well, step up to the plate PTS, at least for performance regressions. Yaaay!! PTS FTW.
            Could you expand on your "proper regression tests" for Win and Mac?

            Bisection is extremely powerful, but unfortunately, most people don't really know or understand it.

            Now of course to blow you mind, IMHO, *any* unexpected change (beneficial or detremintal) are regressions. If you get an extra 20% in performance when you weren't expecting it, there is either something bad happening elsewhere - or - you hit some good Juju accidently, and you should really understand why and get some more.

            Regards,

            Matthew

            Comment


            • #16
              Can't speak for OS X, but Windows' idea of a regression test is RTM.

              Comment


              • #17
                mtippet: they have a few halls of machines running unit tests every (week|day|?). i guess you could call that regression testing.


                on the topic; please read this and the three weblinks as well:
                http://en.wikipedia.org/wiki/Delta_Debugging

                Comment


                • #18
                  Originally posted by Apopas View Post
                  XFS rocks!
                  yeah but only if you're not running anything exotic underneath (e.g. encrypted home with LUKS, ...)

                  that leads with 100% certainty to system corruption / no return -> new installation

                  and I can't afford to install my system from scratch only because of an occasional hardlock (fglrx ) and the filesystem which means that it has to zero all files or much files out because of security concerns


                  @mtippett:

                  I just saw that catalyst 9.10 will get out during the next few hours (if it hasn't already): my box always hardlocks if I do the following

                  Code:
                  sudo su (
                  or su)

                  then
                  Code:
                  time sync && reboot
                  or

                  Code:
                  time sync && shutdown
                  if I forgot to enter a command before wanting to reboot I e.g. would enter
                  Code:
                  df -h
                  ,
                  Code:
                  genlop -t
                  ... (just before it switches to terminal)

                  for omitting having to rebooting into my system again and it hardlocks

                  -> no magic sysrq key, etc. ...

                  this didn't happen with nvidia's or the opensource radeon driver (afaik) so I don't know why fglrx is insisting on complete "silence" while shutting down

                  hope you understand what I mean

                  thanks


                  @on topic:

                  nice ! so in the future the kernel devs can set up some test boxen to continuously check kernels for regressions and subsequently bisecting it automagically
                  Last edited by kernelOfTruth; 10-22-2009, 03:30 PM.

                  Comment


                  • #19
                    Awesome feature! Congratulations, Michael!

                    Comment


                    • #20
                      Originally posted by frische View Post
                      mtippet: they have a few halls of machines running unit tests every (week|day|?). i guess you could call that regression testing.
                      Okay. I thought you meant something public that end-developers could use. Regression farms, yeah, most good companies will have that, independent of OS.

                      on the topic; please read this and the three weblinks as well:
                      http://en.wikipedia.org/wiki/Delta_Debugging
                      With continuous integration or at least the ability to simulate it after the fact, this appears as though it is a good unclean regression mechanism. Have you experience in it?

                      Regards,

                      Matthew

                      Comment

                      Working...
                      X