Announcement

Collapse
No announcement yet.

Fenrus Linux: A Distro For Performance, Developers

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • #46
    Originally posted by fenrus View Post
    I've been thinking about this, how to fix it, and I haven't figured it out yet. In some way, "-50%" indicates "half as bad", and "+100%" is "twice as high". It also allows me to do math on the data points (the average of 25 fps and 75% fps is 50fps, and that math works also in -50% and +50%)

    the primary objective for the graphs (for me at least) is to see if there's a regression, or an improvement, compared to a previous series of builds.
    I do want to keep the graphs useful for that.
    fwiw this linearity is useful, including for the graphing software (R with ggplot2) to calculate and draw the std deviation bars

    Comment


    • #47
      Originally posted by fenrus View Post
      I've been thinking about this, how to fix it, and I haven't figured it out yet. In some way, "-50%" indicates "half as bad", and "+100%" is "twice as high". It also allows me to do math on the data points (the average of 25 fps and 75% fps is 50fps, and that math works also in -50% and +50%)

      the primary objective for the graphs (for me at least) is to see if there's a regression, or an improvement, compared to a previous series of builds.
      I do want to keep the graphs useful for that.
      This sounds sane to me. The sunspider benchmark does the same thing.

      Comment


      • #48
        hey fenrus will this remain your pet project or would you actually consider forming a small community and bring in more devs ?

        Comment


        • #49
          Originally posted by fenrus View Post
          I've been thinking about this, how to fix it, and I haven't figured it out yet. In some way, "-50%" indicates "half as bad", and "+100%" is "twice as high". It also allows me to do math on the data points (the average of 25 fps and 75% fps is 50fps, and that math works also in -50% and +50%)

          the primary objective for the graphs (for me at least) is to see if there's a regression, or an improvement, compared to a previous series of builds.
          I do want to keep the graphs useful for that.
          I appreciate your answer but if +100 points in a graph represent a 2/1 ratio (i.e., "twice") then -100 points would represent a 1/2 ratio (i.e., "one half"). If your distro is twice faster on test A, but the other distro is twice faster regarding test B then, on average, both distros have the same performance. This is the precisely the result obtained if you take the average of +100 and -100, which is zero. This graph is accurate and represent reality.

          However, taking the average on your misleading graphs we would obtain (100% - 50%)/2 = 25%, giving the false impression that your distro is a 25% faster on the average, when the truth is that both distros have the same average performance (i.e. your distro is 0% faster).

          The problem here is that you are mixing a percentage (i.e. a relative value) with a baseline (i.e., a reference for an absolute value).

          You do not need misleading graphs for seeing improvements or regressions. Your misleading graphs will be only giving a false appearance of performance in cases where there is none.
          Last edited by juanrga; 03-27-2013, 11:25 AM.

          Comment


          • #50
            Originally posted by fenrus View Post
            fwiw this linearity is useful, including for the graphing software (R with ggplot2) to calculate and draw the std deviation bars
            Not a valid argument, if data X is linear then subtracting a baseline will maintain that linearity: (X - b) is linear as well.

            Comment


            • #51
              Originally posted by juanrga View Post
              I appreciate your answer but if +100 points in a graph represent a 2/1 ratio (i.e., "twice") then -100 points would represent a 1/2 ratio (i.e., "one half"). If your distro is twice faster on test A, but the other distro is twice faster regarding test B then, on average, both distros have the same performance. This is the precisely the result obtained if you take the average of +100 and -100, which is zero. However, taking the average on your misleading graphs we obtain (100% - 50%)/2 = 25% giving the false impression that your distro is a 25% faster on the average, when the truth is that both distros have the same average performance (i.e. your distro is 0% faster).

              The problem here is that you are mixing a percentage (i.e. a relative value) with a baseline (i.e., a reference for an absolute value).

              You do not need misleading graphs for seeing improvements or regressions. Your misleading graphs will be only giving a false appearance of performance in cases where is none.
              Now I think you're confused. The average of "half as fast" and "twice as fast" is not "neutral" in any math that I can think of (before my morning coffee at least)

              Comment


              • #52
                Originally posted by Pallidus View Post
                hey fenrus will this remain your pet project or would you actually consider forming a small community and bring in more devs ?
                I'd love to get others to help out. Having said that, I don't quite know how that is going to work yet... I have a load of documentation to write on how to use, and then how to contribute, before that becomes realistic.
                (and writing docs isn't as interesting as integrating, say, gnome 3.8 as I was doing last night)

                Comment


                • #53
                  Originally posted by ryao View Post
                  This sounds sane to me. The sunspider benchmark does the same thing.
                  I find curious that you cite this one, when Apple has recently removed a claim from its website that they had "the world's fastest web browser", whereas Google showed that the results of this benchmark are misleading, and Mozilla developers are now claiming that benchmarks such as Sunspider "suck".

                  Comment


                  • #54
                    Originally posted by fenrus View Post
                    Now I think you're confused. The average of "half as fast" and "twice as fast" is not "neutral" in any math that I can think of (before my morning coffee at least)
                    Ok, I will wait you take your coffee and will explain this to you latter.

                    Comment


                    • #55
                      Originally posted by juanrga View Post
                      Ok, I will wait you take your coffee and will explain this to you latter.
                      Well, you could always do it in log scale.

                      Comment


                      • #56
                        Originally posted by fenrus View Post
                        Now I think you're confused. The average of "half as fast" and "twice as fast" is not "neutral" in any math that I can think of (before my morning coffee at least)
                        The average of 2 and 1/2 cannot be zero, because negative values are being excluded from that range. But negative values can be allowed. They are in my previous +-100 point example, which averages to zero ("neutral"). Your graphs also allow for negative values, but in a non-neutral way, giving a false impression of performance.

                        Consider this other example: three cities A, B, and C, whose distance between cities A and B is the same than between cities B and C. A red car travels at 40 km/h from city A to city B and at 20 km/h from city B to city C. Whereas a blue car travels at 20 km/h from city A to city B and at 40 km/h from city B to city C. What car is faster?

                        Using a methodology as yours, the red car is 100% faster than the blue car in the first span and -50% ("half as fast") in the second span. The average being 25% faster, which is not true.
                        Last edited by juanrga; 03-27-2013, 06:41 PM.

                        Comment

                        Working...
                        X