Announcement

Collapse
No announcement yet.

Intel vs. AMD Performance-Per-Watt On Ubuntu 14.04 Linux

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • #46
    Originally posted by baffledmollusc View Post

    Finally, (physicist here) I'd like to see two values plotted:

    Performance per Watt = benchmark result / (average power * time / time) = benchmark result / average power over task (units of per Watt)
    Energy needed per task = average power * time taken (units of Joules)

    Of course, the performance per Watt formula assumes higher is better, so for those benchmarks where lower is better you'd need to use 1/result or something for the performance.
    The first formula does not make sense. The benchmark result is in seconds, so you would be looking at units of seconds per Watt (not "per Watt"), and that is useless. If you change the benchmark result to be "1/result" then you would have units of (1/sec) / (Joule/sec) = 1 / Joule. But that would just be the reciprocal of "Energy needed per task", so it does not provide any new information over just the Energy required for the computation.

    I'm not sure why people keep suggesting all sorts of strange metrics. The only important metric here for power efficiency is the total energy required to complete the task. Of course, it is also useful to know how quickly the task is completed, and that is already provided by the "Time to Compile" benchmark.

    Bottom line is that this type of benchmark should include two values for each CPU: energy required to complete task, and time required to complete task.
    Last edited by jwilliams; 01-18-2014, 07:00 PM.

    Comment


    • #47
      Originally posted by baffledmollusc View Post
      (average power * time / time)
      really? :-D

      Comment


      • #48
        Originally posted by devius View Post
        I wouldn't hold my breath. This isn't the first time Michael posts obviously wrong performance-per-watt results, people complained, provided the correct formulas to use and yet here's another bad set of results. And I'm sure it's not the last either, so if you'd like to actually post the correct results I'm also interested in that
        Originally posted by baffledmollusc View Post
        I doubt it. I've never seen Michael fix an article, no matter how egregious the error.

        I actually came here to post a comment about the graph fail, but was gratified to see that everybody is already all over it.

        Seriously, how could those graphs in the article ever have been published? They're so obviously, obviously wrong. You don't even have to think - they leap out and slap you around the head with a wet fish.

        Finally, (physicist here) I'd like to see two values plotted:

        Performance per Watt = benchmark result / (average power * time / time) = benchmark result / average power over task (units of per Watt)
        Energy needed per task = average power * time taken (units of Joules)

        Of course, the performance per Watt formula assumes higher is better, so for those benchmarks where lower is better you'd need to use 1/result or something for the performance.
        That's unfortunate. I haven't been a reader for very long but I've regarded Phoronix as a professional Linux oriented site despite being run by only one person. Even people from industry seem to post in these forums (at least looking at their forum titles). So I thought article updates to fix mistakes would be done. I guess that's why I haven't seen titles with "(Update)" or similar in them like I do with other sites.

        I guess I'll go slap something in Octave because I'm interested in this article's subject for the new mini-ITX PC I plan on building. I haven't completely decided on Kaveri or Haswell yet.

        Comment


        • #49
          Originally posted by droste View Post
          really? :-D
          Yes, really.

          Say you get a benchmark score of 80 FLOPS, iterations, whatever. Say it took 30 seconds, and the CPU used an average power of 50W over that time. So

          Performance per Watt = benchmark result / (average power * time / time)

          = 80 FLOPS / (50W * 30s / 30s)
          = 80 FLOPS / 50W
          = 1.6 FLOPS / W
          = 1.6 FLOPS per Watt

          The reason I wrote it that way is average power * time = energy, and to get a power you have to divide by time taken.

          Although now I look at it, I'm prepared to agree that while correct, it's not the most straightforward approach. I think the issue is I always think in terms of energy, and convert that to whatever later.

          Comment


          • #50
            Originally posted by baffledmollusc View Post
            I doubt it. I've never seen Michael fix an article, no matter how egregious the error.
            Actually the original post of *this* article had an error which Michael fixed right after someone reported it -- a second copy of one of the graphs instead of the perf/watt chart.

            http://phoronix.com/forums/showthrea...046#post390046

            Comment


            • #51
              Originally posted by baffledmollusc View Post
              Yes, really.

              Say you get a benchmark score of 80 FLOPS, iterations, whatever. Say it took 30 seconds, and the CPU used an average power of 50W over that time. So

              Performance per Watt = benchmark result / (average power * time / time)

              = 80 FLOPS / (50W * 30s / 30s)
              = 80 FLOPS / 50W
              = 1.6 FLOPS / W
              = 1.6 FLOPS per Watt

              The reason I wrote it that way is average power * time = energy, and to get a power you have to divide by time taken.

              Although now I look at it, I'm prepared to agree that while correct, it's not the most straightforward approach. I think the issue is I always think in terms of energy, and convert that to whatever later.
              Your equation negates time duration, thus assuming a constant Power consumption, when it's truly a mean average. At any rate, showing the irrelevance of time within an energy relationship seems rather odd, especially since reality doesn't match this linear relationship.

              Even the FLOPS itself is a fitted curve.

              Comment


              • #52
                I guess it depends on whether the benchmarks are run for a specific period of time and you "count the results" (eg Tropics, where the "score" the FPS) or whether they run until a certain amount of work is done and stop (eg C-Ray or timed kernel compile where the "score" is effectively 1 over the time taken to complete the work).

                In the first case time cancels out because the score is effectively # frames over energy required or (average FPS * time) / (average W * time), but in the second case the score is effectively a constant (fixed amount of work) over energy required or 1 / (average W * time). The denominator is the same in both cases but you don't have time in the numerator for the first two benchmarks.

                Comment


                • #53
                  Originally posted by bridgman View Post
                  I guess it depends on whether the benchmarks are run for a specific period of time and you "count the results" (eg Tropics, where the "score" the FPS) or whether they run until a certain amount of work is done and stop (eg C-Ray or timed kernel compile where the "score" is effectively 1 over the time taken to complete the work).

                  In the first case time cancels out because the score is effectively # frames over energy required or (average FPS * time) / (average W * time), but in the second case the score is effectively a constant (fixed amount of work) over energy required or 1 / (average W * time). The denominator is the same in both cases but you don't have time in the numerator for the first two benchmarks.
                  They are all basically equivalent.

                  In the compilation benchmark, you can think of the performance as being measured in "compilations per second" (since the compilation takes many seconds, the numerical value would be less than one).

                  So whatever the benchmark, the performance can be stated as "operations per second" for some definition of operation (compilation, frame render, etc.). The denominator is Power (or average Power) which is equal to Energy per second. So in all cases:

                  Code:
                  Performance per Watt = ( operations / sec ) / ( Energy / sec ) = operations / Energy
                  But since the computational task is the same for each CPU within a given benchmark, we can define (normalize) the operation to always be equal to 1 (for any benchmark) -- in other words, one operation equals one compilation, one set of frame renderings, etc.

                  Bottom line is that comparing relative Performance per Watt among various CPUs is equivalent to comparing the reciprocal of total energy required to complete the computation for each CPU. And total energy required can be determined by first finding the average power consumed during the computation and multiplying that by the time required to complete the computation. This is exactly what tuke81 did in his graph in this thread.
                  Last edited by jwilliams; 01-19-2014, 12:57 AM.

                  Comment


                  • #54
                    Just to be clear, I wasn't trying to come up with a new formula, just trying to break what seemed to be a discussion looping around the fact that sometimes time mattered and sometimes it didn't. Agree that you can always include time as long as it's also included in the numerator for things like FPS benchmarks to properly calculate the total # of "operations".

                    Comment


                    • #55
                      Originally posted by dr4go View Post
                      Why did you use the ondemand governor for the AMD cpu whereas the intel processors all have the performance governor set?
                      Just to repost this as I still believe that - nevertheless which formula you use - the scores are pointless, because of that mistake.

                      Also how is it possible, that in Unigine Tropics v1.3 the i7 3770 is faster than any of the Haswells? Could it be, that there is something wrong with the gpu turbo or the Haswell driver?

                      Comment


                      • #56
                        Originally posted by dr4go View Post
                        Also how is it possible, that in Unigine Tropics v1.3 the i7 3770 is faster than any of the Haswells? Could it be, that there is something wrong with the gpu turbo or the Haswell driver?
                        Haswell graphics regressed in kernel 3.13 (I think), so that's why all Haswell CPUs have lower performance than Ivy Bridge in terms of graphics, even though it should be the other way around. There were already some articles stating that, but no explanation why it happens.

                        Comment


                        • #57
                          To complicate further and just generally mess with people, isn't this what some people are grasping for:

                          Performance/Total energy expended = (1/sec)/(average watt * sec) =(1/sec)*(1/watt*sec) = 1/watts*sec^2

                          This to equate a computer thats twice as fast but uses double the energy to complete a certain workload as the comparison. Though i agree it's an odd metric.

                          Comment


                          • #58
                            Well while using SI-units like joules are ok. I have always used Wh for electric energy(batteries and electric grid). Like i.e. c-ray bench would be:


                            The other thing what puzzles me is the used power supply. Is the power supply efficiency bothered at all in the tests? Efficiency is lower the less percentage of power load is used from power supply rating.

                            Lets assume that platinium rated 450W power supply is used, then i.e. kaveri uses approx 90*119W/100=107.1W power which is actually 100%*107.1W/450W=23.8% load. 23.8% load in platinium rating uses over 90% efficiency so math is fine.

                            But now let see the least power usage scenario. i3 2120 uses approx 90*58.8W/100=52.92W power which is 100%*52.92W/450W=11.76% load. 11.76% load on platinium rated power supply is no where near at 90% efficiency the real efficiency will be closer to 80% and real cpu power consumption is in real lower.

                            But of course used power is what is measured. But the thing is if you want to omit power supply efficiency from equation you
                            a) have to take care that power supply efficiency curve is known and calculate real system power usage omitting power supply all together or
                            b) use small enough power supply that load is between 20%-100% with all cpus.

                            Comment


                            • #59
                              Originally posted by Bjotn View Post
                              This to equate a computer thats twice as fast but uses double the energy to complete a certain workload as the comparison. Though i agree it's an odd metric.
                              So, Performance per Joule instead of Performance per Watt.

                              The thing is, I cannot imagine a situation where it would be more useful to know the Performance per Joule instead of knowing the energy AND time required to complete the operation (which is what we know from phoronix's timed benchmark combined with tuke81's graph). If you are trying to minimize energy usage, you choose the one that used the lowest energy to complete the operation. If you have to complete the operation in minimum time, you choose the one that completed the operation in the shortest amount of time.

                              Therefore, I agree that Performance per Joule is an "odd metric". Not very useful.
                              Last edited by jwilliams; 01-19-2014, 12:44 PM.

                              Comment

                              Working...
                              X