The Power Efficiency Between Ubuntu 19.04, Clear Linux & openSUSE Tumbleweed With CompuLab's Airtop 3

[linuxgeex]

linuxgeex Turns out if you set the No Box Plots checkbox, you get the full plot. The CPU temperature plot is just above it. Comparing them, Tumbleweed looks pretty throttled…

I've extracted the samples from the XML, but haven't yet found the system_monitor sys.power polling interval with which to accumulate them.

Cool. I'm glad you have the energy to explore it lol. I noticed the median temp delta and I've had poor experiences over the last 18 years with SuSE's power management so my opinion was a hasty armchair one based on that one observation... if you can tell the story better than me, all I can say is <simpsons_reference>excellent</simpsons_reference>. ;-)

[linuxgeex]

Ah, ok – I'd missed the significance of your request for representing a typical server.

This constraint you usually won't get with a passively cooled system. These results show several tests which required many more repetitions to significantly converge (some as many as 40) on different distros. I suppose you could assume this was negligible, attempt to exclude the extras, or seek them out as an indication of thermal throttling (or obtrusive default tasks/daemons). I don't think the run-individual results and logs are retrievable from openbenchmarking.org.

All very true. The duration being similar is based on the assumption that the test suite runs enough times for the durations to "settle down" to a reasonable noise floor when summed for the entire suite. I trust that to be the case. The performance geomean results were close, so I don't repent suggesting that the suite durations could be called equally "close". ;-)

The median core temperature deltas to ambient however were another story.

Assuming ambient temp of 22C, Ubuntu's median core temp was +42C, while SuSE's was +60C. If the fan speed was static then the dissipation ratio of SuSE to Ubuntu would be 60²/42² = 2.04 = +104% more core heat dissipation for the SuSE run. Not knowing whether the fan speed is static, it's fair to assume that the fans were spinning slower for Ubuntu at 18C cooler, and therefore the dissipation delta was lower than +104%. How much, we don't know. That's the ceiling. But I think it's also fair to assume that SuSE lost some TB2 opportunities running with a median core temp above 80C.

[utrrrongeeb]

Thanks, I'd been looking at the local file for PTS version 8.0. (Assuming that's consistent, my last results are twice what they should be.)

Is the openbenchmarking.org /prospect/ versus /result/ path a known bug?

[Michael]

Glancing at system_monitor.php suggests a default delay of two seconds between readings. If all samples are indeed 2s apart, throughout the benchmarking the energy used by Clear, Ubuntu, and Tumbleweed, respectively, was 291, 231, and 603 kJ (rounded, and still not corrected for test repetitions).

It should be a 1 second interval - https://github.com/phoronix-test-sui...onitor.php#L49

[utrrrongeeb]

Glancing at system_monitor.php suggests a default delay of two seconds between readings. If all samples are indeed 2s apart, throughout the benchmarking the energy used by Clear, Ubuntu, and Tumbleweed, respectively, was 291, 231, and 603 kJ (rounded, and still not corrected for test repetitions).

[utrrrongeeb]

linuxgeex Turns out if you set the No Box Plots checkbox, you get the full plot. The CPU temperature plot is just above it. Comparing them, Tumbleweed looks pretty throttled…

I've extracted the samples from the XML, but haven't yet found the system_monitor sys.power polling interval with which to accumulate them.

[utrrrongeeb]

Thanks! I saw the XML button, but since it was between CSV and PDF, I lazily didn't test it and assumed it was just another format of the same accumulated results…

If going to OpenBenchmarking.org there are options for plotting all the individiaul run data as well, etc.

I'm not seeing an obvious way to do this.

More importantly, is the /prospect/ thing a known bug or feature?

[Michael]

Ah, ok – I'd missed the significance of your request for representing a typical server.

This constraint you usually won't get with a passively cooled system. These results show several tests which required many more repetitions to significantly converge (some as many as 40) on different distros. I suppose you could assume this was negligible, attempt to exclude the extras, or seek them out as an indication of thermal throttling (or obtrusive default tasks/daemons). I don't think the run-individual results and logs are retrievable from openbenchmarking.org.

Individually run results are always maintained: as part of the raw XML data. If going to OpenBenchmarking.org there are options for plotting all the individiaul run data as well, etc. There is/was tooltips for showing the individual data too if mousing over the relevant parts of the graphs, but sadly due to modern web browsers handling of SVG, that no longer is working AFAIK.

[utrrrongeeb]

Ah, ok – I'd missed the significance of your request for representing a typical server.

same suite duration

This constraint you usually won't get with a passively cooled system. These results show several tests which required many more repetitions to significantly converge (some as many as 40) on different distros. I suppose you could assume this was negligible, attempt to exclude the extras, or seek them out as an indication of thermal throttling (or obtrusive default tasks/daemons). I don't think the run-individual results and logs are retrievable from openbenchmarking.org.

[linuxgeex]

Assuming the Watts Up measurements and PTS processing are valid, test run-time (s) multiplied by average power consumption (W) gives energy used for the test (J). Using the Blender graphs (which have many power samples over a relatively long time — the other tests are too short to use with much confidence), the results for Tumbleweed, Ubuntu, and Clear are respectively 91950.64, 86507.72, and 83275.52 joules. (Intermediate values - three significant digits would be generous. Also note that for differing run-counts N, those must be corrected for since the power graph is concatenated while the results are averaged.) In this one test, Tumbleweed uses 10.4% more energy than Clear.

Michael are the openbenchmarking.org links in the lower-left corner of the graphs supposed to go to /prospect/… paths, which are redirected to / index, instead of /result/…? Also, the intra-page graphs are at anchor …/#r-$UUID, instead of at path …/$UUID.

The reason I want the total power at the wall for the duration of the suite is because it would provide a total for all the runs, which is a good indication of power usage on an average server running mixed loads. In my experience, SuSE's power management is poorer. More power usage = lower thermal headroom = more throttling = lower performance.

Michael concluded that Clear outperformed Ubuntu while using only a little more power, but he kinda missed just how badly SuSE under-performed vs Ubuntu given it's much higher thermal/power results. This chart shows SuSE's median temp at what looks like 82C vs Ubuntu at 64C. Assuming 22C ambient, same suite duration, and same fan speed, that would indicate 104% higher core power consumption by the SuSE system. Obviously the fan speeds were significantly lower for Ubuntu, lol.

The Power Efficiency Between Ubuntu 19.04, Clear Linux & openSUSE Tumbleweed