Raspberry Pi 3 Benchmarks vs. Eight Other ARM Linux Boards

[l8gravely]

So what about including the BeagleBone Black into this comparision as well? It's able to run all the software that these other boards run. The big difference is probably graphic outputs....

[edolnx]

For measuring the power for those lower power boards, I picked up a USB Voltage Meter with OLED display from Adafruit for a project I'm working on. It does not have any data logging or output, but it does a great job and just sits in-line between a USB cord and it's socket. Data passes through and power is monitored. Really handy for me to determine power supply requirements on a per-board basis.

[aegrotatio]

I think it's a real shame the Raspberry Pi still puts the network and other high-I/O devices on the USB bus. Other boards at similar prices have managed to move it onto a faster bus. My iperf tests show even the SDcard isn't any faster than USB, either. I realize it's because the USB bus basically comes free on this chipset, but I think a couple extra dollars would have gone a long way, like with the PINE64 and other devices.

[tkaiser]

Banana Pi M3 added: http://openbenchmarking.org/result/1...gv=BananaPi+M3

Nope, you just added another bunch of numbers without meaning to openbenchmarking.org. Compare your results with the last 3 here: http://openbenchmarking.org/result/1...GA-1603090GA04

Also Banana Pi M3. The differences are:

- different software settings (other GCC version, other default compiler switches)
- different hardware configuration (the thermal configuration you used sucks -- SinoVoip has no clue how to do software so using their Raspbian image you end up with a slow device)
- different thermal settings (I used a cheap heatsink + fan on the 1st and 2nd run and switched off the fan on my 3rd)

The PTS is not capable to measure hardware in its current state (ignoring the throttling issue completely and using dumb defaults). You measure software settings instead all the time. That can be tweaked easily.

The best example are the misleading scores Michael is using all the times for Orange Pi PC/Plus: http://openbenchmarking.org/result/1...+Pi+PC+Armbian

He simply ignored that the Raspbian image he tested with implemented thermal throttling the worst way possible: killing CPU cores when the board reaches a specific temperature. A good benchmark should detect that it runs on a system that lost 3 of 4 cores recently. Does not apply to the PTS unfortunately

TL;DR: PTS is producing numbers without meaning with default settings. A benchmark running in 2016 ignoring thermal throttling is insufficient (and there are a few more design flaws that affect the results you get with PTS)

[tkaiser]

One of the most important design flaws the PTS shows is that it totally ignores thermal throttling strategies. For example my last benchmark run with Banana Pi M3 without an active fan and just a heatsink the thermal throttling strategy led to 4 CPU cores killed when starting the JTR test: http://kaiser-edv.de/tmp/GdV10e/Bild...2000.10.49.png

tk@bananapi:~$ grep -c processor /proc/cpuinfo
4

That means that the remaining time the 'benchmark' ran with 4 instead of 8 cores. A benchmark that is not able to detect this isn't worth a look. Surprisingly many benchmark numbers were still ok. But that's caused by weird defaults, eg. running many of the tests single threaded which prevents getting an idea how throttling or something like 'Turbo Boost' might influence performance.

This is alarming and PTS results shouldn't be used to compare different architectures unless this is resolved.

[riklaunim]

Nope, you just added another bunch of numbers without meaning to openbenchmarking.org. Compare your results with the last 3 here: http://openbenchmarking.org/result/1...GA-1603090GA04

Also Banana Pi M3. The differences are:

- different software settings (other GCC version, other default compiler switches)
- different hardware configuration (the thermal configuration you used sucks -- SinoVoip has no clue how to do software so using their Raspbian image you end up with a slow device)
- different thermal settings (I used a cheap heatsink + fan on the 1st and 2nd run and switched off the fan on my 3rd)

The PTS is not capable to measure hardware in its current state (ignoring the throttling issue completely and using dumb defaults). You measure software settings instead all the time. That can be tweaked easily.

I've noticed that thermal throttle (and raspbian is pretty much useless for M3 by default as GPU looks unused). I'm redoing some benchmarks with "cooling" and Ubuntu Mate image with GPU support. Looks better.