Benchmarking A 10-Core Tyan/IBM POWER Server For ~$300 USD

[q66_]

The updated PNOR didn't work, failed to boot, so I restored the original 1.0.

updated PNOR (2016 1.01 from Tyan) works, you just have to reboot the BMC after updating it. (i updated the PNOR via the megarac interface using the .hpm file, then after the machine rebooted, i removed power, plugged it back in, let it boot up)

[q66_]

Also, seconding what has been said about RAM. 8GB sticks are very cheap ($18 or so for one) and there is little point in using bigger ones, as with the best configuration (16 sticks) you can get as much as 128GB RAM, which is as much as one could usually want.

[curaga]

https://tyan.com/Barebones=TN71-BP01...BTO)=downloads is the link where I got the images. I rebooted several times, but it never got to petitboot, though I don't remember if I pulled power. In the BMC audit logs there were critical messages that PNOR failed to boot several times, trying golden side. Not sure if playing with these more is worth it, the only advantage mentioned was better fan scaling.

[q66_]

I had some kernels fail to boot on the old PNOR, and a friend had the machine freezing when doing 3D graphics stuff with a Radeon GPU put in it, which got solved by updating as well.

[illuhad]

That's also the firmware file I used - it should work. If I remember correctly, I had to reset the BMC as well (not sure if I pulled the plug or just issued reset command). I can also confirm issues with a radeon GPU and the original firmware. I had the graphics card mysteriously disappearing from lspci from time to time before the upgrade..

Since there are now several people owning this system (and likely more to come with the advertisement by phoronix), is there a better place than this thread for us as a user community to discuss issues, share configurations, cooling solutions, hardware compatibility information and so on?

[curaga]

I don't think so, the recently started openpower-community-dev ML is close, but not quite ideal.

[kgardas]

Well, thanks for the article, but it contains several issues/errors but those are quite fine. What's not fine is how you hurt box with bad cooling! Man, this is 190W TDP CPU and you need to blow a volume of air right *THOROUGH* the CPU heatsink. I guess even my simple solution using Noctua's industrial 2k rpm fan is better than your and yet you use way much powerful fan. See for inspiration how easily you can cool this box: https://twitter.com/KarelGardas/stat...35421902835713

Enough rants! Now, you do have POWER8. Congratulations, this machine is truly unique since it combines quite powerful cores together with ability to divide them into big number of low-power threads to mitigate RAM latency. This is also a box which CPU is connected to 4 Centaurs which means not only you do have 8*8MB of cache on CPU, but you also do have additional 4*16MB of cache between RAM and CPU. So good for memory intensive operations. But to make that truly working you definitely need to add more RAM.
Back to threads. You have probably run in 64threads configuration. This is not so well for some tests. For example you post 800 signs/s on RSA 4096. How is it possible that I can post 1200 signs/s while using the same box? See https://openbenchmarking.org/result/...SP-POWER832T47 -- it's easy, this particular test is more speedy when you not divide CPU into max number of threads (which is 8 threads per 1 core), but just allow it to run with 4 threads per 1 core resulting in 32 threads on CPU. How is it done? Easily:

$ sudo ppc64_cpu --smt=4

This way I'm able to post 1200 signs/s and get to the level of Epyc 7251/Silver 4108. And it does not costs anything just give machine a bit time and play with tuning. :-) I guess the same may be done also on other tests but I'll leave that to interested user...
Enjoy!
Karel

[illuhad]

As promised, here are pictures of my cooling solution involving quieter Supermicro fans and an air duct:

https://twitter.com/illuhad/status/1109224555313332224

It's not silent like some of the other presented solutions, but it's quiet enough for me and allows for the lid to be closed. Temps are low enough to run on 3.5 GHz turbo boost permanently.

[kgardas]

https://openbenchmarking.org/result/1903220-SP-POWER8THR74

Here is 7zip on the same box. 44k. Quite a difference with the article.

[illuhad]

Interesting! I even got 64k with the 10 core:

https://openbenchmarking.org/result/1903171-KH-POWER8CPU73

I would not have expected it to be that much faster on my machine with 2 more cores (at slower clockspeeds). I can only imagine that this is the effect of having more DIMMs installed (?), although I'm not sure how memory-bound compression workloads are.