AMD BIOS Tuning Guide Impact For Boosting AI/ML Performance On EPYC 9005 Series

[jruhe]

The prompt processing speed seems much too slow for a 96-core, avx-512 cpu. Did you try without OpenBlas?

[fintux]

I have to assume that this tuning guide for AI/ML must have detrimental effects on non AI/ML workloads.

If it doesn't then the question becomes why not have these be the default settings?

That is why it called tuning. A bit like you tune your attention to something, so you pay less attention to something else. But the EPYC servers are often used for a specific task so of course it makes sense to optimize for that.

Maybe they send power usage through the roof.

Power usage was already included in the benchmarks.

There is no such thing as a free lunch.

That is why I always disable the cache, SMT and all except one core on my CPU. Because all that performance does come at a cost.

[sophisticles]

I have to assume that this tuning guide for AI/ML must have detrimental effects on non AI/ML workloads.

If it doesn't then the question becomes why not have these be the default settings?

Maybe they send power usage through the roof.

There is no such thing as a free lunch.

[skeevy420]

"Auto" setting value in ASUS motherboards can be anything. It doesn't tell you the current value.

I wish there was more transparency

My ASRock AM5 board is similar. Sometimes it says what Auto does, but what I don't like are the settings can be changed in multiple places and none reflect what is set where. Something like C-States can be set to On in one submenu and will be set to Auto (Disabled) or Off in a different one.

My Gigabyte board before this, back on AM4, had the same differing submenu problem. Damn that's annoying.

[Ferrum Master]

"Auto" setting value in ASUS motherboards can be anything. It doesn't tell you the current value.

I wish there was more transparency

That's for sure. But it is more complicated as it is shitSUS... on my motherboard if I disable some enchantments, default is auto, then it also disables AMD PBO part, there are settings that overlap.

Last time I opened a ticket for them to fix BIOS, they replied me that their motherboards are designed to be used with Windows and use the dreaded armory crate. You can imagine my reaction to this.

[Kjell]

"Auto" setting value in ASUS motherboards can be anything. It doesn't tell you the current value.

I wish there was more transparency

[phoronix]

Phoronix: AMD BIOS Tuning Guide Impact For Boosting AI/ML Performance On EPYC 9005 Series

Following the release last month of the EPYC 9005 series processors, AMD published a BIOS and Workload Tuning Guide of straight-forward settings recommendations for those running new EPYC Turin servers to optimize the performance of different workloads like databases and Java to HPC and AI/ML software. Recently I started running some benchmarks to look at the impact of AMD's recommended BIOS tuning and beginning this comparison by looking at the performance (and power) impact across a range of AI / machine learning workloads on a 5th Gen AMD EPYC server.

https://www.phoronix.com/review/amd-epyc-ai-ml-tuning