AMD BIOS Tuning Guide Impact For Boosting AI/ML Performance On EPYC 9005 Series

Written by Michael Larabel in Software on 29 November 2024 at 10:36 AM EST. Page 2 of 4. 6 Comments.
OpenVINO benchmark with settings of Model: Age Gender Recognition Retail 0013 FP16, Device: CPU. AI/ML Tuning Recommendations was the fastest.
OpenVINO benchmark with settings of Model: Age Gender Recognition Retail 0013 FP16, Device: CPU. AI/ML Tuning Recommendations was the fastest.
OpenVINO benchmark with settings of Model: Weld Porosity Detection FP16-INT8, Device: CPU. AI/ML Tuning Recommendations was the fastest.
OpenVINO benchmark with settings of Model: Weld Porosity Detection FP16-INT8, Device: CPU. AI/ML Tuning Recommendations was the fastest.

It didn't take any time at all with AI/ML workloads to see the AMD BIOS workload tuning guide make a positive impact on performance.

OpenVINO benchmark with settings of Model: Vehicle Detection FP16-INT8, Device: CPU. AI/ML Tuning Recommendations was the fastest.
OpenVINO benchmark with settings of Model: Vehicle Detection FP16-INT8, Device: CPU. AI/ML Tuning Recommendations was the fastest.
OpenVINO benchmark with settings of Model: Person Vehicle Bike Detection FP16, Device: CPU. AI/ML Tuning Recommendations was the fastest.
OpenVINO benchmark with settings of Model: Person Vehicle Bike Detection FP16, Device: CPU. AI/ML Tuning Recommendations was the fastest.

Not only was the throughput benefiting from this AMD-recommended BIOS tuning but the latency was lower as well.

OpenVINO benchmark with settings of Model: Machine Translation EN To DE FP16, Device: CPU. AI/ML Tuning Recommendations was the fastest.
OpenVINO benchmark with settings of Model: Machine Translation EN To DE FP16, Device: CPU. AI/ML Tuning Recommendations was the fastest.
OpenVINO benchmark with settings of Model: Face Detection Retail FP16-INT8, Device: CPU. AI/ML Tuning Recommendations was the fastest.

The AMD EPYC 9005 series is already offering leading server CPU performance, but with the additional BIOS tuning there is additional headroom to tap.

OpenVINO benchmark with settings of Model: Face Detection Retail FP16-INT8, Device: CPU. AI/ML Tuning Recommendations was the fastest.

Among the tuning suggestions are applying the power determinism mode so in some cases this does mean slightly higher CPU power use but still respecting the TDP limits of the processor.

OpenVINO benchmark with settings of Model: Handwritten English Recognition FP16-INT8, Device: CPU. AI/ML Tuning Recommendations was the fastest.
OpenVINO benchmark with settings of Model: Road Segmentation ADAS FP16-INT8, Device: CPU. AI/ML Tuning Recommendations was the fastest.
OpenVINO benchmark with settings of Model: Person Re-Identification Retail FP16, Device: CPU. AI/ML Tuning Recommendations was the fastest.

The tuning guide does recommend putting the EPYC processor(s) at their maximum cTDP value, but in the case of the EPYC 9655 that is already at its 400 Watt configurable TDP limit. But for those with EPYC 9005 SKUs where the default TDP is lower than the max cTDP, you may see even greater performance benefits albeit with greater CPU power use.

OpenVINO GenAI benchmark with settings of Model: Phi-3-mini-128k-instruct-int4-ov, Device: CPU. AI/ML Tuning Recommendations was the fastest.

There were some smaller gains to find as well from OpenVINO with its generative AI testing when following the AMD BIOS tuning guide.

Related Articles
New AMD ERAPS Feature Yields Additional Performance Gains On Zen 5
Initial Benchmarks Of The AMD AOCC 5.0 Compiler On 5th Gen EPYC
Linux 6.12 Features Are Super Exciting With Real-Time, Sched_ext, Intel Xe2 & Raspberry Pi 5
AmpereOne Performance With GCC vs. LLVM Clang Compilers
AMD Ryzen 9 9950X vs. Ryzen 9 7950X/7950X3D For Workstation Graphics
AMD Zen 5 Not Affected By Inception/SRSO, mitigations=off Yields No Benefit On Ryzen 9000 Series