Announcement

Collapse
No announcement yet.

AMD BIOS Tuning Guide Impact For Boosting AI/ML Performance On EPYC 9005 Series

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • AMD BIOS Tuning Guide Impact For Boosting AI/ML Performance On EPYC 9005 Series

    Phoronix: AMD BIOS Tuning Guide Impact For Boosting AI/ML Performance On EPYC 9005 Series

    Following the release last month of the EPYC 9005 series processors, AMD published a BIOS and Workload Tuning Guide of straight-forward settings recommendations for those running new EPYC Turin servers to optimize the performance of different workloads like databases and Java to HPC and AI/ML software. Recently I started running some benchmarks to look at the impact of AMD's recommended BIOS tuning and beginning this comparison by looking at the performance (and power) impact across a range of AI / machine learning workloads on a 5th Gen AMD EPYC server.

    Phoronix, Linux Hardware Reviews, Linux hardware benchmarks, Linux server benchmarks, Linux benchmarking, Desktop Linux, Linux performance, Open Source graphics, Linux How To, Ubuntu benchmarks, Ubuntu hardware, Phoronix Test Suite

  • #2
    "Auto" setting value in ASUS motherboards can be anything. It doesn't tell you the current value.

    I wish there was more transparency

    Comment


    • #3
      Originally posted by Kjell View Post
      "Auto" setting value in ASUS motherboards can be anything. It doesn't tell you the current value.

      I wish there was more transparency
      That's for sure. But it is more complicated as it is shitSUS... on my motherboard if I disable some enchantments, default is auto, then it also disables AMD PBO part, there are settings that overlap.

      Last time I opened a ticket for them to fix BIOS, they replied me that their motherboards are designed to be used with Windows and use the dreaded armory crate. You can imagine my reaction to this.
      Last edited by Ferrum Master; 29 November 2024, 12:50 PM.

      Comment


      • #4
        Originally posted by Kjell View Post
        "Auto" setting value in ASUS motherboards can be anything. It doesn't tell you the current value.

        I wish there was more transparency
        My ASRock AM5 board is similar. Sometimes it says what Auto does, but what I don't like are the settings can be changed in multiple places and none reflect what is set where. Something like C-States can be set to On in one submenu and will be set to Auto (Disabled) or Off in a different one.

        My Gigabyte board before this, back on AM4, had the same differing submenu problem. Damn that's annoying.

        Comment


        • #5
          I have to assume that this tuning guide for AI/ML must have detrimental effects on non AI/ML workloads.

          If it doesn't then the question becomes why not have these be the default settings?

          Maybe they send power usage through the roof.

          There is no such thing as a free lunch.

          Comment


          • #6
            Originally posted by sophisticles View Post
            I have to assume that this tuning guide for AI/ML must have detrimental effects on non AI/ML workloads.

            If it doesn't then the question becomes why not have these be the default settings?
            That is why it called tuning. A bit like you tune your attention to something, so you pay less attention to something else. But the EPYC servers are often used for a specific task so of course it makes sense to optimize for that.

            Originally posted by sophisticles View Post
            Maybe they send power usage through the roof.
            Power usage was already included in the benchmarks.

            Originally posted by sophisticles View Post
            There is no such thing as a free lunch.
            That is why I always disable the cache, SMT and all except one core on my CPU. Because all that performance does come at a cost.

            Comment


            • #7
              The prompt processing speed seems much too slow for a 96-core, avx-512 cpu. Did you try without OpenBlas?

              Comment

              Working...
              X