Announcement

Collapse
No announcement yet.

Intel Xeon Platinum 8490H "Sapphire Rapids" Performance Benchmarks

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • #21
    aaahaaap

    ​It's a huge improvement from Ice-Lake. But yes, this article is focused too much on AI inference. No DB benchmarks for example.

    On one hand I can understand that because AMX is new, on the other hand I can see this as a result of Intel marketing their "accelerators" for months.

    It's not like SPR is going to make a real difference for AI inference because there are two real world scenarios for inference: 1. You really need to be fast and get a GPU or dedicated AI accelerator. 2. You just do inference here and there for small models where the performance of most server CPUs will be enough. The area between these two is probably very small.

    Michael writes "at least in AI workloads with AMX and other workloads able to take advantage of new Sapphire Rapids features, the price can actually be justified": Having two $17k CPUs in your system just to do the job of a $1600 GPU slower and with much higher power consumption doesn't make much sense. I would like to see some benchmarks comparing SPR to an old GPU like the T4.

    Most important: I'm pretty sure that Intel will drop AMX soon and integrate inference accelerators into Xeon chips as dedicated circuitry, not into cores. This approach is very inefficient from power and die area perspective. Intel will have dedicated AI accelerator circuitry in their next mobile CPU, AMD has announced those for Epyc and their APUs and Apple has their Neural engine while Qualcomm has their NPUs. Everyone can see how efficient those are.

    Comment


    • #22
      Originally posted by aaahaaap View Post
      Wow, that's a pretty bad showing for Intel compared to AMD, not sure why the article is so positive.
      Because the new CPUs are better for some applications. They're a massive improvement over older generations of Xeons. They're not universally better. The review is fair.

      If Michael were unfair Intel wouldn't send him a $40,000 machine.

      Comment


      • #23
        There is one thing I don't understand in this article. Why is Michael so positive about Sapphire Rapids when it gets destroyed by Genoa (except in a handful of corner case workloads).

        Comment


        • #24
          Michael

          I just noticed: In your Openvino benches you say the data type is FP16 but AMX only supports INT8 and BF16. Was BF16 used for SPR?

          Does that have any implications for accuracy for benches like face detection. A smaller mantissa but bigger exponent can have both higher and lower accuracy (even though size is the same and multiplication is easier).

          Comment


          • #25
            $17k for Xeon, $9k for EPYC 9554. I think you can buy an AMD Instinct for $8k.

            Comment


            • #26
              smotad

              These list prices shouldn't be taken serious. You can buy the 9654 for 8600€ in Europe, including 19% taxes. The 9554 is going for 6900€ incl. VAT.

              Retailer have them in stock (CPU availability is good but boards or systems are hard to get from retailer).

              Comment


              • #27
                Some of Michael's reviews are great, some are so-so and some prove that he doesn't understand what he is reviewing.

                This is one of those reviews.

                He is given a high end, purpose built server, with two $17,000 Xeon Platinum 8490H, each with 16 dedicated accelerators, and he runs benchmarks like code compilation.

                This is the equivalent of a car reviewer being given a Ford 350 Lariat Super Cab Long Box with a goose neck and doing 0-100-0, skid pad and slalom tests.

                No one one is buying a Xeon Platinum 8490H to run any of the software bench marked in this review.

                They are buying them to run specialized software that can benefit from the on-board accelerators.

                Thus this whole review, any any others like it by other reviews, is basically a waste of time.

                Comment


                • #28
                  Originally posted by sophisticles View Post
                  Some of Michael's reviews are great, some are so-so and some prove that he doesn't understand what he is reviewing.

                  This is one of those reviews.

                  He is given a high end, purpose built server, with two $17,000 Xeon Platinum 8490H, each with 16 dedicated accelerators, and he runs benchmarks like code compilation.
                  There was also OpenVINO, oneDNN, and other 'specialized' workloads too... It's not like there were just code compilation benchmarks. As mentioned multiple times in the review, I only had 4 days with the server so far and thus haven't had the time yet to setup various accelerator-friendly workloads especially with Intel not allowing public redistribution of their recommended workloads / script configurations. As mentioned, follow up benchmarks will come in the coming days/weeks looking more at these areas, this is just what I could throw together since Friday.
                  Michael Larabel
                  https://www.michaellarabel.com/

                  Comment


                  • #29
                    "They are buying them to run specialized software that can benefit from the on-board accelerators"

                    What kind of software are you talking about? He tested AMX and regarding QAT I'm pretty sure that nobody is buying high-spec/high-core chips at high cost just to have IPSec accelerated. DPUs are better for those kind of tasks anyway.

                    Comment


                    • #30
                      As I said, it's obvious that Michael does not understand the purpose of these processors, but he is clearly not alone as evidenced by the comments from "crudeboil:".

                      The accelerators found on these chips have specific uses, in compression and decompressing, encryption and decryption, databases (analytics):

                      https://www.intel.com/content/www/us...analytics.html

                      https://www.intel.com/content/www/us.../overview.html

                      https://www.intel.com/content/www/us...-overview.html

                      https://www.intel.com/content/www/us...l#gs.mc7xob​

                      The Xeon Platinum 8490H are Intel's answer to the processors found in IBM's Z15/16 servers, those processors are basically a collection of accelerators bound together with a general purpose cpu; the sole purpose of the cpu is to boot up the OS and initialize the peripherals, once that's done everything is handled by accelerators, encryption, decryption, math, I/O, compression, decompression, all of it, performed by accelerators, programmed using COBOL.

                      These Xeons are made for running custom software, much of it coded in Python, running libraries that can leverage the on-board accelerators.

                      The fact that Michael tested this server in the way he did, and then defending his test, tells me he doesn't understand the purpose of this system and frankly should never have been supplied with a test unit.

                      Last edited by sophisticles; 11 January 2023, 11:19 AM.

                      Comment

                      Working...
                      X