Announcement

Collapse
No announcement yet.

NVIDIA GH200 CPU Performance Benchmarks Against EPYC Zen 4 & Xeon Emerald Rapids

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • #71
    Originally posted by sophisticles View Post

    Welcome to Phoronix.

    By now you must have realized that there is at least one poster, who i suspect posts under various names, that is an absolute AMD fanatic while hating NVIDIA and Intel; any benchmarks that do not show AMD beating the two by significant margins will be dismissed as flawed, biased or inconsequential because there are better methods of performing the task.

    This will be their stance even when clear proof is provided that the above is not the case.

    In the interest of fairness, from the links you provided, using the easiest metric for comparison:

    AMD Instinct MI300A - Peak Double Precision FP64 performance 61.3 TFLOPs

    NVIDIA H100 GPU - FP64 34 TFLOPS

    Five AMD Instinct MI300A would be 306.5 TFLOPS

    The Grace CPU is capable of 7.1 TFLOPS according to NVIDIA, so that's a total of 41.1 TFLOPS for the NVIDIA superchip.

    Now this is only for FLOPS and it could be that in actual workloads the massive bandwidth the chip has allows it to match five MI300A.
    To be fair, it is currently unkown how Mi300 and GH200 compare until there is benchmark data available. But as a rough estimate, i believe when comparing sytems with the same amout of memory, which is important because you need that much to swiftly run inference with the biggest LLMs currently available, that the GH200 seems to be the much better choice. Its much cheaper, much more energy efficient and the performance is either roughly the same or it is not extremly far of. We will see. As a mather of fact all new bleeding edge AI supecomputers are based von GH200 linked coherently via NVlink 900 GB/s.

    Contrary to your statement (NVIDIA H100 GPU - FP64 34 TFLOPS) Nvidia has released the following performance for the GH200: 67 teraFLOPS FP64 see nvidia website. But comparing flops is not the whole story. Since the major bottle neck is the slowest data connection. Here the fully coherent architecture of the superchip has a massive advantage compared to PCIe,OAM,SXM.
    Last edited by GPTshop.ai; 11 February 2024, 08:09 AM.

    Comment


    • #72
      Originally posted by coder View Post
      P.S. I happen to know who sponsored your Phoronix Premium membership and they quickly concluded it was a mistake.
      Wait, are you saying that Michael did not give me a complimentary membership?

      I had planned to buy a membership at full price as a thank you when I started working but now that you say that, someone paid for it out of their pocket?

      That was very nice of them, who was it?

      Was it you?

      Comment


      • #73
        Originally posted by GPTshop.ai View Post
        Contrary to your statement (NVIDIA H100 GPU - FP64 34 TFLOPS) Nvidia has released the following performance for the GH200: 67 teraFLOPS FP64 see nvidia website.
        The reason for the confusion is that each CDNA and Hopper have different numbers for vector throughput vs. tensor products.

        Model Vector TFLOPS (fp64) Tensor TFLOPS (fp64)
        H200 33.5 66.9
        MI300A 61.3 122.6
        MI300X 81.7 163.4






        Originally posted by GPTshop.ai View Post
        But comparing flops is not the whole story.
        Yup. Software support and real-world performance is what ultimately matters.

        If they fall short, this wouldn't be the first time AMD has won the specs war but lost on actual fps.

        Comment


        • #74
          Originally posted by sophisticles View Post
          Wait, are you saying ...
          I shouldn't have said anything, but I worry about feeding your narcissism.

          You're a smart guy, no doubt. The thing is that smart is worthless. Deeds are the only thing that ultimately matter.

          People don't just want to work with competent folks, but also those who they enjoy being around. I think you'd be much happier & probably more successful, if you'd learn that you can afford to have a bit more humility and prove your worth through different means.

          I'm genuinely sympathetic for you, over whatever trauma in your life taught you differently.

          Comment


          • #75
            Originally posted by coder View Post
            The reason for the confusion is that each CDNA and Hopper have different numbers for vector throughput vs. tensor products.
            Model Vector TFLOPS (fp64) Tensor TFLOPS (fp64)
            H200 33.5 66.9
            MI300A 61.3 122.6
            MI300X 81.7 163.4








            Yup. Software support and real-world performance is what ultimately matters.

            If they fall short, this wouldn't be the first time AMD has won the specs war but lost on actual fps.
            Thank you, that is very valueable info. They do everthing to confuse people. Maybe I need to adjust my performance estimate a little. Eventhough the main difference between AMD Mi300 and GH200 is that Mi300 is severly limited by the 128 GB/s infinity link versus the 900 GB/s Nvlink. Also Mi300A can only scale up to a maximum of 4 which is only 512 GB of memory.
            Last edited by GPTshop.ai; 11 February 2024, 08:07 AM.

            Comment


            • #76
              Originally posted by coder View Post
              I shouldn't have said anything, but I worry about feeding your narcissism.

              People don't just want to work with competent folks, but also those who they enjoy being around. I think you'd be much happier & probably more successful, if you'd learn that you can afford to have a bit more humility and prove your worth through different means.

              I'm genuinely sympathetic for you, over whatever trauma in your life taught you differently.
              I think it's time I corrected you on something.

              At first i thought it was funny that you claimed i was a narcissist, because you obviously do not know what a narcissist is and even more ironically, since you seem to believe that you are qualified to make a medical diagnosis of someone you never met and you lack the expertise to do so even if we had met.

              As I have stated, for the past 15 years I worked in the biomedical field, 10 years in lab settings and 5 years prior to that in hospitals. In one of those jobs I worked in behavioral health, specifically the PICU.

              While I am not a Psychiatrist, Psychologist, BHC or LSW, I did learn a thing or two, among them that before a diagnosis of a disorder can be made there needs to be a uninterrupted, continuous 6 months of symptoms. Having any one or two symptoms of any disorder for a short period of time does not warrant a diagnosis of a particular disorder.

              The reason I know this is because my job was to prepare the involuntary commitment papers and get judicial approval for temporary commitment and make sure that the hospital was adhering to all state regulations as well as ethical and medical standards.

              It's actually extremely comical that you deign to lecture me on humility while ignoring the irony of your statements.

              I can't even begin to unwrap the twisted thought process that is required to conclude that just because someone has a different outlook on life in general that they must have had a traumatic event.

              If you wish to learn more about various disorders:

              Overview of Personality Disorders - Etiology, pathophysiology, symptoms, signs, diagnosis & prognosis from the Merck Manuals - Medical Professional Version.


              Comment


              • #77
                Originally posted by GPTshop.ai View Post
                the main difference between AMD Mi300 and GH200 is that Mi300 is severly limited by the 128 GB/s infinity link versus the 900 GB/s Nvlink.
                Yeah, I remember reading the MI250X appeared to software as two separate GPUs, even though they shared a package. I'd be curious to know if MI300 shows up as multiple GPUs or if it appears as a single, unified device.

                Anyway, Hopper's key feature is AI performance, and there it edges out the MI300A.

                Comment


                • #78
                  Originally posted by sophisticles View Post
                  you seem to believe that you are qualified to make a medical diagnosis of someone you never met and you lack the expertise to do so even if we had met.
                  I only reached for that label in exasperation at your seemingly endless reserve of self-confidence, often far outpacing your finite reserve of knowledge about many subjects. I'm baffled why you feel qualified to hold forth on subjects like software licenses, without even a basic grounding in the subject. And that's just one example.

                  This is not normal. Far from impressive, it makes you look buffoonish. Your ego is also a nuisance, interfering with your ability to participate constructively on these forums and increasing the signal-to-noise ratio the rest of us have to deal with.

                  You're right that I have no training in mental health, so maybe take it as informal feedback and certainly not an actual diagnosis. What you get back from the world has a lot to do with what you put out there, and if you want to start experiencing more positive feedback and outcomes, then I'd recommend taking a hard look at what's going on at your end.

                  Originally posted by sophisticles View Post
                  It's actually extremely comical that you deign to lecture me on humility while ignoring the irony of your statements.
                  Let me ask you this: do you believe there are assholes in the world? Does it take a psych degree to know that someone is an asshole? It may be that only a board-certified psychologist can properly pick apart why someone is misbehaving in a certain way, but the rest of us can usually at least spot a troublemaker when we see one.

                  Here's the thing: I don't really care about you, one way or another. Ideally, I want you to be well, happy, and successful. But, at the end of the day, what I care about is being able to use these forums to share knowledge and information with others and not have them flooded with misinformation and trolls. So, go ahead and ignore my feedback, but it's ultimately your behavior that I have a problem with and which I'm objecting to.
                  Last edited by coder; 11 February 2024, 02:47 AM.

                  Comment


                  • #79
                    Originally posted by GPTshop.ai View Post

                    32 Nvidia superchips can be connected together coherently called NVL32.
                    I have to correct myself, the limit is 256 not 32!!!
                    Last edited by GPTshop.ai; 11 February 2024, 08:07 AM.

                    Comment


                    • #80
                      Originally posted by coder View Post
                      I only reached for that label in exasperation at your seemingly endless reserve of self-confidence, often far outpacing your finite reserve of knowledge about many subjects. I'm baffled why you feel qualified to hold forth on subjects like software licenses, without even a basic grounding in the subject. And that's just one example.
                      There is another explanation, that I have a deeper understanding of the subject matter that you are unable to accept.

                      Taking your example of software licenses, it doesn't take a lawyer to understand the GPL or to see the consequences of it, where a handful of people are making millions of dollars from the work of thousands of others that work for free because they are convinced that they are doing something good for their fellow man.

                      If it walks like a duck, and it quacks like a duck, it's a duck but if it walks like a duck, quacks like a walk and looks like a goose, then it's a scam.

                      Just like the GPL is.

                      Originally posted by coder View Post
                      This is not normal. Far from impressive, it makes you look buffoonish. Your ego is also a nuisance, interfering with your ability to participate constructively on these forums and increasing the signal-to-noise ratio the rest of us have to deal with.
                      If you are going to insult me and try to use a technical argument to do so, then at least correctly use the technical argument.

                      Signal to noise ratio comes from radio broadcasting, where signal is that which you want transmitted, i.e. the music or commentary and noise is anything other than what was transmitted, which in the early days manifested itself as noise or static.

                      If I am increasing the signal to noise ratio, that is a good thing, SNR is measured in decibels and anything over 45dB is considered pristine.

                      So think about what you did, you have referred to me as a noob, you called me a buffoon, and then misused the insult you thought would make you look smart.

                      What you should have said is that i am reducing the PSNR to below 15dB, that would have been a clever insult but instead you showed a complete lack of understanding of a basic concept and forced me to insult myself.

                      You wanted to know about the trauma in my life, this is it, me being forced to correct your insults because you are not smart enough to realize how stupid your comments are.



                      Originally posted by coder View Post
                      You're right that I have no training in mental health, so maybe take it as informal feedback and certainly not an actual diagnosis. What you get back from the world has a lot to do with what you put out there, and if you want to start experiencing more positive feedback and outcomes, then I'd recommend taking a hard look at what's going on at your end.
                      Thanks Confucius.

                      I take it you do not see the irony of insulting me in post after post and then advising me to follow the teachings of Matthew 7:12.

                      Originally posted by coder View Post
                      Let me ask you this: do you believe there are assholes in the world? Does it take a psych degree to know that someone is an asshole? It may be that only a board-certified psychologist can properly pick apart why someone is misbehaving in a certain way, but the rest of us can usually at least spot a troublemaker when we see one.
                      Please keep in mind that I am a buffoon with not much education so please explain to me how this above paragraph arises from the advice you offered in the previous paragraph.

                      Please do so slowly, because i am not sure my simple mind can understand.

                      By the way, I don't know if you know this or not, but they made a song about me, it's quite catchy, sing it all the time:



                      Originally posted by coder View Post
                      But, at the end of the day, what I care about is being able to use these forums to share knowledge and information with others and not have them flooded with misinformation and trolls.
                      Is that how you see yourself? As someone that is trying to educate this forum and increase the knowledge of the uninitiated but i alone am standing in the way of you accomplishing your life's work?

                      And I'm the narcissist?

                      Interesting that you only seem to have a problem with me, not any of the others, like the guy that things all colleges have conspired to promote NVIDIA, or the guy that seemed to have a mayo fetish about Microsoft or any of the other users who lack even the basic knowledge of how OSes work and the only thing they contribute is how NVIDIA, Microsoft, Apple, BSD, all suck.

                      i know what your problem is with me, it's been obvious for a while, you are threatened by me.

                      For a while you were able to pass yourself as knowledgeable on certain subjects but i came along and demonstrated a superior grasp of most subjects and you can't handle that.

                      Worse, I shone a light on the absurdity of working for free while others make millions from your work, and I suspect that you may have contributed GPL code at some point in your life, and my comments have caused you to have an existential crisis when you realized i was right.

                      It never had occurred to you that Linus had tens of millions of dollars in the bank, or that he earned over 1.7 million a year, or that the Linux Foundation rakes in 177 million a year or that the Rocky people stole Red Hat;s work so that they can get 26+ million dollars in funding, ignorance is bliss.

                      I came along and opened every one's eyes and you resent me for that.

                      The truth hurts, but it must be shared.

                      I really am sorry that my knowledge has hurt your ego i am sorry that i have shattered the illusion of the GPL and i am sorry that you can't seem to handle it.

                      If you going to respond with an insult, let me do you a solid and filter it through me first, so that I can make sure it is structured correctly and you don't come off like a self-important, pompous ass that doesn't understand the implications of what you are saying.
                      Last edited by sophisticles; 11 February 2024, 02:17 PM.

                      Comment

                      Working...
                      X