NVIDIA GH200 CPU Performance Benchmarks Against EPYC Zen 4 & Xeon Emerald Rapids

[a5ehren]

I know you said the lower knobs aren’t in the kernel, but do you have any idea what the system power was? Curious if it hangs near TDP

[sophisticles]

Interesting it can both lead and trail the benchmarks. This would lead me to think it is extremely unbalanced and tries to roll on some magic tricks they decided to employ to show certain benchmarks can be dominated.

The system is clearly not meant to be a general purpose computer, as evidenced by the 40+ thousand dollar price tag.

The systems target the very lucrative AI and HPC markets and for me the most telling result is the NWChem benchmarks, single 72 core GH200 Superchip in this system almost tied two EPYC that have a combined 128C/256T?

Making it even more impressive is that this software scales to thousands of cores easily:

https://www.nwchem-sw.org/

They run this software on Franklin Cray XT-4's that have over 19,000 cores:

https://www.top500.org/system/175161/

So you can't say the software doesn't scale, 128C/256T should be manhandling 72 Cores

Can't wait for all the "But Can It Run Crysis" jokes.

BTW, there are analysts predicting that NVIDIA's stock could hit over $1200 a share.

[GPTshop.ai]

I know you said the lower knobs aren’t in the kernel, but do you have any idea what the system power was? Curious if it hangs near TDP

There are some system power metrics in the additional supplemented results: https://openbenchmarking.org/result/...VIDIAGH254&sor

As the builder of the system, I can also confirm personally that CPU under heavy load alone does not draw more than 400 Watts max. Total system TDP CPU+GPU+memory is around 1000W.

[qarium]

The 41 grand is for the base model and it goes up from there.
They offer some beautiful glass cases for these systems, almost like they read my posts to System76 telling them to do the same thing.
They make some amazing performance claims:
The H100 costs 30 grand each, so 8 of them would cost 240 grand, divided by 5 equals 48 grand, so that claim is accurate.
Assuming the rest of their claims are accurate:
One H100 consumes 700 Watts of power, eight would consume 5600 Watts of power, meaning that this system consumes roughly 560 Watts of power.
This seams like very low power consumption.
I can't wait for NVIDIA's desktop ARM chips in 2025.

Its really mysterious your activity here on phoronix. if AMD release a Threatripper who is faster than a AMD Epyc 9554
then the AMD Threatripper is a scam and you claim people should buy a intel 14000 instead.
the AMD Ryzen Threadripper 7980X is only 5337€ keep this in mind.

but then if Nvidia release a 72-core ARM CPU "NVIDIA GH200" who is in fact slower and more expensive than a Threatripper then of course you praise it as if it is pure Gold..... and of course its only 47,500 € the most basic model...

of course you believe cuda is best but outside of this insanity people couly build a Threadripper 7980X and put in some Instinct MI100/MI200/mi300 cards or RadeonPRO w7900
and i am pretty sure from this 47500€ you save 15000€ or more if you do so.

but i know your kind of person they believe if something is slower and more expensive they then believe they are more elite persons because they can afford the premium price.

[mrg666]

I would still go with EPYC Bergamo as it seems to be the best option for both price and performance. Thanks, Michael! This is very good data to have.

[pWe00Iri3e7Z9lHOX2Qx]

of course you believe cuda is best but outside of this insanity people couly build a Threadripper 7980X and put in some Instinct MI100/MI200/mi300 cards or RadeonPRO w7900
and i am pretty sure from this 47500€ you save 15000€ or more if you do so.

but i know your kind of person they believe if something is slower and more expensive they then believe they are more elite persons because they can afford the premium price.

People aren't really cobbling together Threadripper and Instinct systems on Amazon though. GH200 is tailor made for the LLM craze. And it isn't just raw CPU or GPU performance. You can get AWS instances of these with 4.5TB of HBM3e. The CPU to GPU interconnect is 900GB/s. They are stupid fast at what they are made to do.

[sophisticles]

Its really mysterious your activity here on phoronix. if AMD release a Threatripper who is faster than a AMD Epyc 9554
then the AMD Threatripper is a scam and you claim people should buy a intel 14000 instead.
the AMD Ryzen Threadripper 7980X is only 5337€ keep this in mind.

but then if Nvidia release a 72-core ARM CPU "NVIDIA GH200" who is in fact slower and more expensive than a Threatripper then of course you praise it as if it is pure Gold..... and of course its only 47,500 € the most basic model...

of course you believe cuda is best but outside of this insanity people couly build a Threadripper 7980X and put in some Instinct MI100/MI200/mi300 cards or RadeonPRO w7900
and i am pretty sure from this 47500€ you save 15000€ or more if you do so.

but i know your kind of person they believe if something is slower and more expensive they then believe they are more elite persons because they can afford the premium price.

Allow me to clear up the mystery for you.

Threadrippers are a scam because of their positioning in the market.

They are great if you want to build a system to run a large number of virtual machines.

They are great if you want to run a game or web server.

They are not great if you want to build system for encoding video, or editing video and/or audio because they are extremely expensive and there are more efficient and cost effective solutions for those tasks.

This system is gold for certain tasks and is faster than a Threadripper for those tasks,

This system is a highly specialized machine, kind of like how a diesel pickup truck is unbeatable. Years ago I owned a Dodge pickup with a turbo diesel straight six, it only had 190 hp and 420 lb-ft torque. It only revved up to 2500 rpm.

You were not going to win too many drag races with that thing and I don't think it would even do 80 mph, but if you were in the middle of a blizzard, and needed to put a 500 pound snow plow on the front, carry 1500 pounds of salt and tow a salt spreader up a steep hill, it could not be beaten.

As for CUDA, it is unbeatable for certain tasks, which is why every major university offers CUDA classes, either as mandatory or electives for their computer students:

https://people.csail.mit.edu/xchen/gpu-programming/

http://courses.cms.caltech.edu/cs179/

https://people.maths.ox.ac.uk/~gilesm/cuda/

[Volta]

Another crap from nvidia. It seems 'highly specialized' is fanboys replacamenet to: 'it sucks'.

[drakonas777]

Threadrippers are a scam because of their positioning in the market.

Objectively not true. Such a big exaggeration is a sign that your feelings were hurt by the fact Intel's HEDT/WS segment up until recently was a complete shit show, so you have to come up with some nonsense which attacks AMD HEDT.

They are not great if you want to build system for encoding video, or editing video and/or audio because they are extremely expensive and there are more efficient and cost effective solutions for those tasks.

Some people do CPU-based en/decoding and rendering. TR is great for that. But generally I would agree with you, GPUs and fixed function offer far better value in the majority of massively parallel workloads. That's why Intel "many E core" Desktop strategy has no crucial practical meaning for the most part and that's why your proposition you made in the other thread that desktop CPUs should have hundreds of weak cores is an absolute dogshit, contradicting you own post here.

[GPTshop.ai]

... people couly build a Threadripper 7980X and put in some Instinct MI100/MI200/mi300 cards or RadeonPRO w7900
and i am pretty sure from this 47500€ you save 15000€ or more if you do so.

but i know your kind of person they believe if something is slower and more expensive they then believe they are more elite persons because they can afford the premium price.

Comparison of GH200 to alternative systems with the same amount of memory:

• Compared to 4x AMD Mi300X, GH200 costs 4x less, consumes 4x less energy and is not far off in terms of performance.
• Compared to 5x AMD Mi300A, GH200 costs 3x less, consumes 3x less energy and has at least the same performance.
• Compared to 8x RadeonPRO w7900 which has significantly less memory (only 384GB), GH200 costs the same, consumes 3x less energy and has a higher performance.