Intel Xeon Max 9480/9468 Show Significant Uplift In HPC & AI Workloads With HBM2e

[phoronix]

Phoronix: Intel Xeon Max 9480/9468 Show Significant Uplift In HPC & AI Workloads With HBM2e

Today is a very fun and interesting round of benchmarking... Recently Supermicro sent over their Hyper SuperServer SYS-221H-TNR and Intel supplied the Xeon Max 9468 and Xeon Max 9480 for finally being able to benchmark Xeon Max processors, the Sapphire Rapids parts featuring 64GB of HBM2e memory. For this initial benchmarking article is a look at the Xeon Max 9468/9480 dual socket performance when running in HBM-only mode and HBM-caching mode for showing some of the workloads where Xeon Max can deliver significant uplift compared to when running in flat (1LM) mode without assigning anything to the HBM memory for seeing the impact when the specialized memory goes unused.

https://www.phoronix.com/review/xeon-max-9468-9480-hbm2e

[brucethemoose]

llama.cpp doesn't support AMX, but I was still hoping to see it in a benchmark on these things. They could be interesting LLM nodes.

The 9462 in particular is perfect, as llama.cpp doesn't linearly scale with thread count, and a 64GB pool is just right for a 65B model.

I guess it doesn't really matter, as Xeon Max will be sold/priced for HPC use anyway, right?

[qarium]

impressive demonstration that DDR5 is maybe not the future. for servers this 64gb HBM2e looks little

but for desktop and notebooks thats more than the most people need.

i just image how a steam deck would perform if the ram would not be SODDR5 instead it would be HBM2e..

[sophisticles]

Michael

This is an excellent article, one of the better ones I have seen on this site in a while. It's so good that had i not been laid off in March, i would buy a lifetime subscription.

What I did do what was read through the article with my usual Firefox browser with ad blocking enabled and then read it again with Chrome and no ad blocker.

If i can give you some feedback, as a guy that also has some websites, the ads you serve your readers and their placement is very obtrusive and they are poorly formatted and tend to obscure large parts of the page, specially if you scale the page either larger or smaller than 100%.

You are shooting yourself in the foot, in an effort to make more money from ads you have created a situation were you are basically forcing your readers to use an ad blocker in order to be able to read the articles.

I think if you posted an article like the one you normally post adverting the subscriptions, but this time make a different offer. You agree to only include 3 unobtrusive, properly formatted ads per page and we agree not to use ad blockers when visiting your site as a way of supporting your work.

I think you would find your ad revenue significantly jump, because i have to tell you that your site is unreadable without an ad blocker.

[coder]

I know the focus of the article is HPC and AI, but I'm always curious what exotic memory configurations do for more mundane (but still highly-parallel) things like kernel compilation. Perhaps 64 GB wouldn't be enough for so many cores. Might start swapping, if HBM-exclusive mode is used.

[Michael]

I know the focus of the article is HPC and AI, but I'm always curious what exotic memory configurations do for more mundane (but still highly-parallel) things like kernel compilation. Perhaps 64 GB wouldn't be enough for so many cores. Might start swapping, if HBM-exclusive mode is used.

Not much for compilation workloads... And yeah some compilation workloads fail at less than 2GB per make job. So OOM / failures for some of the larger codebase tests.

[JEBjames]

Michael

Maybe include the 9468 price ($9,900) in the article? Save my poor eyes from having to squint at the pricing picture.

Great article.

Crazy to see the entire working set of all these benchmarks being essentially in-cache.

[coder]

MichaelMaybe include the 9468 price ($9,900) in the article? Save my poor eyes from having to squint at the pricing picture.

Yes, it is.

And you can always open the image in a new tab and use your browser's zoom feature (usually Ctrl-+/-) to see it larger.

[JEBjames]

Michael

Sorry, I wasn't clear.

I meant I wished you had it in the article as text "$9,900" vs. having to look at the picture at all.

Again, excellent article.

[Michael]

Michael

Sorry, I wasn't clear.

I meant I wished you had it in the article as text "$9,900" vs. having to look at the picture at all.

Again, excellent article.

I didn't bother since I don't know how accurate it is... That was the launch price six months ago. So far I haven't seen Xeon Max in any Internet retailers... I know some folks that have been trying to buy them since launch and still haven't had any luck. So long story short, not sure how accurate the price is or the current availability.