AMD Ryzen Threadripper 7980X & 7970X Linux Performance Benchmarks

[Alpha64]

Does your work Threadripper use an I/O die or have the memory controllers on the CCDs (1xxx, 2xxx). Asking because my (low quality) recollection was that NPS4 made a big difference in the latter case because there really were multiple memory pools with different latencies (#hops through IF) but once the I/O hub was added the memory latency was more or less consistent between banks.

Mine is a Threadripper Pro 5975WX. It is a great machine, but as I understand it, the I/O die being separate from the multiple compute dies makes the NUMA emulation actually a closer description of reality than naive SMP. I believe it probably has more to do with the caches staying hot and trying to hint the scheduler to keep programs from jumping between CCDs needlessly. But, that is just my guess.

As Micheal's newer benchmarks show on the Zen 4 Threadrippers, NPS4 seems to help these as well quite a lot - at least in code compilation workloads (which is my primary use for my Threadripper).

[bridgman]

Another potentially relevant question - do you have the UEFI set to emulate a NUMA system at all? I think it is called NPS in AMD terminology. NPS4 would probably give the scheduler the best hinting for this 4-channel RAM configuration. I have observed some kernels perform better with that set with my work Threadripper...

Does your work Threadripper use an I/O die or have the memory controllers on the CCDs (1xxx, 2xxx). Asking because my (low quality) recollection was that NPS4 made a big difference in the latter case because there really were multiple memory pools with different latencies (#hops through IF) but once the I/O hub was added the memory latency was more or less consistent between banks.

[sharpjs]

I'm considering the 7960X for a workstation build, same motherboard ( ASUS Pro WS TRX50-SAGE WiFi).
What I'm most interested in is how stable the system is under load and when idling? Any insights here would be of great help.

I have this configuration with 128GB RAM and have had no stability issues whatsoever. Currently it's running Windows, but I have at least verified that an Arch install ISO boots on it. BIOS is mostly untouched other than disabling "Armoury Crate" and (IIRC) the SATA controllers and tweaking the fan curves. The BIOS does expose a lot more settings than any I've used before.

I have the Kingston KF560R32RBEK4-128 RAM kit that is JEDEC 4800 MT/s with EXPO profiles at 5600 MT/s @1.25V and 6000 MT/s @1.35V. The 6000 MT/s profile works but the high voltage leads to 10°C higher temperatures (on which component I don't remember), which is too much for me. Right now I have it running at JEDEC but plan to do some manual RAM timing tweaks in a few months' time. Currently my attention is distracted by a Framework 16 and upcoming travel plans.

[aamwgm]

That's disappointing.

My theory is that stability and commonality go somewhat hand-in-hand. Therefore, the more esoteric the platform, the more important it is to buy from a reputable OEM/integrator who's willing to stand behind it.

I’ve been hearing more positive things about the non-Pro processors, though.

Esoterism aside, I’m still surprised at these teething issues on the Pro line. Didn’t AMD test these processors at all on Linux?

[coder]

I've asked on level1techs forum, and got this rather worrying reply

That's disappointing.

My theory is that stability and commonality go somewhat hand-in-hand. Therefore, the more esoteric the platform, the more important it is to buy from a reputable OEM/integrator who's willing to stand behind it.

[aamwgm]

I haven't had any stability issues with my extensive 7980X testing (and some 7960X) with this board.

Hi Michael,
Thank you for being so prompt. I truly appreciate your reviews.

Just a follow up question:

Have you used the system with and without load (long idle periods?) (I've asked on level1techs forum, and got this rather worrying reply), however, that is the pro line.

Thanks again, much appreciated.

[Michael]

I'm considering the 7960X for a workstation build, same motherboard ( ASUS Pro WS TRX50-SAGE WiFi).
What I'm most interested in is how stable the system is under load and when idling? Any insights here would be of great help.

I haven't had any stability issues with my extensive 7980X testing (and some 7960X) with this board.

[aamwgm]

I'm considering the 7960X for a workstation build, same motherboard ( ASUS Pro WS TRX50-SAGE WiFi).
What I'm most interested in is how stable the system is under load and when idling? Any insights here would be of great help.

[coder]

Except your correction was about my theories for performance issues.

"Theories" is too strong a word. You made some random statements that weren't rooted in data. Instead of speaking off the top of your head and then freaking out when someone calls you on it, try sanity-checking yourself. If you can't be bothered to do at least that much, then maybe it's better if you don't post it.

The whole point of me pointing out that product was to exemplify where I got my "random nonsense" from. Again, I didn't say any of them are the problem; it's possible none of them are. The thing is, history is not immune to repetition. This next-gen Threadripper is much more intelligently designed but it still has a lot of the same differences, which can basically be summarized as sharing a physical socket with a server platform but everything is cut down.

Name-dropping another product is not supporting your argument. You need to actually make the case, on the basis of what issues it had and why you believe those same issues are present in the new platform & processors. Again, if you can't be bothered to do that much, then maybe you're not helping?

At this point I'm curious how many times you're going to say something along the lines of "I don't have time for this" and yet respond anyway. You think you're winning here but you're going against your own word.

There's no winning. I know every minute I spend reading or replying to you is lost.

It's genuinely strange to me how you can honestly dismiss all of my ideas as unsupportable,

I didn't say they're all unsupportable, but they are unsupported. That's because for all your willingness to type pages of text at me, you can't be bothered to look at the data and make a clear, logical, and data-driven argument for any of them. Without that, they're worthless.

That dual-socket Epyc has triple the core count of a 7980X; that's a lot of additional bandwidth needed for the disk. The SN850 is no slouch of a SSD.

Which data are you citing? I'm not aware of a test involving the EPYC 9654 which used the SN850. ServeTheHome didn't say what storage they used, but you can bet it wasn't a SN850. In his latest round of EPYC vs. Xeon tests Michael didn't use it, either.

I explained why I think the SN850 might've become a bottleneck which doesn't involve it being a "slouch" for its intended client workloads. For now, it's still conjecture. Needs more data.

Anyone genuinely interested in solving this problem and not suffering from a superiority complex would acknowledge that some (not all) of the things I and Alpha64 mentioned are plausible.

No, because even the data we have doesn't support some of them. You seem like someone who makes a lot of excuses. Without supporting data, that's all they are. Excuses are not actionable. Unless the actual cause is found, it cannot be addressed.

I don't really care if I'm wrong;

That much is clear.

I acknowledged there are other possibilities and I didn't try to create an exhaustive list. What I care about is how you're judging me because your autistic brain can't handle more than one possible problem at a time, or, perhaps your ego can't accept there are other possibilities that you didn't come up with.

You apparently can't handle any problems at a time. Compared to that, focusing on one at a time is an infinite improvement.

And no, not a single one of your excuses is news to me. You probably don't have a clue when & where at least half of those effects would come into play, much less how to quantify them. What good is that, then? Before appealing to what you can't characterize or measure, why not first focus on the things you can?

Oh, right. Because you don't care if you're wrong. So, tell me: what value is there is posting wrong answers?

You kinda did with the whole I/O thing...

I presented an argument as an appeal for more data, which is incumbent on Michael taking additional action. If I couldn't make a case for it, then why would he even bother?

I'm not obligated to devote time to help some stranger satisfy his/her curiosity. I pitched in my two cents because that's how much I care to give. It's not nothing,

Nobody is asking you to help. Posting wrong answers is worse than nothing at all. If you're going to try and answer a question, then you should at least do a basic level of sanity-checking to avoid leading someone down the wrong path. There's enough half-assed shit in the world, as is.

what are you going to do if compiling benchmarks are still too slow after a better drive is swapped in?

I already said.

That whole sentence is hypocritical.

Not sure you know what that means, but whatever.

I'm here because I care about the mission of Linux benchmarking. If there's indeed a systemic problem with Michael's test methodology, maybe it's something we can address.

[coder]

The compiles I run benefit from multicore up to a point, but then are IO- or frequency-dependent. That also tracks with my other workload, which is large database doodling. That’s why I opted for “just” the 7960X. There’s a balancing act between core count and frequency and IO, and for me it makes sense to maximize frequency within the Threadripper range, all of which have enough cores.

Yeah, I guess the main benefit of the 7960X is double the memory channels of the 7950X. The tradeoff is you give up a little frequency for that + the 8 extra cores. I guess you also get 33% more L3 per core, since it seems like they made it by using 4 CCDs with only 6 cores enabled on each.

For me, even all of that would be enough to justify the higher cost. However, if you need the additional PCIe lanes, that could basically force your hand.

I do hear you on the frequency point. Much of the time, I'm doing small incremental builds, where I really want decent single-thread performance more than a ton of threads.

BTW, if you've got the budget for a ThreadRipper, I sure you you're using a datacenter grade SSD and not consumer trash. That should help a lot with your database I/O, as well as minimize the chance of I/O-related build bottlenecks (but then, so should adequate RAM). If you want to learn more, start here:

https://www.storagereview.com/enterprise/ssd