The ECC DDR4 RAM Overclocking Potential With AMD Threadripper On Linux

[phoronix]

Phoronix: The ECC DDR4 RAM Overclocking Potential With AMD Threadripper On Linux

In this guest post by Phoronix reader Max E, he has shared with us his experience and Linux benchmarking results when overclocking ECC RAM on an AMD Threadripper box. The process ended up being surprisingly easy and his results are quite compelling. Thanks to Max for this guest post, which we happily accept on Phoronix that cover interesting technical topics...

http://www.phoronix.com/scan.php?page=news_item&px=ECC-DRAM-Overclocking-TR

[tchiwam]

More of this ECC is a nice feature when you start process datasets taking days or weeks to crunch. The workaround is often to run the process twice and hope for an identical outcome on both runs.

[milkylainen]

That's some seriously nice scaling!

But I don't see why ECC should be a much harder overclock?
It's a few bits more to store the ECC...
Sure. Slightly more load and xtalk, but should be negligible if it's properly wired.
Some latency added.. you might not reach ludicrous overclocks... but still respectable ones.

[pegasus]

Key question is how does TR behave with two dimms per channel ... Epyc server boards we have claim that with 2dpc clock goes down to 2133. Haven't tried manually bumping it higher (yet) ...

Also I see that numa + openmp is mistery to many people on the benchmarking scene, including phoronix test suite. I may be motivated to collect some nice material from the hpc sites and put together some text if Michael is willing to publish it. It's high time that wider world learns how to make best use of numa machines ...

[rene]

also still need to try EEC memory on my Mini-ITC Ryzen 2700 build, … https://www.youtube.com/watch?v=jZNvj67PAEc

[MaxToTheMax]

That's some seriously nice scaling!

But I don't see why ECC should be a much harder overclock?
It's a few bits more to store the ECC...
Sure. Slightly more load and xtalk, but should be negligible if it's properly wired.
Some latency added.. you might not reach ludicrous overclocks... but still respectable ones.

Yeah, my findings are that it might not reach the same peak clocks, but getting decent clocks is trivially easy.

Much of the scaling is probably due to the higher Infinity Fabric clocks. On Threadripper systems, it's probably worth putting up with crap timings just to get higher raw clocks, for that reason. Although I didn't test this.

[ddriver]

AMD seems to be confident enough about zen 2 to even certify ddr 3200 ecc, provided of course jedec do their part and add that speed to the official specification. Otherwise it will cap at 2933 it seems. Sadly, AMD will probably once more only certify epyc and leave the consumer and prosumer workstation ecc support to the uncertainty and whims of motherboard vendors.

There is nothing intrinsically obstructing the overclock of ecc ram, other than chip binning. I've tried a number of 2400 modules and most of them seem to be able to do 2933 with just some timing adjustment, and an eventual modest voltage boost. It is just that the chips that go into ecc memory don't have to be exceptionally good, so it is just market economics to blame that ecc modules can't match the speed for the fastest consumer memory.

Another factor that prevents crazy clocks is that the industry generally doesn't care about memory corruption on consumer systems, wheras for the enterprise that is unacceptable, so clocking is a tad more conservative, as going overboard will just throw too much errors, and unlike a consumer system, those errors will show.

At any rate, decent yield dram chips will have no problem enabling 3200 ecc modules. Although the incipient 32 gig udimm modules will hardly be able to hit those clocks. But for epyc, with its 8 channel MC, it is also kinda redundant as bandwidth is ample even at lower clocks.

[schmidtbag]

That's some seriously nice scaling!

But I don't see why ECC should be a much harder overclock?
It's a few bits more to store the ECC...
Sure. Slightly more load and xtalk, but should be negligible if it's properly wired.
Some latency added.. you might not reach ludicrous overclocks... but still respectable ones.

In general, overclocking ECC is pretty counterproductive. The whole point of having ECC is to improve stability and reliability. When you overclock, you're going beyond the manufacturer's specs, therefore pushing beyond the intended limits of the product. So basically the benefit you're paying extra for is negated.
Despite all of this though, I think it's great AMD at least gives users an option, and I also think DDR4 ECC is clocked way too low.

[ddriver]

When you overclock, you're going beyond the manufacturer's specs, therefore pushing beyond the intended limits of the product. So basically the benefit you're paying extra for is negated.

Overclocking doesn't imply pushing to the limit, merely above the stock clocks. It doesn't need to introduce instability. Crucal has ecc modules rated at 2400 and 2666 that are practically identical, the only thing that differs is the timings, and not surprising in the least, they all cost the same down to the cent. It is just two SKUs of the same hardware product, so overclocking the 2400 one to 2666 is something I doubt will result in the slightest increase in instability.

There is quite a margin for ddr4 to safely overclock as many chips are already capable of doing in excess of 4000, and the only reason ecc memory is still stuck at 2666 is jedec never bothered to update their specifications, which they didn't do not because it is unfeasible, but because it is something that they haven't deemed necessary for the time being.

Power usage is also a concern, especially in high density datacenters, especially when using high capacity fb memory. Enterprise hardware, in general, aims for the optional power\performance ratio, which is also why even low core count server cpus are fairly underclocked, even if they have ample TDP budget to increase per-cpu performance.

[MaxToTheMax]

In general, overclocking ECC is pretty counterproductive. The whole point of having ECC is to improve stability and reliability. When you overclock, you're going beyond the manufacturer's specs, therefore pushing beyond the intended limits of the product. So basically the benefit you're paying extra for is negated.
Despite all of this though, I think it's great AMD at least gives users an option, and I also think DDR4 ECC is clocked way too low.

I'm not exceeding spec on the actual RAM. If anything, ECC is advantageous for overclocking, because it'll warn you if your OC isn't stable, and may actually correct errors caused by a bad OC until you can ease it back.