A 20 Year Old Chipset Workaround Has Been Hurting Modern AMD Linux Systems

[Radtraveller]

This was obviously found by someone doing workload profiling and then going on a hunt for the offending "dummy reads", as indicated in the patch:

"Sampling certain workloads with IBS on AMD Zen3 system shows that a significant amount of time is spent in the dummy op"

The way you phrased it makes it sound like the issue was discovered through code inspection.

linux suffers irritable bowel syndrome on some versions of bowels…(?)

not sure why, I am not the coder type, a “modern” distro that limits to hardware no older than say… 2 years wouldn’t be popular enough to garner dev and users.
I am still attempting to build a kernel without a bunch of stuff that doesn’t look needed…. Until something fails.. oh well.. I know I am an amateur, but it is fun and a learning experience. Lots to read and comprehend.

[filbo]

Great find by the AMD engineer, i hope this lands in 6.0!

Interestingly, the patch which is actually enqueued for inclusion is by an Intel engineer (link near the bottom of Michael's article). That patch's writeup characterizes it as an old Intel bug which is being cleaned up to the benefit of AMD.

So, kudos to Dave Hansen -- and to Intel for apparently fostering a work environment where Dave feels empowered to issue patches aimed squarely at improving Enemy Number One's product line...

[ryao]

Yeah, you would need to run a kernel profiler, and who does that?

As someone who does that, I can tell you that this does not appear in it. Profiling the kernel is typically done to see what is done when the CPU is busy, not how much it is in power states. You would not see this in a typical perf profile like the kind most kernel developers use.

[Jahimself]

I find this fascinating that someone discovered this after so many years and managed to fix it!

[yump]

To me it didn't come off sounding like dumb luck to have found it. Are there even tools for profiling all nooks and crannys of the code to find these kinds of things?

Perf can collect events system-wide with -a, but it sounds like this was discovered with something called "instruction-based sampling", which sounds pretty cool in the whitepaper. IDK if profiling with usual cycles or instructions PMCs would've found it.

There is apparently support for it in perf-events, but it may not be wired up to be as user friendly as PMC sampling. Unfortunately I don't have any recent enough AMD hardware to play with it. There may be something useful in this recent patch or the discussion around it, or more likely this thread from when the feature was first added. The perf tools seem to change a lot though, and the documentation around how they should be used is not great. I didn't even know about event modifiers or "precise levels" until just now.

...actually, it looks like you can just

perf top -a -e cycles:P

and get full-system profiling with the maximum precision your hardware is capable of. Nice! Most userspace programs don't have debug symbols, though, and I've not had great luck getting perf to use debuginfod.

[yump]

As someone who does that, I can tell you that this does not appear in it. Profiling the kernel is typically done to see what is done when the CPU is busy, not how much it is in power states. You would not see this in a typical perf profile like the kind most kernel developers use.

Yeah, I would really like to see details of how this was found.

[coder]

linux suffers irritable bowel syndrome on some versions of bowels…(?)

No, IBS refers to Instruction-Based Sampling, which is a hardware-assisted profiling mechanism.

https://www.phoronix.com/news/Linux-6.0-Zen4-IBS-Perf-Tools

[ATLief]

Does anyone know if the "performance" governor has this problem? The way it's described it sounds like a schedutil thing (or whatever the older version is called).

[yump]

Does anyone know if the "performance" governor has this problem? The way it's described it sounds like a schedutil thing (or whatever the older version is called).

This is unrelated. Schedutil is (one of) the CPU frequency governor(s). The governor this is confusing is the CPU idle governor. cpufreq says how to work slower. cpuidle says what to do when there is literally nothing to do.

[CochainComplex]

Is no one else going to say it? Ok.

...They should rewrite the Linux kernel in Rust.

Since the Kernel has some experimtal support for Rust written drivers, it might be possible in the far future. I dont know how fissible it is, but maybe rewriting parts/modiles peu a peu could transform the kernel in the longrun. And rewriting those parts would force people to completly reassess existing code. Albeit not breaking compatibly at once. On the other hand maybe some new concepts wouldn't be incorporated to the code base due to obvious incompatiblities.