Originally posted by Anux
View Post
AMDGPU Linux Driver No Longer Lets You Have Unlimited Control To Lower Your Power Limit
Collapse
X
-
Originally posted by Anux View PostHow so? A continous curve is nothing more than infinite F/V-pairs. If any of those infinite F/V-pairs destroys your card how do they make sure this doesn't happen with some random workload that uses this F/V-pair at standard TDP? I could limit my games FPS from 312 to 1 and achive any possible TDP far below standard TDP.
And what about booting with amdgpu.ppfeaturemask=0xffffffff or any other combination? If that is allowed without kernel recompile there is no argument for limiting the min TDP.
No, as long as there is amdgpu.ppfeaturemask, overclocking, fancontrol and under volting it's not. Heck I could wrap my card in a towel and call AMD support to tell them about my over heating issues.
Yes they can do that, if we get our control back it's totally fine by me. Taint the kernel if there is an altered UEFI setting, taint it if dual boot is detected, taint it on full moon, I couldn't care less.
1. Lower and upper power limits are obeyed and read-only without the overdrive bit set in amdgpu.ppfeaturemask, along with the sysfs files that change voltage/frequency relations.
2. With the overdrive bit set, they are writable.
3. If any of them are written, that tains the kernel.
The reason for tainting only on write is that I have amdgpu.ppfeaturemask in the ansible playbook I run on all family machines, but changing that would only be a small hassle.
IIRC, the status quo is that the overclocking files are writable but do nothing without the overdrive bit, which is... less than optimal I think.
No it doesn't. For fixed frequency under volting decreases current. And if power management selects a higher state, than that's not under volting and it would still only stay at roughly the same current we had before under volting. That is how we get higher performance at same TDP with under volting.
And when power management selects a higher frequency, it will do so constrained by the power limit. P=IV, so current increases. It is not roughly the same current.
Sorry for being a bit too sarcastic. :/ Until recently there was no doubt that my next GPU would be an AMD card, now that I know I couldn't use it in Windows (there's no way I'm running a card above 150 W for extended periods) I have lost faith in them and am seriously considering Intel GPUs.
I think I got their memo, they hate power efficiency and custom builds and want us to buy competitor cards. Intel seems to have no problem going down to 95 W in Win: https://www.kitguru.net/wp-content/u...-oc-scaled.jpg and I noticed they are really working hard to make their drivers good/usable. Maybe 2 more generations and we won't ever have to think about radeon again.
I can only hope this doesn't spread to their CPU lineup.
It shouldn't be forgotten how much Valve, Collabora, et. al. have contributed to FOSS support for AMD graphics, or that when AMD was working alone what we got was... Catalyst.
Originally posted by DumbFsck View Post
Correct. I guess what I meant in the first reply is more in the sense that I think I never in my life heard anyone talk about amperes in this context. Or as I stated somewhat inflammatory, we wouldn't say like he did that current has to increase, we'd say that at iso power and lower voltage f or C is higher, or both (so I guess an example would be that we would see GPU usage being higher at the same tdp and for approx. same amount of work [which could be frames or hashes or w/e]).
Almost certainly it's f that varies. The power management controller could theoretically tell the scheduler to idle CUs or parts of CUs, but I'm pretty sure reducing frequency (and voltage, because its a lookup table or parametric curve) saves more power per unit performance lost unless you've set a really low power limit on a large chip.
The question I have now is: is it still possible to limit tdp, but in a less transparent way?
People in the bug report mentioned you can set SCLK, after talking to you guys I looked the documentation and there apparently you can set VDD curve, pp tables and acrivate/deactivate performance levels, and set manual performance levels.
I don't know how fast the PMU firmware's power control loop is, but a userspace software loop would certainly be fast enough for efficiency/noise control (you don't want to go sub-frame anyway) and certainly faster than the thermal inertia of the heatsink. There's no need to worry about overheating the chip because you'll always be backstopped by the regular power and thermal limits.
It would be less reliable than the firmware, however, and I've had a lot of trouble getting my RX 580 to go into really deep power saving states (below D0) if anything is poking around with the sysfs files. I have to disable all of corectrl's monitoring, and my own system fan control daemon goes hands off the GPU if it sees utilization dip below a threshold.
Comment
-
-
Originally posted by yump View PostSuppose the parameters are a high-order polynomial or something.
And when power management selects a higher frequency, it will do so constrained by the power limit. P=IV, so current increases. It is not roughly the same current.
Comment
-
-
On my RDNA2 card (RX 6650XT) the power limit just makes the driver decrease the frequency/clocks until the power usage is below the limit. The core clock on my card can go from 700 to 2765 MHz by default. The default power limit for my card is 164 W, so when playing a game the driver will increase the frequency as high as it can until the power usage hits 164 W. Heavier games max out the 164 W with a lower frequency (e.g. 2600 MHz) while lighter games will run at the max 2765 MHz while only consuming e.g. 130 W. FurMark can even get to 164 W at just 2400 MHz. If I set the power limit to e.g. 130 W, the maximum frequency will be lower, in the exact same way as if I've set a lower max frequency through OverDrive.
The dynamic adjustment of the frequency to stay below the power cap is completely normal (default!) driver behavior. There are no out-of-spec changes to voltages or currents with decreased power limits; in contrast to under/overvolting, overclocking, and increased power limits! And if you set the power limit at something ridiculous like 0 W, then the driver will just keep the clocks to the minimum allowed without any issues (actual power usage then is e.g. 40 W).
AMD is probably just lying about "potential hardware damage" to protect future GPU sales, as a -20% power cap will significantly extend hardware life in exchange for a minor performance loss.
For those who want a workaround without recompiling the kernel: you can set your maximum shader clock a few hundred MHz lower than default through OverDrive, but you'll lose a bit of performance on light games that would normally max out the clocks while staying at lower power usage.
Comment
-
-
Originally posted by Anux View PostI'm not sure what the exact shape of the curve has to do with anything, my argument doesn't change even if it was a linear function. You could represent any curve with infinite F/V-values.
Like, it's conceivable that for frequencies << typical furmark@TDP, applied voltage might actually go up as frequency goes down, because the manufacturer didn't care to tune that part of the curve and it only ever experiences near-zero utilization. Perhaps power-vs-frequency is actually non-monotonic if you go low enough, and perhaps the power limit control loop goes nuts on some cards.
But maybe it's not that kind of curve; I don't know. I don't have anything newer than Polaris, and until/unless this matter is resolved I'm not inclined to chuck any money AMD's way.
You need to take into account that the frequency changes if you select a higher F/V pair. That changes the resistance (impedance) of the silicon (given a fixed workload) which in turn limits the current at a given voltage. Remember the formula DumbFsck posted above.
Comment
-
-
Originally posted by Anux View PostI'm not sure what the exact shape of the curve has to do with anything, my argument doesn't change even if it was a linear function. You could represent any curve with infinite F/V-values.
You need to take into account that the frequency changes if you select a higher F/V pair. That changes the resistance (impedance) of the silicon (given a fixed workload) which in turn limits the current at a given voltage. Remember the formula DumbFsck posted above.
That simply isn't true.
There are components inside the GPU beyond just the shader cores that need to run at specific speeds at all times to do their jobs. The display engines have to run at a certain speed in order to push pixels out through the display ports fast enough. The memory controller needs to run at a high enough speed to adequately feed framebuffer data to display engine and keep the graphics memory alive. The video decoder/encoder block has its own specific clock domain and speed it runs at separate from everything else. The PCI-E PHY has a specific speed it needs to run on to stay synchronized and active on the PCI-E bus.
Plus, most (if not all) modern GPUs use clock & power gating to achieve their idle clocks. So when you check your GPU stats at idle, and you see "75 MHz, 9 watts" - that's not actually true. The shader cores are waking up, doing some work at a much higher speed (say 400 MHz) at a high power (say 25w), and then going back to sleep, and then power-gated off. So over a short time interval (say 1-2 seconds), the average speed and power will be 75Mhz and 9w - but the instantaneous will be far higher.
Since those components all have fixed speed (or strict minimum speeds) they need to run at it to ensure stability and basic functionality of the card - that means they have minimum voltages that HAVE to be met. That by definition means the card has a bare minimum lower power limit. This power limit can very based on specific card configuration, silicon bin, and perhaps even accessories. I know on nVidia cards, things like fans and RGB lights can count towards card power limits.
So if the card is programmed with a parametric/equation-based F/V curve, this can lead to severe issues if you allow users to set arbitrary power limits. Let's say your card idles at "9w" reported. You want it lower, so you set the power limit to 7w. System immediately crashes, as the display engine and memory controller have hardware minimum clock limits. The driver is forcing the 7w power limit, so the VRM has no choice but to force the core voltages below the F/V curve, and the display engine crashes.
Another more realistic example: The card idles at "9w" according to the software telemetry. In basic desktop use, you never see the power consumption go above 15w. You decide you want to cap the card to "15w". You set the limit, everything seems fine. The GPU core has gone to sleep since nothing is changing on the screen. The display engines are just running in a repeat/auto-refresh mode since nothing new needs to be rendered. You wiggle the mouse, and suddenly the GPU crashes. The GPU core woke up to render your mouse movement. The shaders have a minimum clock speed they have to operate at. The GPU immediately plowed into the 15w TDP limit in a manner of microseconds, the VRM immediately cratered the vCore trying to meet the 15w limit imposed by the driver, and then the GPU core crashed due to low voltage.
Remember, the power readings, temperatures, core speeds, etc... you are reading through the OS are just averaged and filtered values reported by the driver. At the actual hardware level, the power, temperatures, clock speeds, etc... are being sampled hundreds, if not thousands of times per second.
The main impetus for this, is that people will set power limits too low, and will crash GPUs. While it seems unlikely, I believe that a driver-power-limit-imposed-momentarily-voltage-hiccup could cause "SCR latchup" inside the core. That's a situation where transistors that would never normally be on at the same time, could both be triggered, causing a short circuit.
Yeah losing some control over the hardware stinks, but the reality is that being able to adjust GPU power limits at all through software is still a "relatively" new thing. It wasn't that long ago the only thing you could adjust was clocks and voltage - and even that wasn't always a given.Last edited by AmericanLocomotive; 07 March 2024, 10:57 PM.
Comment
-
-
[QUOTE=bosslog;n1448693
AMD is probably just lying about "potential hardware damage" to protect future GPU sales, as a -20% power cap will significantly extend hardware life in exchange for a minor performance loss.
That is a very far fetched statement, having a card die of old age isn't common....at all, even on normal power limits, and even on overclocked cards.....
And even if that was true, lets do some napkin math :[LIST][*]linux is 4% of the OS market share[*]AMD is less than 25% of the GPU market (even less if we put the people only having iGPU in the equation)[*]probably less than 5% of the users will ever reduce the power limit (I'm being extremely generous here)[/LIST]so we're very optimistically talking about 0.05% of GPU market potentially having a longer lasting hardware, and I won't even try to estimate how many of these will actually wait until their GPU dies out to replace it, because that number is probably way too close to zero to make sens
So, AMD made this choice to have a, in absolute best case scenario where everyone wait until their card dies and buy again an AMD card increase in sales of a whooping 6350 Cards per year (out of the 12 700 000 dGPU sold per year total).
So, no, they didn't do this for the increase in sale.
Comment
-
-
AmericanLocomotive I'm pretty sure the power limit acts over much longer timescales than that, based Igor's power spike data. https://www.igorslab.de/en/radeon-rx...0xt-review/11/
There are physical limits to how quickly VRM output voltage can be adjusted, and it would be extremely weird for a power limit to be used as anything other than an outer control loop shrinking the allowed range of frequency selection. If it could extend the range below Vmin, or force unsafe combinations of frequencies from different domains, then you'd have to validate the power limit against all possible workloads. Better to write the firmware so that it does not do that, and you can list the constraints to be obeyed explicitly.
Intel CPUs, for example, will violate the power limit to maintain base clock, but they won't violate the thermal limit to do that. And if they get all the way down to the minimum frequency (800 MHz) and are *still* overheating, then they switch to clock modulation instead (forced idle-injection on a duty cycle). But at no point will they attempt to run below fused minimum frequencies and voltages. They'll emergency shutdown first.
Yeah losing some control over the hardware stinks, but the reality is that being able to adjust GPU power limits at all through software is still a "relatively" new thing. It wasn't that long ago the only thing you could adjust was clocks and voltage - and even that wasn't always a given.
Comment
-
-
Originally posted by yump View Post
Amperes are very important in the context of reliability, and amperes are what cause stress on the VRM.
I honestly didn't think it was something to consider with cpu/gpu design, as I though that due to the infinitesimal area, the effects on velocity would already negate all of it.
But as I said, all I know about it is dodgy murky misremembering. The other day after reading you guys' comments I realised I didn't even remember wtf was a S. Just for you to see how little remains in my head about the subject.
Also I can't even remember exactly what velocity entails. And somehow I though that with semiconductors being in a lattice structure it would mess up a lot of migration.
Originally posted by yump View PostAlmost certainly it's f that varies. The power management controller could theoretically tell the scheduler to idle CUs or parts of CUs, but I'm pretty sure reducing frequency (and voltage, because its a lookup table or parametric curve) saves more power per unit performance lost unless you've set a really low power limit on a large chip.
As in, I thought that due to the longer time that it would take to "fill up" a capacitor with the lower voltage, the C would have to rise at iso f.
E.g. To game at nnnfps frame limited in game, cenario A with a tdp cap of 100 and cenario B with a tdp cap of 50, if the voltage is dropped for cenario B (as it should, since it has a disproportional impact in tdp) both could still boost to fmax, but A would show 50% utilisation and B would show 100% (I think in gpu_busy)(every number not related to reality, not even their relationships, cause of course I don't think halving power requires doubling activity. It's just an illustration)
As per your latest comment in response to American locomotive. A larger area of the die would have to be active at any given time for it to reach the max clock.
____---_____
s_j_newbury I'd think using dpm to create and set a mode in which the gpu simply can't reach a tdp above what the user wants would be the way they "support". Not guessing internal state from userspace. It shouldn't be needed, right?
Comment
-
Comment