NVIDIA GPU sticks to P0 unless explicitly used once (server, headless)

[panni]

Hey,

I've recently switched from a GT710 to a GTX1660 on my Ubuntu Server (22.04.1 LTS) to enable turing hw acceleration in Plex Media Server.

After playing with multiple driver versions I've noticed a bug that's 100% reproducible (tested with nvidia-driver-470, nvidia-driver-515-server and nvidia-driver-515):
Unless the GPU is at least used once after a reboot, its power state stays at P0 (~20-22W) and does not drop.

Un-stickying this power state can be achieved by (obviously) utilizing the GPU in Plex, running nvtop, or adding a poor-man's approach to the startup sequence (in my case rc.local), such as:

ffmpeg -y -vsync 0 -hwaccel cuda -hwaccel_output_format cuda -f lavfi -i testsrc=duration=1 -t 50 -vf hwupload -c:a copy -c:v h264_nvenc -b:v 4M -f null -

This then results in the proper GPU state (P8 in this case):

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 515.65.01 Driver Version: 515.65.01 CUDA Version: 11.7 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|===============================+================= =====+======================|
| 0 NVIDIA GeForce ... Off | 00000000:09:00.0 On | N/A |
| 0% 45C P8 10W / 130W | 3MiB / 6144MiB | 0% Default |
| | | N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes: |
| GPU GI CI PID Type Process name GPU Memory |
| ID ID Usage |
|================================================= ============================|
| No running processes found |
+-----------------------------------------------------------------------------+
​

Maybe this helps users who wonder about their power utilization after a reboot, who rarely use their GPU.


Best regards,

panni