EDIT: it turns out the second part of this patch is not working (sclk modification), instead please downclock by patching the VBIOS with PolarisBiosEditor and reflash.
WARNING: updating your VBIOS works fine in Linux, but the Windows driver will fail to load! (Windows falls back to non accelerated VESA driver) because the signature is invalid. So make sure you keep a backup of your old VBIOS!
Here's a short writeup that explains how you can underclock and undervolt your card, by patching the amdgpu kernel driver. As I explained in another post, undervolting leads to massive gains in consumed power at a small performance cost. For example, by underclocking to 1055 MHz and undervolting to 818mV, luxmark consumes 40% less power (at the wall), but the score is just 5% lower.
The first step is to determine your sweet spot of desired underclock and undervolt with wattman under Windows. Make sure to also set the memory voltage to your target limit, as the default value of 1000mV will override any voltage that is set for the graphics engine. I have tested with several Sapphire RX 470 (the reference design, blower style) that the 1055 MHz / 818mV is very stable, even for hours. Some can handle up to 1080 MHz, but it's best to have a safety margin since as the chip gets hotter it will require more juice due to increased leakage.
So first you need the edit polaris10_smc.c and add the relevant lines of code (lines 114 and 772). This file resides under /usr/src/amdgpu-pro-YOURVERSION/amd/powerplay/smumgr/polaris10_smc.c:
Note the undervolt above (818mV) and the underclock (13% which leads to the maximum frequency being 1055 MHz). Change those values according to your card's sweet spot.
Recompile the kernel, copy the module and rebuild the initramfs:
Make sure to replace 4.4.0-53 above with your actual kernel version, and 16.50-362463 with your actual amdgpu version.
After rebooting with the new module, verify the new frequencies are correctly set before running any benchmark:
In addition, you may want to set a higher value for the fan pwm, to keep temperatures low and so have less current leakage:
I have tested the changes above and they appear to work safely. However be very careful when tweaking voltages, for example consider this assert in the code:
Finally, for the reference I'm going to post the kernel log that was generated when I was trying to figure out how to implement underclocking/undervolting by adding lots of printk everywhere; it could be useful to others:
WARNING: updating your VBIOS works fine in Linux, but the Windows driver will fail to load! (Windows falls back to non accelerated VESA driver) because the signature is invalid. So make sure you keep a backup of your old VBIOS!
Here's a short writeup that explains how you can underclock and undervolt your card, by patching the amdgpu kernel driver. As I explained in another post, undervolting leads to massive gains in consumed power at a small performance cost. For example, by underclocking to 1055 MHz and undervolting to 818mV, luxmark consumes 40% less power (at the wall), but the score is just 5% lower.
The first step is to determine your sweet spot of desired underclock and undervolt with wattman under Windows. Make sure to also set the memory voltage to your target limit, as the default value of 1000mV will override any voltage that is set for the graphics engine. I have tested with several Sapphire RX 470 (the reference design, blower style) that the 1055 MHz / 818mV is very stable, even for hours. Some can handle up to 1080 MHz, but it's best to have a safety margin since as the chip gets hotter it will require more juice due to increased leakage.
So first you need the edit polaris10_smc.c and add the relevant lines of code (lines 114 and 772). This file resides under /usr/src/amdgpu-pro-YOURVERSION/amd/powerplay/smumgr/polaris10_smc.c:
Code:
--- amd/powerplay/smumgr/polaris10_smc.c.orig 2016-11-30 03:02:00.000000000 +0100 +++ amd/powerplay/smumgr/polaris10_smc.c 2016-12-16 13:34:18.325472304 +0100 @@ -114,6 +114,7 @@ VOLTAGE_SCALE; *voltage |= 1 << PHASES_SHIFT; + *voltage = (*voltage & 0xFFFF0000) + 818*VOLTAGE_SCALE; return 0; } } @@ -772,6 +774,9 @@ for (i = 0; i < dpm_table->sclk_table.count; i++) { + int clk = dpm_table->sclk_table.dpm_levels[i].value; + dpm_table->sclk_table.dpm_levels[i].value -= (13 * clk) / 100; + result = polaris10_populate_single_graphic_level(hwmgr, dpm_table->sclk_table.dpm_levels[i].value, (uint16_t)smu_data->activity_target[i],
Recompile the kernel, copy the module and rebuild the initramfs:
Code:
cd /usr/src/amdgpu-pro-16.50-362463 ./pre-build.sh 4.4.0-53-generic make KERNELRELEASE=4.4.0-53-generic -C /lib/modules/4.4.0-53-generic/build M=/usr/src/amdgpu-pro-16.50-362463 find /lib/modules/4.4.0-53-generic -name amdgpu.ko -exec rm -v {} \; cp /usr/src/amdgpu-pro-16.50-362463/amd/amdgpu/amdgpu.ko /lib/modules/4.4.0-53-generic/kernel/drivers/gpu/drm/amd/amdgpu/amdgpu.ko depmod -a update-initramfs -u
After rebooting with the new module, verify the new frequencies are correctly set before running any benchmark:
Code:
cat /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0/pp_dpm_sclk 0: 171Mhz 1: 266Mhz 2: 430Mhz * 3: 583Mhz 4: 615Mhz 5: 645Mhz 6: 669Mhz 7: 1057Mhz
Code:
echo 140 >/sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0/hwmon/hwmon0/pwm1
Code:
/* need to make sure vddgfx is less than 2v or else, it could burn the ASIC. */ PP_ASSERT_WITH_CODE((vddgfx < 2000 && vddgfx != 0), "Invalid VDDGFX value!", return -EINVAL);
Code:
ATOM BIOS: D00003 [drm] GPU posting now... amdgpu 0000:01:00.0: VRAM: 4096M 0x0000000000000000 - 0x00000000FFFFFFFF (4096M used) amdgpu 0000:01:00.0: GTT: 4096M 0x0000000100000000 - 0x00000001FFFFFFFF [drm] Detected VRAM RAM=4096M, BAR=256M [drm] RAM width 256bits GDDR5 [TTM] Zone kernel: Available graphics memory: 16303650 kiB [TTM] Zone dma32: Available graphics memory: 2097152 kiB [TTM] Initializing pool allocator [TTM] Initializing DMA pool allocator [drm] amdgpu: 4096M of VRAM memory ready [drm] amdgpu: 4096M of GTT memory ready. [drm] GART: num cpu pages 1048576, num gpu pages 1048576 [drm] PCIE GART of 4096M enabled (table at 0x0000000000040000). amdgpu 0000:01:00.0: amdgpu: using MSI. [drm] amdgpu: irq initialized. phm_get_sclk_for_voltage_evv before assert: sclk=0 virtual_voltage_id=65281 entryId=8 voltageId=7Can't find requested voltage id in vdd_dep_on_sclk table! phm_get_sclk_for_voltage_evv before assert: sclk=0 virtual_voltage_id=65282 entryId=1 voltageId=1 phm_get_sclk_for_voltage_evv after assert: sclk=46600 virtual_voltage_id=65282 entryId=1 voltageId=1 phm_get_voltage_evv_on_sclk (polaris10): 818, ret=0 phm_get_sclk_for_voltage_evv before assert: sclk=51600 virtual_voltage_id=65283 entryId=2 voltageId=2 phm_get_sclk_for_voltage_evv after assert: sclk=75100 virtual_voltage_id=65283 entryId=2 voltageId=2 phm_get_voltage_evv_on_sclk (polaris10): 824, ret=0 phm_get_sclk_for_voltage_evv before assert: sclk=75100 virtual_voltage_id=65284 entryId=3 voltageId=3 phm_get_sclk_for_voltage_evv after assert: sclk=101900 virtual_voltage_id=65284 entryId=3 voltageId=3 phm_get_voltage_evv_on_sclk (polaris10): 850, ret=0 phm_get_sclk_for_voltage_evv before assert: sclk=101900 virtual_voltage_id=65285 entryId=4 voltageId=4 phm_get_sclk_for_voltage_evv after assert: sclk=107400 virtual_voltage_id=65285 entryId=4 voltageId=4 phm_get_voltage_evv_on_sclk (polaris10): 893, ret=0 phm_get_sclk_for_voltage_evv before assert: sclk=107400 virtual_voltage_id=65286 entryId=5 voltageId=5 phm_get_sclk_for_voltage_evv after assert: sclk=112600 virtual_voltage_id=65286 entryId=5 voltageId=5 phm_get_voltage_evv_on_sclk (polaris10): 937, ret=0 phm_get_sclk_for_voltage_evv before assert: sclk=112600 virtual_voltage_id=65287 entryId=6 voltageId=6 phm_get_sclk_for_voltage_evv after assert: sclk=116900 virtual_voltage_id=65287 entryId=6 voltageId=6 phm_get_voltage_evv_on_sclk (polaris10): 981, ret=0 phm_get_sclk_for_voltage_evv before assert: sclk=116900 virtual_voltage_id=65288 entryId=7 voltageId=7 phm_get_sclk_for_voltage_evv after assert: sclk=121600 virtual_voltage_id=65288 entryId=7 voltageId=7 phm_get_voltage_evv_on_sclk (polaris10): 1025, ret=0 amdgpu: powerplay initialized [drm] Found UVD firmware Version: 1.79 Family ID: 16 [drm] Found VCE firmware Version: 52.4 Binary ID: 3 [AVFS] Something is broken. See log! atomctrl_get_voltage_table_v3: i=0 value=850 smio_low=1048576 atomctrl_get_voltage_table_v3: i=1 value=900 smio_low=2 atomctrl_get_voltage_table_v3: i=2 value=950 smio_low=1048578 atomctrl_get_voltage_table_v3: i=3 value=1000 smio_low=1081344 smu7_setup_dpm_tables_v1: i=0 sclk=30000 smu7_setup_dpm_tables_v1: i=1 sclk=46600 smu7_setup_dpm_tables_v1: i=2 sclk=75100 smu7_setup_dpm_tables_v1: i=3 sclk=101900 smu7_setup_dpm_tables_v1: i=4 sclk=107400 smu7_setup_dpm_tables_v1: i=5 sclk=112600 smu7_setup_dpm_tables_v1: i=6 sclk=116900 smu7_setup_dpm_tables_v1: i=7 sclk=121600 smu7_setup_dpm_tables_v1: i=0 mclk=30000 smu7_setup_dpm_tables_v1: i=1 mclk=175000 phm_get_voltage_index: voltage=800 -> index=0 phm_get_voltage_index: voltage=818 -> index=1 phm_get_voltage_index: voltage=824 -> index=2 phm_get_voltage_index: voltage=850 -> index=3 phm_get_voltage_index: voltage=850 -> index=3 phm_get_voltage_index: voltage=893 -> index=5 phm_get_voltage_index: voltage=900 -> index=6 phm_get_voltage_index: voltage=937 -> index=7 phm_get_voltage_index: voltage=950 -> index=8 phm_get_voltage_index: voltage=981 -> index=9 phm_get_voltage_index: voltage=1000 -> index=10 phm_get_voltage_index: voltage=1025 -> index=11 phm_get_voltage_index: voltage=1050 -> index=12 phm_get_voltage_index: voltage=1100 -> index=13 phm_get_voltage_index: voltage=1150 -> index=14 phm_find_closest_vddci: vddci=600 i=0 value=850 polaris10_get_dependency_volt_by_clk: i=0 clock=26100 voltage=1185156224 mvdd=0 phm_find_closest_vddci: vddci=618 i=0 value=850 polaris10_get_dependency_volt_by_clk: i=1 clock=40542 voltage=1185156296 mvdd=0 phm_find_closest_vddci: vddci=624 i=0 value=850 polaris10_get_dependency_volt_by_clk: i=2 clock=65337 voltage=1185156320 mvdd=0 phm_find_closest_vddci: vddci=650 i=0 value=850 polaris10_get_dependency_volt_by_clk: i=3 clock=88653 voltage=1185156424 mvdd=0 phm_find_closest_vddci: vddci=650 i=0 value=850 polaris10_get_dependency_volt_by_clk: i=3 clock=93438 voltage=1185156424 mvdd=0 phm_find_closest_vddci: vddci=650 i=0 value=850 polaris10_get_dependency_volt_by_clk: i=3 clock=97962 voltage=1185156424 mvdd=0 phm_find_closest_vddci: vddci=650 i=0 value=850 polaris10_get_dependency_volt_by_clk: i=3 clock=101703 voltage=1185156424 mvdd=0 phm_find_closest_vddci: vddci=693 i=0 value=850 polaris10_get_dependency_volt_by_clk: i=4 clock=105792 voltage=1185156596 mvdd=0 polaris10_get_dependency_volt_by_clk: i=0 clock=30000 voltage=1185156224 mvdd=0 polaris10_get_dependency_volt_by_clk: i=1 clock=175000 voltage=1191710624 mvdd=0 phm_find_closest_vddci: vddci=600 i=0 value=850 polaris10_get_dependency_volt_by_clk: i=0 clock=30000 voltage=1185156224 mvdd=0 polaris10_get_dependency_volt_by_clk: i=0 clock=30000 voltage=1185156224 mvdd=0 phm_find_closest_vddci: vddci=600 i=0 value=850 polaris10_populate_smc_vce_level: count=0 MinVoltage=-2146655162 Frequency=418775040 phm_find_closest_vddci: vddci=650 i=0 value=850 polaris10_populate_smc_vce_level: count=1 MinVoltage=1208853574 Frequency=-2012413696 phm_find_closest_vddci: vddci=700 i=0 value=850 polaris10_populate_smc_vce_level: count=2 MinVoltage=269395014 Frequency=-131858176 phm_find_closest_vddci: vddci=750 i=0 value=850 polaris10_populate_smc_vce_level: count=3 MinVoltage=-670129082 Frequency=1748762880 phm_find_closest_vddci: vddci=800 i=0 value=850 polaris10_populate_smc_vce_level: count=4 MinVoltage=-1609587642 Frequency=-263257856 phm_find_closest_vddci: vddci=850 i=0 value=850 polaris10_populate_smc_vce_level: count=5 MinVoltage=1745921094 Frequency=2019754240 phm_find_closest_vddci: vddci=900 i=1 value=900 polaris10_populate_smc_vce_level: count=6 MinVoltage=806422599 Frequency=7799040 phm_find_closest_vddci: vddci=950 i=2 value=950 polaris10_populate_smc_vce_level: count=7 MinVoltage=-133075897 Frequency=-1601830656 phm_find_closest_vddci: vddci=600 i=0 value=850 phm_find_closest_vddci: vddci=650 i=0 value=850 phm_find_closest_vddci: vddci=700 i=0 value=850 phm_find_closest_vddci: vddci=750 i=0 value=850 phm_find_closest_vddci: vddci=800 i=0 value=850 phm_find_closest_vddci: vddci=850 i=0 value=850 phm_find_closest_vddci: vddci=900 i=1 value=900 phm_find_closest_vddci: vddci=950 i=2 value=950 phm_find_closest_vddci: vddci=600 i=0 value=850 polaris10_populate_smc_uvd_level: count=0 VclkFrequency=-131858176 DclkFrequency=-1864237056 MinVoltage=-2146655162 phm_find_closest_vddci: vddci=650 i=0 value=850 polaris10_populate_smc_uvd_level: count=1 VclkFrequency=-2143813376 DclkFrequency=418775040 MinVoltage=1208853574 phm_find_closest_vddci: vddci=700 i=0 value=850 polaris10_populate_smc_uvd_level: count=2 VclkFrequency=139198720 DclkFrequency=-1610022656 MinVoltage=269395014 phm_find_closest_vddci: vddci=750 i=0 value=850 polaris10_populate_smc_uvd_level: count=3 VclkFrequency=-1470430976 DclkFrequency=672989440 MinVoltage=-670129082 phm_find_closest_vddci: vddci=800 i=0 value=850 polaris10_populate_smc_uvd_level: count=4 VclkFrequency=1617363200 DclkFrequency=-936640256 MinVoltage=-1609587642 phm_find_closest_vddci: vddci=850 i=0 value=850 polaris10_populate_smc_uvd_level: count=5 VclkFrequency=410190080 DclkFrequency=-2143813376 MinVoltage=1745921094 phm_find_closest_vddci: vddci=900 i=1 value=900 polaris10_populate_smc_uvd_level: count=6 VclkFrequency=-797048576 DclkFrequency=943980800 MinVoltage=806422599 phm_find_closest_vddci: vddci=950 i=2 value=950 polaris10_populate_smc_uvd_level: count=7 VclkFrequency=-1601830656 DclkFrequency=-263257856 MinVoltage=-133075897 polaris10_populate_smc_boot_level: BootVddc=4110 BootVddci=18445 BootMVdd=0 polaris10_populate_clock_stretcher_data_table: i=0 Sclk_voltageOffset=4 polaris10_populate_clock_stretcher_data_table: i=1 Sclk_voltageOffset=6 polaris10_populate_clock_stretcher_data_table: i=2 Sclk_voltageOffset=12 polaris10_populate_clock_stretcher_data_table: i=3 Sclk_voltageOffset=20 polaris10_populate_clock_stretcher_data_table: i=4 Sclk_voltageOffset=21 polaris10_populate_clock_stretcher_data_table: i=5 Sclk_voltageOffset=24 polaris10_populate_clock_stretcher_data_table: i=6 Sclk_voltageOffset=26 polaris10_populate_clock_stretcher_data_table: i=7 Sclk_voltageOffset=28 polaris10_init_smc_table: TemperatureLimitHigh=23040 TemperatureLimitLow=22784 polaris10_thermal_setup_fan_table: TempMin=10240 TempMed=16640 TempMax=27904 Slope1=4352 Slope2=5632 ... phm_find_closest_vddci: vddci=600 i=0 value=850 polaris10_get_dependency_volt_by_clk: i=0 clock=22707 voltage=1185156224 mvdd=0 phm_find_closest_vddci: vddci=618 i=0 value=850 polaris10_get_dependency_volt_by_clk: i=1 clock=35272 voltage=1185156296 mvdd=0 phm_find_closest_vddci: vddci=624 i=0 value=850 polaris10_get_dependency_volt_by_clk: i=2 clock=56844 voltage=1185156320 mvdd=0 phm_find_closest_vddci: vddci=650 i=0 value=850 polaris10_get_dependency_volt_by_clk: i=3 clock=77129 voltage=1185156424 mvdd=0 phm_find_closest_vddci: vddci=650 i=0 value=850 polaris10_get_dependency_volt_by_clk: i=3 clock=81292 voltage=1185156424 mvdd=0 phm_find_closest_vddci: vddci=650 i=0 value=850 polaris10_get_dependency_volt_by_clk: i=3 clock=85227 voltage=1185156424 mvdd=0 phm_find_closest_vddci: vddci=650 i=0 value=850 polaris10_get_dependency_volt_by_clk: i=3 clock=88482 voltage=1185156424 mvdd=0 phm_find_closest_vddci: vddci=693 i=0 value=850 polaris10_get_dependency_volt_by_clk: i=4 clock=105792 voltage=1185156596 mvdd=0 [drm] Initialized amdgpu 3.9.0 20150101 for 0000:01:00.0 on minor 0 [drm] Memory usable by graphics device = 2048M checking generic (d0000000 6c0000) vs hw (d0000000 10000000) [drm] Atomic commit: SET crtc id 0: [ffff8808035d3000] [drm] [IfTrace_DC] dc_commit_targets: 1 targets [drm] [IfTrace_DC] core_target 0xa205ac0: stream_count=1 [drm] [IfTrace_DC] core_stream 0xa6ac400: src: 0, 0, 1680, 1050; dst: 0, 0, 1680, 1050; [drm] [IfTrace_DC] pix_clk_khz: 119000, h_total: 1840, v_total: 1080 [drm] [IfTrace_DC] sink name: PL2280W, serial: 1811 [drm] [IfTrace_DC] link: 1 [drm] [Conn_Mode] [DP-2] {1680x1050, 1840x1080@119000Khz}^ [drm] DM_NOT_IMPL: dm_bandwidth_update phm_find_closest_vddci: vddci=600 i=0 value=850 polaris10_get_dependency_volt_by_clk: i=0 clock=19756 voltage=1185156224 mvdd=0 phm_find_closest_vddci: vddci=618 i=0 value=850 polaris10_get_dependency_volt_by_clk: i=1 clock=30687 voltage=1185156296 mvdd=0 phm_find_closest_vddci: vddci=624 i=0 value=850 polaris10_get_dependency_volt_by_clk: i=2 clock=49455 voltage=1185156320 mvdd=0 phm_find_closest_vddci: vddci=624 i=0 value=850 polaris10_get_dependency_volt_by_clk: i=2 clock=67103 voltage=1185156320 mvdd=0 phm_find_closest_vddci: vddci=624 i=0 value=850 polaris10_get_dependency_volt_by_clk: i=2 clock=70725 voltage=1185156320 mvdd=0 phm_find_closest_vddci: vddci=624 i=0 value=850 polaris10_get_dependency_volt_by_clk: i=2 clock=74148 voltage=1185156320 mvdd=0 phm_find_closest_vddci: vddci=650 i=0 value=850 polaris10_get_dependency_volt_by_clk: i=3 clock=76980 voltage=1185156424 mvdd=0 phm_find_closest_vddci: vddci=693 i=0 value=850 polaris10_get_dependency_volt_by_clk: i=4 clock=105792 voltage=1185156596 mvdd=0 polaris10_get_dependency_volt_by_clk: i=0 clock=30000 voltage=1185156224 mvdd=0 polaris10_get_dependency_volt_by_clk: i=1 clock=175000 voltage=1191710624 mvdd=0 [drm] [IfTrace_DC] dc_pre_update_surfaces_to_target: commit 1 surfaces to target 0xa205ac0 [drm] DM_NOT_IMPL: dm_bandwidth_update [drm] DM_NOT_IMPL: dm_bandwidth_update [drm] DM_NOT_IMPL: dm_bandwidth_update [drm] DM_NOT_IMPL: dm_bandwidth_update [drm] DM_NOT_IMPL: dm_bandwidth_update [drm] DM_NOT_IMPL: dm_bandwidth_update [drm] DM_NOT_IMPL: dm_bandwidth_update [drm] DM_NOT_IMPL: dm_bandwidth_update [drm] DM_NOT_IMPL: dm_bandwidth_update [drm] DM_NOT_IMPL: dm_bandwidth_update [drm] DM_NOT_IMPL: dm_bandwidth_update [drm] Atomic commit: RESET. crtc id 0:[ffff8808035d3000] [drm] [IfTrace_DC] dc_commit_targets: 0 targets [drm] DM_NOT_IMPL: dm_bandwidth_update phm_find_closest_vddci: vddci=600 i=0 value=850 polaris10_get_dependency_volt_by_clk: i=0 clock=17188 voltage=1185156224 mvdd=0 phm_find_closest_vddci: vddci=600 i=0 value=850 polaris10_get_dependency_volt_by_clk: i=0 clock=26698 voltage=1185156224 mvdd=0 phm_find_closest_vddci: vddci=618 i=0 value=850 polaris10_get_dependency_volt_by_clk: i=1 clock=43026 voltage=1185156296 mvdd=0 phm_find_closest_vddci: vddci=624 i=0 value=850 polaris10_get_dependency_volt_by_clk: i=2 clock=58380 voltage=1185156320 mvdd=0 phm_find_closest_vddci: vddci=624 i=0 value=850 polaris10_get_dependency_volt_by_clk: i=2 clock=61531 voltage=1185156320 mvdd=0 phm_find_closest_vddci: vddci=624 i=0 value=850 polaris10_get_dependency_volt_by_clk: i=2 clock=64509 voltage=1185156320 mvdd=0 phm_find_closest_vddci: vddci=624 i=0 value=850 polaris10_get_dependency_volt_by_clk: i=2 clock=66973 voltage=1185156320 mvdd=0 phm_find_closest_vddci: vddci=693 i=0 value=850 polaris10_get_dependency_volt_by_clk: i=4 clock=105792 voltage=1185156596 mvdd=0 polaris10_get_dependency_volt_by_clk: i=0 clock=30000 voltage=1185156224 mvdd=0 polaris10_get_dependency_volt_by_clk: i=1 clock=175000 voltage=1191710624 mvdd=0
Comment