Linux Support Expectations For The AMD Radeon RX 6000 Series

[fenixex]

I've also got the 480 and Tumbleweed. B450 tomahawk MB. Unfortunately I get ~80% lower fps in games like DOOM and Wolfenstein when I turn it on.

[mibo]

I can't find anything like Above 4G in my Tuf Gaming x570, unlike my old Sabertooth Mark 2. Am I out of luck?

My ASUS TUF B550M just got a new BIOS 1202 (had 1004 before) and now the Option Advanced -> PCI Subsystem Settings -> Above 4G Decoding shows up (it was not there before).
Unfortunately, after enabling this option, under openSuse Thumbleweed with a rx480 graphics card I still get the BAR=256M message :-(

Anything else i need to activate?

[smitty3268]

Yeah, that's why I think it's a bitter victory. For now it is exclusive to DX12, because they have not developed it for anything else.

NEvermind us though, I bet Sony must be pretty pissed off too, considering they can't use this stuff either. This might be a bit of a heavy blow for them actually.

Khronos has a (unreleased) Vulkan spec for raytracing, that's being worked on by AMD along with various other vendors.

My understanding is that it's pretty far along, so they likely already have support in their internal drivers. When it gets released publicly is a good question, but hopefully it will come soon now that AMD has cards out that need it.

Also the PS5 definitely already has ray tracing.

[discordian]

Yeah, that's why I think it's a bitter victory. For now it is exclusive to DX12, because they have not developed it for anything else.

NEvermind us though, I bet Sony must be pretty pissed off too, considering they can't use this stuff either. This might be a bit of a heavy blow for them actually.

Why can't sony use what stuff? They aren't bound by any apis and from what I red about the added coherency protocols spanning cpu, gpu and down to their custom io processor "paging" in data from compressed files they have alot of stuff that doesn't yet exist in public apis.

[rabcor]

tl;dr - linux is a second class citizen, but it's not being completely blocked from revolutionary new tech

Yeah, that's why I think it's a bitter victory. For now it is exclusive to DX12, because they have not developed it for anything else.

NEvermind us though, I bet Sony must be pretty pissed off too, considering they can't use this stuff either. This might be a bit of a heavy blow for them actually.

[bisby]

Feels like a bit of a bitter victory to me, AMD finally competing with Nvidia again, which is great.

But some of it's best features require

A: A Ryzen 5000 or newer AMD CPU
B: DirectX 12

And as you all know, we may not mind the former, but we don't have the latter. Their entire raytracing technology for instance is built around directx 12, and osme other things such as smart access memory also seem to rely on it. Which means that these cards, while still pretty good, will be somewhat crippled on linux.

A huge chunk of the thread is people getting "SAM" to work on non 5000 CPUs. So for linux at least, ryzen 5000 isn't required specifically. It can guarantee compatibility, but not mandatory. Unless "SAM" has something bigger under the hood that wasn't really touched upon.

And unless their hardware is purpose built for DX12 and not vulkan, its all just software. So it is disappointing that DX12 is such a focus, but it also means that theres nothing preventing the open source community from doing these things themselves. Their raytracing technology isn't exclusive to DX12... they just have announced a DX12 interface for using it. Vulkan and Wine+VKD3D will almost certainly have these things in the long run.

tl;dr - linux is a second class citizen, but it's not being completely blocked from revolutionary new tech

[rabcor]

Feels like a bit of a bitter victory to me, AMD finally competing with Nvidia again, which is great.

But some of it's best features require

A: A Ryzen 5000 or newer AMD CPU
B: DirectX 12

And as you all know, we may not mind the former, but we don't have the latter. Their entire raytracing technology for instance is built around directx 12, and osme other things such as smart access memory also seem to rely on it. Which means that these cards, while still pretty good, will be somewhat crippled on linux.

[geearf]

I can't find anything like Above 4G in my Tuf Gaming x570, unlike my old Sabertooth Mark 2. Am I out of luck?

[drm] Detected VRAM RAM=8192M, BAR=256M

[smitty3268]

I have a RX560 from 07/2017. Not much sense in going for a RX 580 now. When upgrading, I will go for the latest generation.
The 5500 XT is a tad too expensive. I was waiting for a normal RX 5500 (single fan ideally) but I haven't seen it anywhere since it was supposed to be available (a year ago).

The cheaper cards will come next year, but nothing has been officially announced.

I believe the rumor is a 6700 will be coming in Q1, so my guess is the $200 cards are probably due next summer.

[oiaohm]

Yeah thats precisely the problem.

No if that exactly the problem with AMD its PCIe Hot Reset not working so requiring baco reset to work around broken bit of function. The cards doing that are not exactly to AMD intended specification.

Well from what I heard at work (I don't directly deal with this) you get GPU lockups very frequently with AMD cards that require a power reset in order to get working. With NVidia we don't get this issue. A lot of people claim that this is due to AMD's FLR.

The claim is that it due to not implementing FLR is no. Nvidia FLR is power cycling the Nvidia GPU as well as it really the only way to get a GPU out of a unknown state causing a jam. PCIe Hot reset you setup to power cycle the full card good part about this in theory is any power disruption on the card should be handled because you are resetting everything. Different markers of AMD cards managed to come up with ways this does not work out. Yes powering down and back up the GPU fairly quickly as what Nvidia FLR does can also bring out card defects in the same way PCIe Hot reset does on AMD.

If PCI Hot reset works you should have shorter turn around to where the GPU is usable again compared to Nvidia FLR this is even on Nvidia cards. Yes BACO reset is slower than a hot reset with AMD cards on the cards where it works.

To be honest I am personally not that familiar with this.

And it shows.

Sure, from my suspicion this is a case of some issues being worse then other and having to do a hard power cycle reset is one of the worst things that can happen in sever environment (next to cards just not working or thermal problems).

Hard power cycle being required as in a normal system reboot not working fix card states is a sign that that PCIe Hot Restart is broken with all PCIe cards. Really it should be more than clear that EFI bios don't do FLR instructions in a normal software reboot when different AMD and Intel motherboards chip-set controllers lock up solid when they receive a PCIe FLR.

PCIe Hot Reset is mandatory feature that should work without issue or you will have system reset issues.

PCIe FLR being optional part of the PCIe specification horrible means its legal to implement FLR in devices as a Halt and Catch fire command and still be exactly to specification.

Can you point me to the spec where it states this? I will forward it to relevant people.

https://alexforencich.com/wiki/en/pcie/hot-reset-linux

Note that this only affects that specific function of the device, not the whole device, and devices are not required to implement function-level resets as per the PCIe specification.

I have gave that link before but if you dig though the complete PCI specificaiton you are not allowed to disconnect from PCIe bus using FLR for any reason. If the cause of lockup is some bad data in transport connection FLR should not be able to help you.

Fun part about the PCIe specification on FLR is it only to effect a specific function like vendor if they wanted to be funny could implement FLR on their graphics card as a feature that turns a LED on the card on and off and be valid to PCIe specification.

PCIe Hot Reset has a define meaning that the card should be returned to equal state as it would have been when the system powered on from the wall this includes disconnecting from the PCIe bus and going though the reconnect process.

Well according to the thread you can speak to level1techs who also constantly have this problem in context of VFIO

Something to remember level1techs was having the problem a while back. Yes level1techs has an "AMD Customer Engagement Representative" they are one of the one who did reports that made AMD work out that PCIe Hot Reset on AMD cards was not always working right. This is why we have BACO reset now as backup plan. Heck AMD has implemented Baco reset on as many cards as possible including cards that are that old that if they were Nvidia you would not be getting any support.

Really you have been asking the wrong question has the new AMD cards this time from all vendors had PCIe Hot Reset implemented without defect. AMD graphics cards with defective PCIe Hot Reset was a major pain in ass.

Please note the AMD problem with their GPUs not being able to reset was not only their stuff being defective a few case were that the motherboard chipsets were defective. Yes Nvidia using FLR means vendors had not been working out card disconnecting and reconnecting to PCIe bus as much as they should of.

https://www.nvidia.com/en-us/geforce/forums/game-ready-drivers/13/100694/graphics-card-disappearing-after-reboot/

This one is over 10 years old yes is Nvidia. But its classic example of PCIe Hot Reset not working. If PCIe Hot Reset is not working you have the magic disappearing PCIe cards after reboot. Of course its a dice roll if it happens or not. More often you are doing a PCIe Hot Reset the more likely it will show that you have a PCIe reconnect issue either in motherboard chipset or card. Defective chipsets and boards for handling PCIe Hot Resets is not a new thing.

hard power cycle reset is one of the worst things that can happen in sever environment

There is worse than this. I had a server motherboard that you power cycled the machine and you had a 1 in 3 chance that a particular card would appear. Remember on power on all cards get a single PCIe hot reset. So when you have a card with suspect PCIe Hot Reset functionally combined with motherboard with suspect PCIe connect logic in chipset the server comes a nice random generator if it works. Yes that was it not finding it Raid controller. Some ways the sooner I find out I have a motherboard with Hot Reset problems the better. Its not like you want to have a power outage for some reason and find out over half your servers are playing dice attempt to get them to restart.

Glitches are part of running servers like it or not. I do see Nvidia FLR as a high advantage. I prefer to use AMD PCIe Hot Reset because as well as restarting the GPU card its checking basic motherboard functionality that it stable.

Yes if you have a AMD GPU card in a server that is giving you trouble and card does not have a PCIe Hot Reset bug itself and its glitching you have a motherboard chipset problem or damaged traces on motherboard that are both really bad.

This is one of these fun things tunneled focused on getting X to work can allow you to go past that hang on this failure I am seeing is telling me that I have really big possible data destorying hardware defect. Motherboard PCIe bus system not being properly functional for disconnect and connect that performing lots of PCIe Hot Resets is testing can be a precursor sign to data being damaged crossing over the PCIe bus.

Having to use BACO reset with AMD or FLR with Nvidia is not sign of good things if the card does not have a defect. Also regularly locking up GPUs that is normally signs of power issues or cooling issues as well.

VMs crashing should be rare so should GPU lock ups. Of course those overclocking stuff going past what is stable comes with price of being unstable.

This is the problem needing to use BACO with AMD GPU is sign of something in system is not right being either the GPU or the motherboard has a problem. You are praying if you are using BACO reset a lot with AMD GPU that is the GPU. If it the motherboard the PCIe connected SSD with your file system on could be nicely not be receiving the bits you are sending it instead storing random bits of junk on you.

Nvidia usage of FLR has been allowing not quite to PCIe specification motherboard chipsets to be used as well. The AMD GPU reset problem is a broad problem. Shows with AMD GPU but the problem is not always AMD GPU.

This is why when people get tunnelled versioned and go hey Y vendors cards don't have X feature so that has to be the problem. Lot of cases two different vendors implement something differently and you really need to understand what the problem is. AMD in this case there has been some dud cards and there are some dud motherboards that AMD reset method exposes.

So its not that AMD chosen reset method is wrong out the PCIe specification it really should be the most dependable method because its a mandatory method. Its really horrible when something that should be mandatory turns out to be defective by many parties.