Valve Developer Andres Rodriguez Lands First Patches Into RADV Vulkan Driver

[funfunctor]

<snip>
@bridgman

You raise good points although I am aware of them. You misunderstand me regarding the workstation market because I was not being clear enough so let me reiterate another way; It is not the workstation market size today I am trying to estimate, it is the workstation projected growth.

Regarding amdkfd, I am really being "dramatic", if you believe I am then you never actually went though the patches pipled into the repo properly. I would not be at all surprised if people kill their hardware with it ! It is extremely obvious how it has been developed - some week(s) were found in the middle between projects, stuff was worked on and then something else came up so everyone just did a 'git add -u && git commit -m "whatever"' and when they came back forgot what they were even working on. It is also obvious that cycle of "forgot -> hack away -> forgot" has occurred a number of times. Further, no tracking has occurred from upstream or your other development branches, in fact its not at all unlike a random Android kernel branch chucked on github.

I spent many weeks looking it over and spending time getting things working locally *properly* and not in some show-and-tell form, amdkfd requires a concerted effort to make it usable let alone upstream ready. Right now you have under voltage hacks because of integer overflows and things and the fans set on full in the hops that the hardware does not go up in smoke.

I am sorry but yes it is a mess and I know it sounds super harsh, particularly because of the huge volume of effort AMD is doing and how much things have improved over the last year alone. I would say this is the one area that has been completely dismissed for everything else to succeed and I can see it not being without reason due to limited resources.

The part that concerns me is that I have reached out personally on a number of occasions (with patchsets to back me up) to help sort it out by just trying to coordinate a little and that community interaction has been sub-optimal. Like I have a great number of AMD patches rebased on master and some even cleaned up and AMD ignores that, makes no sense to me duplicated and wasted effort. That last part says to me that its worse than "just not enough time" it says to me "this is just a pet project so really not _that_ interested" or just poor understanding of community coordination. I now think the latter could be the case by watching the DAL situation on the side, because it feels reminiscent of the workflow process in and around amdkfd.

But... what do I know, I'm just some guy I suppose.

[extraymond]

https://www.phoronix.com/scan.php?pa...-1.4-Available

https://github.com/RadeonOpenCompute/ROCm

Any chance on getting this on tonga?
I'm eager for trying hsa and numba on tonga.
It would provide such improvement for numerical analysis if I understand correctly.

[bridgman]

You raise good points although I am aware of them. You misunderstand me regarding the workstation market because I was not being clear enough so let me reiterate another way; It is not the workstation market size today I am trying to estimate, it is the workstation projected growth.

Actually I didn't misunderstand you at all - I do see the difference in growth rates (both magnitude and sign), but I am saying that if cutting off a significant source of revenue in the short term would make Linux development funding worse not better. That wasn't the case back when fglrx was a completely separate code base but now that most of the code is shared across Linux stacks and markets all of the business opportunities largely pull in the same direction.

Regarding amdkfd, I am really being "dramatic", if you believe I am then you never actually went though the patches pipled into the repo properly. I would not be at all surprised if people kill their hardware with it ! It is extremely obvious how it has been developed - some week(s) were found in the middle between projects, stuff was worked on and then something else came up so everyone just did a 'git add -u && git commit -m "whatever"' and when they came back forgot what they were even working on. It is also obvious that cycle of "forgot -> hack away -> forgot" has occurred a number of times. Further, no tracking has occurred from upstream or your other development branches, in fact its not at all unlike a random Android kernel branch chucked on github.

I spent many weeks looking it over and spending time getting things working locally *properly* and not in some show-and-tell form, amdkfd requires a concerted effort to make it usable let alone upstream ready. Right now you have under voltage hacks because of integer overflows and things and the fans set on full in the hops that the hardware does not go up in smoke.

I am sorry but yes it is a mess and I know it sounds super harsh, particularly because of the huge volume of effort AMD is doing and how much things have improved over the last year alone. I would say this is the one area that has been completely dismissed for everything else to succeed and I can see it not being without reason due to limited resources.

The part that concerns me is that I have reached out personally on a number of occasions (with patchsets to back me up) to help sort it out by just trying to coordinate a little and that community interaction has been sub-optimal. Like I have a great number of AMD patches rebased on master and some even cleaned up and AMD ignores that, makes no sense to me duplicated and wasted effort. That last part says to me that its worse than "just not enough time" it says to me "this is just a pet project so really not _that_ interested" or just poor understanding of community coordination. I now think the latter could be the case by watching the DAL situation on the side, because it feels reminiscent of the workflow process in and around amdkfd.

But... what do I know, I'm just some guy I suppose.

I do agree with most of what you say here, however it does seem like somewhat of a "snapshot" view. I hate even saying that because I know you put a lot of work into the early code but we weren't really set up for community development at that point (and probably still won't be for another month or two).

1. I think you will find that most of the scary things you saw in release N were gone or at least less scary in release N+1... early releases were aligned with demo schedules and functionality was top priority, while the focus more recently has been on stability and extending HW support. That said I haven't gone back and checked that for a couple of months, so need to get back to that once I can step away from the new product train for a bit.

2. We have been syncing with internal amdgpu trees regularly, typically about every second release (and we have had 5 so far IIRC), but there certainly some of the early releases used relatively old amdgpu code. At the point you were most involved we were probably furthest from upstream, so no argument there. The currently-in-process jump ahead will be to 4.9 kernel for internal trees (Alex just pushed a 4.9 amdgpu staging tree), and next jump should be to tip of upstream then hopefully we can stay there.

3. Community coordination has been very limited so far - we are set up to accept bug reports and bug fixes but nothing much more than that, at least at KFD level.

It is probably fair to say that there has been more community focus at userspace level in support of application porting than on the kernel side. We had to get through a pile of KFD development work before we could even start doing much work with the community and upstream, basically catching up with the RFC we wrote ~18 months ago. I think we are mostly through that now, and hopefully you should see community engagement start to ramp up over the next month or so.

[Geopirate]

Ask developers of Talos Principle why their game isn't sold on GOG. Or many other such developers. They are locked into using Steamworks.

Many developers are not interested in releasing DRM free games, or are restricted by their publishers, a parent, company, etc. It's super easy to live in a DRM-free echo chamber and lose sight of the fact that the AAA gaming market is still heavily dominated by DRM. That developer has actually opened up their stuff well after they made their $$$ on the DRM market.

[shmerl]

Many developers are not interested in releasing DRM free games, or are restricted by their publishers, a parent, company, etc. It's super easy to live in a DRM-free echo chamber and lose sight of the fact that the AAA gaming market is still heavily dominated by DRM.

Talos Principle is developed by Croteam and published by Devolver Digitial who are a quite DRM-free friendly publisher (many of the games published by them are sold on GOG, including recent ones). So it's not about Croteam being interested in DRM. It's about them not caring about lock-in and not being interested in releasing anything to non Steam users. They said so explicitly. This demonstrates how "open" Steam ecosystem really is. I.e. it's not, because many Steam tools introduce lock-in for developers.

[humbug]

And where is SteamVR supposed to come from? Can games distribute it without Steam? I doubt that. So you can may be launch it without Steam in the best case, but you need to be Steam user to get SteamVR runtime itself first.

Yes that is true, you get SteamVR via steam.

But Valve did provide OpenVR

https://github.com/ValveSoftware/openvr

Watch

https://www.youtube.com/watch?v=4Gs5k2Fti1U

GOG Galaxy and Origin and anybody else can create software for it, just like Valve did with SteamVR.

In theory. But no one implemented OpenVR besides for Steam (yet). So de-facto it's tied to Steam.

Looks like there are already non-steamVR implementations of Valve's openVR.
https://www.reddit.com/r/Vive/commen...am_vive_games/

[humbug]

No, GOG games are all DRM-free and their developers can release them through any other store.

Steam games can and are released through other stores as well, there are no exclusivity agreements unlike in the console world. Yes most steam games have DRM, unlike GOG. The conversation was not about this.
The conversation was about closed as in single gatekeeper access to something. So Steam is closed because Valve is the gatekeeper. Ubuntu software center is closed because Canonical is the gatekeeper. Origin is closed because EA is the gatekeeper. GOG is closed because CD Projekt is the gatekeeper etc... And that's ok, it's necessary. However Linux and Windows are open ecosystems because users have the choice to use all/any of the above.

[funfunctor]

Actually I didn't misunderstand you at all - I do see the difference in growth rates (both magnitude and sign), but I am saying that if cutting off a significant source of revenue in the short term would make Linux development funding worse not better. That wasn't the case back when fglrx was a completely separate code base but now that most of the code is shared across Linux stacks and markets all of the business opportunities largely pull in the same direction.



I do agree with most of what you say here, however it does seem like somewhat of a "snapshot" view. I hate even saying that because I know you put a lot of work into the early code but we weren't really set up for community development at that point (and probably still won't be for another month or two).

1. I think you will find that most of the scary things you saw in release N were gone or at least less scary in release N+1... early releases were aligned with demo schedules and functionality was top priority, while the focus more recently has been on stability and extending HW support. That said I haven't gone back and checked that for a couple of months, so need to get back to that once I can step away from the new product train for a bit.

2. We have been syncing with internal amdgpu trees regularly, typically about every second release (and we have had 5 so far IIRC), but there certainly some of the early releases used relatively old amdgpu code. At the point you were most involved we were probably furthest from upstream, so no argument there. The currently-in-process jump ahead will be to 4.9 kernel for internal trees (Alex just pushed a 4.9 amdgpu staging tree), and next jump should be to tip of upstream then hopefully we can stay there.

3. Community coordination has been very limited so far - we are set up to accept bug reports and bug fixes but nothing much more than that, at least at KFD level.

It is probably fair to say that there has been more community focus at userspace level in support of application porting than on the kernel side. We had to get through a pile of KFD development work before we could even start doing much work with the community and upstream, basically catching up with the RFC we wrote ~18 months ago. I think we are mostly through that now, and hopefully you should see community engagement start to ramp up over the next month or so.

bridgman unfortunately I heard all this before however the situation remains, the only code that currently exists is the code that is open - unicorns that are behind the AMD firewall mean diddly squat so don't count. The message you effectively send to the community there is this; "Don't bother helping us or contributing back because we are about to release xyz star child that is gonna solve all your problems". I strongly suggest getting that out of so called internal-branches before it becomes DAL all over. "We take bug reports" isn't going to cut it, you *need* to start coordinating during the actual development cycle and get out of this drop tarballs modus operandi, mainline moves *very* fast as your well aware.

[bridgman]

unfortunately I heard all this before however the situation remains, the only code that currently exists is the code that is open - unicorns that are behind the AMD firewall mean diddly squat so don't count.

Not sure what you mean here... there isn't any code behind the firewall... all the eviction logic etc.. is in the public repos already.

The message you effectively send to the community there is this; "Don't bother helping us or contributing back because we are about to release xyz star child that is gonna solve all your problems".

Well no, the message is more like "we are about to release XYZ which is gonna solve these specific problems", but until we are starting to get caught up with upstream I think that is the only practical message, for better or worse. If we try to coordinate development too early then we end up working at cross purposes despite everyone's best intentions.

It's nice to say "oh you should stop worrying about adding new features until you get upstream" but the world and the markets aren't going to stop and wait for us - we have to do both in parallel, like it or not.

I strongly suggest getting that out of so called internal-branches before it becomes DAL all over. "We take bug reports" isn't going to cut it, you *need* to start coordinating during the actual development cycle and get out of this drop tarballs modus operandi, mainline moves *very* fast as your well aware.

Yes, but as I have said multiple times until we had code which had a chance of being accepted upstream all the coordination in the world wasn't going to help. We are just getting to that point now, and as you say it is time *now* to start shifting the development model to a more upstream-centric one.

[funfunctor]

Not sure what you mean here... there isn't any code behind the firewall... all the eviction logic etc.. is in the public repos already.



Well no, the message is more like "we are about to release XYZ which is gonna solve these specific problems", but until we are starting to get caught up with upstream I think that is the only practical message, for better or worse. If we try to coordinate development too early then we end up working at cross purposes despite everyone's best intentions.

It's nice to say "oh you should stop worrying about adding new features until you get upstream" but the world and the markets aren't going to stop and wait for us - we have to do both in parallel, like it or not.



Yes, but as I have said multiple times until we had code which had a chance of being accepted upstream all the coordination in the world wasn't going to help. We are just getting to that point now, and as you say it is time *now* to start shifting the development model to a more upstream-centric one.

I understand all your reasoning already bridgman however not coordinating kernel work openly is wrong and that is the crux of the problem. I am not so naive to understand certain things are done behind firewalls for all sorts of reasons and there is a balance to be had there. But `drivers/gpu/drm/amd/amdkfd` isn't a AMD silo where only AMD are allowed to touch, I could have just as easily sent in patches to make my own take on ioctl() pinning semantics and where would AMD be then if they wanted a variation on that? I literally didn't out of being mindful of it but why should I be mindful if AMD isn't coordinating? That isn't how kernel development works as you well know by now and that is my point, I am not trying to flame you I am trying to coordinate effort with you.