AMDGPU's Scheduler Might Get Picked Up By Other DRM Drivers

[polarathene]

Is the scheduler considered to be in good shape by the AMD driver devs? I'm a little bit concerned that if this is picked up for generic use that AMD driver devs will have a more difficult time improving it for their own use cases if others start depending on it. Is this a valid concern?

They could perhaps update to utilize this adaptation to make it available to others, so that their work/contributions go to a single codebase and don't need to be ported over?

I hope to see other drivers pick it up Is it beneficial only to actual hardware drivers like Intel or also useful for the virtualized ones from VirtualBox, VMWare, QEMU(BOCHS, Cirrus, virtio-gpu), Virgl?

[tg--]

Is the scheduler considered to be in good shape by the AMD driver devs? I'm a little bit concerned that if this is picked up for generic use that AMD driver devs will have a more difficult time improving it for their own use cases if others start depending on it. Is this a valid concern?

This is a very typical process in the linux kernel.
In this case there are basically 2 common sense rules that apply:
a) never break userspace
- if this code isn't used by userspace (which seems likely for a internal kernel scheduler), no worries
b) if you change some infrastructure, you can't break other in-tree users
- if another driver is using the scheduler and your changes would break that, you'll have to fix the other driver to work with said changes
- if the someone else changes the scheduler, he'll have to make sure your driver works by either working with you or providing said changes
- any improvements made by others automatically improve your driver

This means there is some more bureaucracy to make changes, but you'll have to understand that bureaucracy isn't necessarily bad, and in this case will in the mid- to long-term pay out by decreasing your maintenance cost and gets you free improvements.

It's a that give-and-take that makes functioning open source projects great, and exactly what makes people and companies want to use and contribute to open source projects like the Linux kernel. In fact, that small bureaucracy "burden" pays off so well, that even entities who strongly dislike the GPL (which encourages working with upstream to a greater degree) will happily use and contribute to projects using it, Google as one major contributor to Linux and Coreboot amongst others as the best example.

[bridgman]

Is the scheduler considered to be in good shape by the AMD driver devs? I'm a little bit concerned that if this is picked up for generic use that AMD driver devs will have a more difficult time improving it for their own use cases if others start depending on it. Is this a valid concern?

There's a difference between "in good shape" (probably yes) and "ready to be frozen/shared and not require major changes for new hardware" (possibly no). Leveraging the code is always a good thing but sharing a single copy of the code between HW vendors with significantly different hardware might be premature... we'll need to discuss.

[timofonic]

There's a difference between "in good shape" (probably yes) and "ready to be frozen/shared and not require major changes for new hardware" (possibly no). Leveraging the code is always a good thing but sharing a single copy of the code between HW vendors with significantly different hardware might be premature... we'll need to discuss.

So why not start that discussion with all involved parties ASAP? Why wait for it? Are there some planning about how to make the code as much reusable as possible without performance drops or other issues? In my opinion, source code should follow that philosophy. You are a skilled and experienced developer and I want to be one someday, so I would love to know your points of view and appreciations about this.

Thanks in advance, sincerely.

[pal666]

Anyone with half a brain knows that once you open-source something, people can and will take advantage of it.

this depends on license

[pal666]

And, that's the reason, why developers shouldn't be too eager to get their code mainlined, if the code isn't ready.

why is that a reason? nobody could reuse non-mainlined code, so how non-mainlining will help with reuse? it will not help. but mainlining will help with maintenance, so, contrary to your belief, developers should be eager to get their code mainlined

[ElectricPrism]

AMD getting DC mainlined has me chomping at the bit to get my paycheck ready to buy a RX Vega 64. (I just am contemplating if I need it water-cooled @ stock or if air is fine in a small rig)

After using a open driver I have 0% reason to ever prefer a closed driver over MESA.

Even if hell froze over and nvidia decided to open source or join in MESA by making their own adaptation I would still prefer AMD because of their history of loyalty to open source.

This is the exact kind of commits that I have been excited to see once DC was mainlined, it's going to be a exciting rodeo this next 6 months.

[bridgman]

So why not start that discussion with all involved parties ASAP? Why wait for it? Are there some planning about how to make the code as much reusable as possible without performance drops or other issues?

If you mean "discussion about this specific topic" that's certainly going to happen. If you mean "discussion about reworking all our code to make it as shareable as possible" that is one of a bunch of background activities that ends up in the "as time permits" bucket simply because there is so much urgent work that needs to be done as well.

Every time another driver shares the same copy of code that we use it puts a non-trivial burden on anyone needing to make major changes to that code - either in the form of pro-active monitoring of how every other driver uses the code to avoid getting stuck in a situation where we can't make the changes we need without breaking other drivers or in the form of having to rework a bunch of other drivers we know nothing about in order to accomodate changes we need for future hardware, so part of any decision to share a single copy of code needs to be knowing where you are going to find the developer time to fund that extra work. You also need to find the senior developer time for the initial discussion about whether live-sharing a single copy is a good idea in the first place.

Code sharing ends up as a fairly ad-hoc affair, driven primarily by people identifying candidates for sharing (ie exactly what happened here) followed by a case-by-case determination of how likely it is that the original code will either be able to stay relatively stable or be extended with reasonable effort to support future hardware. Unfortunately the planning horizon for "what we are likely to need" is relatively short in the grand scheme of things, so the decision-making process generally ends up as "I dunno, I guess it should be OK" followed maybe 1/3 of the time by "Oh crap now what are we doing to do... we're probably going to end up having to fork a private copy of that shared code".

Where that leads, obviously, is the conclusion that the code-sharing decision also needs to consider how likely it is that the shared code is going to get wired into something larger and more complex over time to meet someone else's requirements, with the consequence that "forking our own code back" will end up being a lot more expensive "after sharing" than it would have been to maintain a private copy of the code in the first place unless we rip out a lot of the changes that were made to support other hardware... and of course future hardware plans are one of the things that generally nobody is allowed to talk about except in the vaguest possible terms which makes the discussion even more fun.

Bottom line I guess is that this is the kind of thing that does get discussed every year at developer conferences, but not even companies 10x-20x our size can afford to pro-actively drive or support these discussions for all of their code. As a result the process does end up being fairly ad-hoc as I mentioned earlier, with maybe a couple of candidates for sharing being "hallway-discussed" each year.

[GruenSein]

My experience is quite limited in this context so I am not sure if this makes sense but wouldn't it make sense to offer the generic, self-contained parts (tall order, I know) of the driver such as a scheduler as libraries? Whenever you need to make significant changes, you release a new version. Other driver developers can then choose to upgrade their driver to take advantage of the new version. If they lack the ressources, they could also stick to the previous version and simply submit the ocasional maintainance patch to it. This is not a "one copy" code sharing approach but it invites community feedback and input without tying down the progress of the original developer. The big question here seems to be how modular the driver is and if it lends itself well to swapping out specific parts.

[geearf]

Second, he sought out open-source for the sake of getting help, and he got it. Anyone with half a brain knows that once you open-source something, people can and will take advantage of it. He was willfully ignorant of this, he (to my understanding) didn't use any open-source license to protect himself, and as a result, had no right to complain.

This is not my understanding.

I believe he opened sourced it to create a new market for something no-one cared about yet or believed in, and then hoped to gain customers with his new improved and proprietary code. Alas customers stuck with Crunch as it's free instead of using Basis, and now his old free offering became his main competition. I am not sure how another open source license would have helped, unless maybe releasing under a dual license both with a more strict FOSS license and then a proprietary one, then moving people from the Crunch proprietary license to the Basis one would be an easier path (but would there be anyone using that license?)