David Airlie Shares His Thoughts On Current Challenges With Linux GPU Compute Stacks

[mdedetrich]

One of the hard realities here is its next to impossible to get the kernel developers today to agree on change. The cost was looked at when GPLv3 license change was being considered. Remember to change license you need every developer who has code in the project to sign off or replace their code. If they have died you have to find who inherited there rights.

The whole premise of what is being talked about is that the kernel dev's agree to it, otherwise the whole point is moot

The current EXPORT_SYMBOL does not say is free for all non GPL usages. Other licenses can be at play.

Yes but this is irrelevant to the point being discussed. Any such hypothetical graphics driver ABI would have the same restrictions and freedoms as the current syscall API and thats all that matters

Read this a little more carefully. You just wrote new code ABI. This is new material if you don't do extra steps and new material in a GPL work falls straight under derivative work so straight under GPL.

Yes and? I know it would be derivative of GPL, but in practice this doesn't amount to any practical problem

Now if that work making the new code ABI as lets say appears first in the freebsd kernel or as some form of standalone work. Then it comes preexisting work so when it comes into contact with Linux GPL then derivative work clause no longer triggers. This protection we don't need to change COPYING any more just be really careful with work origin points..

You don't need to attempt to herd the cats of Linux developers to get agreement that could take you 30+ years to complete to change COPYING using the right method.

Yes correct, and this is because modifying COPYING is not considered relicensing which is why you don't need to get permission from everyone and this is what I mentioned before.

Again read the law.

I know the law here, I have to deal with this frequently (on an almost weekly basis) because of my work

I am not saying you need to create a totally new and novel the law in fact creating new is taking the biggest risk with GPL derivative clause the safest path is find something preexisting not covered by GPL. Percentage of the existing linux graphics API in kernel space can be taken as is because when you check the derivative work origin point of the API function define its not Linux but some other OS so that stuff is already not covered by GPL. Yes big legal audit required.

Why are you bringing other OS's? The whole point of the kernel ABI is that its specific to Linux and you just admitted yourself that in the end because we are talking about code that was originally written in Linux.

If there is code in the Linux kernel that originated from other sources as long as the license is compatible you just need to acknowledge it.

And I have had to personally do this on a codebase that is ~5 million lines of code with another person with that codebase being cleared.

I can also guarantee you that if there does happen to be a problem in this area, it would have already been noted because its really not relevant to a new ABI. Basically it means that someone has put incompatible code into the Linux kernel.

When you understand how derivative work functions it always been shocking that its Linux that has the latest graphical API/ABI first I would have thought parties like Nvidia would have been smart and invested in freebsd or equal graphic stack development so they had immunity by having preexisting work to stop GPL derivative work clause dead in it tracks..

Your creating a problem when one doesn't exist. NVidia does not see any legal/IP/copyright ownership value in ABIs/API's for operating systems they support. For NVidia, the only thing they would care about some hypothetical Linux kernel ABI/API are technical matters (i.e. does their driver integrate well with it), thats pretty much it.

When Microsoft created their new graphics kernel ABI in Vista (i.e. WDDM), they just worked with the major graphics card vendors i.e. NVidia/AMD/Intel to design an API that works for everyone and thats pretty much it.

Lot of ways the Linux kernel needs a serous code audit by a major legal team working out what is what. Parties like Nvidia who want to do closed source modules have not done it publicly and there is the possibility not at all..

As I said, no such audit would need to be made. You are blowing things out of proportion

mdedetrich the most important thing. A "preexisting work" as in a work that predates you code anything taken from that cannot be infected by GPL derivative work clause in fact comes a barrier to GPL derivative work spread. Yes that that includes API taking defines.

mdedetrich the question that needs to be answered is the EXPORT_SYMBOL functions what ones are "preexisting work" to the Linux work so walls off GPL derivative work and don't contain any other code in the function causing different copyright license replace GPL if GPL is not in use? After answering this it then fill in the gaps.

I don't understand what you are saying or implying here.

Look its pretty clear, there are no real legal ramifications for the Linux kernel team making a graphics driver ABI as long as they don't do something so momentously stupid like copying pre-existing work outside of the Linux kernel with an incompatible license, line by line.

[Developer12]

Not "any random vulkan driver". It will need to have adequate extensions, and I doubt they exist to support the full richness of OpenCL's kernel language. SVM is one example of something lacking from Vulkan.

As pointed out by Airlie, Vulkan lacks OpenCL's runtime libraries and has a different SPIR-V format. Neither of those seem trivial gaps to bridge, as I think the format difference also aligns with some functional differences (or deficiencies, on the Vulkan side).

Those are gaps that have been bridged. There's already a working PoC.

Yess, you need various extensions, but many of the vulkan drivers out there, especially those in mesa, implement them.

[coder]

Those are gaps that have been bridged. There's already a working PoC.

Interesting. Got an links or other details?

Yess, you need various extensions, but many of the vulkan drivers out there, especially those in mesa, implement them.

My point is that, even in this thread, people make these blind leaps of faith that just because a platform has a Vulkan driver, you'd be able to run any compute job on it.

It's like saying that just because a program is written using OpenGL, it should run on anything with an OpenGL driver, totally ignoring the fact that a lot of devices only supported OpenGL ES, with some not even making it to 3.0, not to mention the matter of extensions.

As far as Mesa is concerned, the only up-to-date drivers are for Intel, AMD, and Qualcomm. Everything else is sitting at 1.0, with less than half the extensions supported, according to MesaMatrix. So, merely appealing to Mesa doesn't get you very far, in the real world.

In other words, Vulkan is far from being a panacea, toward the ideal of a run-anywhere, open compute API. Yet, too many people are still high on the Vulkan hype train and remain blind to this fact. If you actually watch Airlie's presentation, he doesn't have a lot of time for Vulkan. He outlines some of its shortcomings and leaves it at that. I think that says a lot, actually.

[oiaohm]

The whole premise of what is being talked about is that the kernel dev's agree to it, otherwise the whole point is moot

So then why did you bother posting it. The cost and the complexity of re-licensing the linux kernel was looked at back in 2006-2007 with the GPLv3 mess.

mdedetrich the kill problem finding all the Linux kernel developers who have ever contributed to the Linux kernel who still have their code in the Linux kernel. Lots of the contact information is absolutely out of date and anyone you miss could turn up in future.

My line is that there is a legal way to avoid need agreement.
https://en.wikipedia.org/wiki/NDISwrapper its has been used by items like ndiswrapper.

My premise is we use the legal limitation on derivative work. Just like ndiswapper here that GPL license bit that goes into the Linux kernel the GPL license does not infect the binary blobs of ndis drivers load because the ndis API is not a Linux API and not fact derivative work from a GPL work but a derivative work from Windows so GPL license does not travel across this ndis API/ABI. You see the same thing with the Solaris porting layer ZFS uses but this is APi/ABI from Solaris but implemented on Linux results in again another barrier against the GPL derivative work clause.

These wrapper do come with some overhead of course.

[Developer12]

Interesting. Got an links or other details?


My point is that, even in this thread, people make these blind leaps of faith that just because a platform has a Vulkan driver, you'd be able to run any compute job on it.

It's like saying that just because a program is written using OpenGL, it should run on anything with an OpenGL driver, totally ignoring the fact that a lot of devices only supported OpenGL ES, with some not even making it to 3.0, not to mention the matter of extensions.

As far as Mesa is concerned, the only up-to-date drivers are for Intel, AMD, and Qualcomm. Everything else is sitting at 1.0, with less than half the extensions supported, according to MesaMatrix. So, merely appealing to Mesa doesn't get you very far, in the real world.

In other words, Vulkan is far from being a panacea, toward the ideal of a run-anywhere, open compute API. Yet, too many people are still high on the Vulkan hype train and remain blind to this fact. If you actually watch Airlie's presentation, he doesn't have a lot of time for Vulkan. He outlines some of its shortcomings and leaves it at that. I think that says a lot, actually.

The only real requirement to run rusticl is compute shaders need to work. That's pretty much it. (so says karol himself)

You're not trying to run a real game with a full rendering pipeline, or need of a full openGL driver.

[oiaohm]

You're not trying to run a real game with a full rendering pipeline, or need of a full openGL driver.

Not quite. https://en.wikipedia.org/wiki/List_o...ications#Games there are 4 known games that use openCL heavily.

You do find yourself needing either the OpenCL to Opengl or the OpenCL to Direct X integration with these 3 of the 4 games. Basically not the go game Leela Zero this is normal compute shaders.

The 3 other games the Issue is physics calculations sometimes opengl and direct x 9-11 near enough is good enough maths turns out not to work so those 3 games are using opencl to get correct maths on mesh transforms yes this leads to needing integration between rendering pipeline and opencl.

Yes those 3 game are where you need GL_ARB_cl_event in opengl and direct X equal to be able to share mesh objects between the render engine in opengl and direct x and opencl compute shaders in a memory safe and synced way.


Developer12 with only 3 games in existence known to do this evil its not surprising you would not know about it. Yes no mesa driver implements GL_ARB_cl_event in opengl.

There are a few other applications in existence that mix opengl with either Opengl or Direct x under 11 for basically the same things. Yes CADs and other odd ball things.

There are mixed rendering pipelines.

Opencl is not normally a rendering pipeline by it self you normally in the cases it in a rendering pipeline its in 3d mesh transformations and other physic calculations normally where the errors from opengl and direct x 11 and under maths on meshs would be bad.

Way to think of it Opengl/direct x up to 11 the maths on meshs is fine when you are starting out from non modified mesh over and over again. Programs that are using opencl in the render pipe line are doing progressive mesh transformations where the next modification is to the past generated mesh this is where math error after math error that so not fine because this error will add on top of error.