The Fallacy Behind Open-Source GPU Drivers, Documentation

[airlied]

Or can they?

No they can't, the bar can only be so low. At some point you have to put the effort into work out how the code works, and that involves doing hard work and reading code. I think reading code scares more people off, they think productivity is in the writing of code which isn't always a good thing.

You seem to think you can write a driver for GPU hardware like you can write apps for a CPU, this isn't the case. Writing a GL app for a GPU is similiar to write an app for CPU, writing a driver for a GPU involves getting to the hardware, like in theory at least for the 3D driver you are in a sandbox, unfortunately the sandbox is made from cheese with lots of holes in it, so you can crash the GPU. Writing a KMS kernel driver involves reloading the base driver for the graphics subsystem, I'm not sure how you think that could work in some sort of isolated sandbox.

Dave.

[kirillkh]

[QUOTE=airlied;165133]I think reading code scares more people off, they think productivity is in the writing of code which isn't always a good thing.[QUOTE]
Getting acquainted with the code is a separate problem.

What I've been talking about is preventing the pain of learning through trial-and-error, where every error results in a system crash. It's a hindrance even when you have a separate system for that, but when you only have one system, it's insurmountable.

You seem to think you can write a driver for GPU hardware like you can write apps for a CPU

I know that's not how things are, but that's how I'd like them to be.

writing a driver for a GPU involves getting to the hardware, like in theory at least for the 3D driver you are in a sandbox, unfortunately the sandbox is made from cheese with lots of holes in it, so you can crash the GPU.

Let's see:
1. Can you (as a kernel developer) maintain a global, stop-the-world, lock on the GPU? Or freeze the GPU when you want to?
2. Can you intercept all attempts to change the GPU state?
3. Can you prevent a GPU crash from crashing the whole system?
4. Can you dump the whole state of the GPU to the main memory?
5. Can you restore the GPU state from a previous in-memory dump?

If you can achieve these 5 things (maybe not 100% of the time, with some quirks, etc.), then you can run your test system in a VM and switch back-and-forth. The guest kernel module would simply re-route all commands to the host kernel module. The host kernel module would intercept all attempts to change the GPU state and either execute or discard them (depending on which context is active currently).

I'm sure things are more complicated than that, e.g. if the host system tries to change the GPU's state while the guest system is in control. But that could be solved or worked around, too. (E.g.: never run 3D or 2D apps in the host system while debugging?)

[Nobu]

Closed-source software should be illegal--if it were, you would immediately be able to tell whether someone copied someone else's work, the legal stuff could be gotten out of the way, and we could continue with our lives without worry.

We might need to make some changes to copyright law and patent law so as not to congest the courts (i.e., make software non-patentable and copyright a little more lenient for software.), but I think there would be much more innovation in both hardware and software.

[glisse]

[QUOTE=kirillkh;165145]

Let's see:
1. Can you (as a kernel developer) maintain a global, stop-the-world, lock on the GPU? Or freeze the GPU when you want to?
2. Can you intercept all attempts to change the GPU state?
3. Can you prevent a GPU crash from crashing the whole system?
4. Can you dump the whole state of the GPU to the main memory?
5. Can you restore the GPU state from a previous in-memory dump?

If you can achieve these 5 things (maybe not 100% of the time, with some quirks, etc.), then you can run your test system in a VM and switch back-and-forth. The guest kernel module would simply re-route all commands to the host kernel module. The host kernel module would intercept all attempts to change the GPU state and either execute or discard them (depending on which context is active currently).

I'm sure things are more complicated than that, e.g. if the host system tries to change the GPU's state while the guest system is in control. But that could be solved or worked around, too. (E.g.: never run 3D or 2D apps in the host system while debugging?)

1 definitive no, current gpu are not preemptible, hw limitation
2 yes
3 definitive no on x86,x86-64,ppc might be doable on others arch but i doubt any arch can recover when the pci bus goes down.
4 not in the way you think, you cannot stop gpu while gpu is busy, you can stop gpu while gpu is idle and save all it's memory (we might be talking about gigs of ram here so you would need a config with 16G or more of ram)
5 you cannot restore the gpu in arbitrary state, only restore it's memory and resume sending cmd to it (linked with point 4)

Trust us, there is no sane hw development in VM, GPU driver is about crashing your computer and in my experience i can spend 50% of my time at rebooting computer, that's why it's good to have several computer so you not just stare at boot process.

That being said, high level driver such as a GL driver can be done without much hassle, i think for r600g i am likely at well less than 100 gpu lockup with reboot in 8month time of development. While working on some others things might lead to more than 100 reboots a day. So most of the high level things can be done without too much trouble, if you know enough to not ask stupid things to the gpu.

[kirillkh]

there is no sane hw development in VM, GPU driver is about crashing your computer and in my experience i can spend 50% of my time at rebooting computer, that's why it's good to have several computer so you not just stare at boot process.

It's really weird to me that AMD have not made their hardware capable of crashless development. One must be an hero to work in these conditions. 'Nuff said.

[bridgman]

It's obviously possible in principle to simulate not only the GPU but the driver developer as well, but in both cases you need full access to the internal hardware design. You can't get the internal design details of the developer without killing them, and we have no intention of giving anyone the hardware internal design.

Kirillkh, we can perform the functions you listed on a hardware emulation farm, but given the cost (maybe $30M per system) and the performance (maybe 1/1000th of real time) most of the work is done other ways.

As glisse said, working without a system like that is normally not much of a problem 99% of the time.

[energyman]

there is no such thing as 'crashless development'. Except maybe usb devices. ...

[bridgman]

It's really weird to me that AMD have not made their hardware capable of crashless development. One must be an hero to work in these conditions. 'Nuff said.

Remember that nearly all of the driver development (everything except the open source driver work) is done in house with access to hardware emulators and other specialized tools which depend on also having access to the hardware design.

Alex and Richard also have access to the hardware emulators (or, more precisely, access to the engineers who work on the emulators).

[kirillkh]

It's obviously possible in principle to simulate not only the GPU but the driver developer as well

Cool! Next step is simulating the Universe, then we can all finally move on to doing something else. Oh wait...

we have no intention of giving anyone the hardware internal design

Implementing "freeze" and "dump"/"restore" instructions would not require internal design exposure.

As to preventing the crash from affecting the whole system, I guess it's the kernel's responsibility? Frankly, I thought that kind of basic stuff was in place for ages... With the level of modularity of a modern system and with the level of control that the kernel has over it, it should be able to reset an arbitrary device and stay afloat.

As glisse said, working without a system like that is normally not much of a problem 99% of the time.

One can learn how to be careful and avoid crashing their system. However, reaching that level is the problematic part here. I had a similar experience with a large program that had to be recompiled after changing a file. You can't reasonably expect people to willingly subject themselves to that kind of pain during their first several months, which is accidentally the time that requires the most experimentation. Even if you use the pauses for something else, it can disrupt the "flow", i.e. you get distracted and lose concentration.

Remember that nearly all of the driver development (everything except the open source driver work) is done in house with access to hardware emulators and other specialized tools which depend on also having access to the hardware design.

Not sure how that helps the FOSS developers.

Alex and Richard also have access to the hardware emulators (or, more precisely, access to the engineers who work on the emulators).

This can't count. Free software development should not depend upon access to the corporate resources. It's nice that today you work together with the FOSS team. But the Lord giveth, and the Lord taketh away.

[bridgman]

Cool! Next step is simulating the Universe, then we can all finally move on to doing something else. Oh wait...
<snip>
This can't count. Free software development should not depend upon access to the corporate resources. It's nice that today you work together with the FOSS team. But the Lord giveth, and the Lord taketh away.

Not sure how to reply to this. You're basically saying that we should redesign our products to make development less stressful for open source driver developers. Honestly, I don't think that is likely to happen.

Each generation of GPU has a few more features to support troubleshooting and performance tuning, and we will try to make those available to open source devs where possible, but I really can't say that we are likely to make substantial changes to our hardware just to simplify driver development outside AMD. I would rather see investment going to areas that make it possible to expose *more* functionality to the open source driver devs.