The Fallacy Behind Open-Source GPU Drivers, Documentation

[libv]

All of these things could be improved but doing more takes developers away from the improvements that users want, so the current level of documentation is intended to be roughly optimal given the (relatively small) number of people with solid programming background and enough time to invest. There is no documentation in the world that will make high performance GPU programming easy, unfortunately...

John, you never cease to amaze.

For those who have had to work with you, this reads as: "I am sorry, i cannot do more actual work, because if i would wield my amazing documenting powers, those poor, poor developers will become so flooded and lost and would not be able to do anything any more! I therefor need to spend _more_ of my time answering questions of the ''community'' which is exactly why my blackberry is pointing at the phoronix forums all the time."

I am sure that this is what you are telling your managers and your companies partners now, and i am already in happy anticipation to learn what you will be telling them after that.

[mirv]

and where from the "surplus" of open-source graphic driver developers should be coming from if they have no ground to bread, so to speak ? you know, without docs, and it's not like they teach you basics of hardware for accelerated graphical computation in universities anywhere.
docs for "simple" modern cards like stuff from VIA would be a great example. but nooo, we have 2 giants representing graphical hardware for entire planet and few little shady establishments trying to imitate them or something.

threshold of entry kinda freaking high on this and it will stay that way while this situation persists. and graphic companies managers like it that way, i think.

I don't know about specifics with graphical computation, but there are courses teaching embedded (assembly, C), circuit design, logic circuit design, processor architecture theory, operating system design, etc etc etc. Which doesn't compare to someone with experience, but it can't hurt for a step in the door.
I'm pretty sure AMD does provide very (very) basic examples (like drawing a triangle) to get the ball rolling on the open drivers - there is probably a good place to start.
Sadly, with the hardware complexities of a modern GPU, writing software for the things is equally complex. That's not likely to change any time soon.

[bridgman]

John, you never cease to amaze.

For those who have had to work with you, this reads as: "I am sorry, i cannot do more actual work, because if i would wield my amazing documenting powers, those poor, poor developers will become so flooded and lost and would not be able to do anything any more! I therefor need to spend _more_ of my time answering questions of the ''community'' which is exactly why my blackberry is pointing at the phoronix forums all the time."

I am sure that this is what you are telling your managers and your companies partners now, and i am already in happy anticipation to learn what you will be telling them after that.

Hi Luc;

We might be talking about different things here. The original question was about X/Mesa/DRI/DRM documentation to lower the entry barrier for new driver developers, and mentoring to help new developers come up to speed.

I was trying to say that (a) the mentoring was happening, and (b) having the developers write in-depth driver documentation to help new developers wasn't an automatic win because time spent writing driver documentation would reduce time spent working on drivers, and the increase in developer contributions resulting from the improved documentation wouldn't necessarily make up for the time spent writing docco in the first place.

That said, after making that post it occurred to me that the re-architecture work has died down for a while, so this probably *is* the best time to write up some "getting started with drivers" documentation since documentation written today would probably remain valid for a longer time than if it had been written a couple of years ago.

[libv]

Hi Luc;

We might be talking about different things here. The original question was about X/Mesa/DRI/DRM documentation to lower the entry barrier for new driver developers, and mentoring to help new developers come up to speed.

I was trying to say that (a) the mentoring was happening, and (b) having the developers write in-depth driver documentation to help new developers wasn't an automatic win.

ok.

The current barrier for getting new and clued developers is in my view not the limited mentoring and limited documentation. Lack of structure and modularity are the biggest stumbling block.

A good example is Christian Koenig: he started out adding HDMI audio to radeonhd, because in the radeonhd driver it was highly obvious where he needed to be and it quickly was clear how things fitted together.

Such structure and modularity is not only lacking from graphics drivers today, but is also lacking from the whole graphics driver stack and its infrastructure.

If people know immediately where they need to be, and can build and test the individual part quickly and easily, then mentoring and documentation become far less important.

[libv]

ok.
A good example is Christian Koenig: he started out adding HDMI audio to radeonhd, because in the radeonhd driver it was highly obvious where he needed to be and it quickly was clear how things fitted together.

He then got his results comparatively quickly, and got the feeling of achievement quickly. This experience meant that he then went and did more work, in much harder to access areas.

[energyman]

my problem with you, Mr James, is, that you are filling my inbox with reply notifications while your postings are not even worth the time deleting them. That is my problem. I spent time to read rubbish.

The problem is: only few people can code graphic drivers and now enough about opengl. There could be more, maybe, if some more money would be involved. But there is only so much man power Redhat&co can pay for. Time for some others to step up. Like Canonical.

[Svartalf]

Shortage of open-source driver development manpower is a problem. So is the "you don't want documentation" attitude among vendors. They're related, but different, problems. Acknowledging one does not invalidate the other, and a solution to one does not solve the other.

Got it in one.

The reason there's a shortage of devs doing the work is that it's NOT easy to do (I know, I've done it as FOSS developer with the previous stack, Utah-GLX, and as a paid engineer on a closed driver...) and in many cases the people capable of doing the work are off doing other things (Want to subsidize me? If I got paid as much as I do right at the moment for the consulting work I do elsewhere, I'd drop it all and do the driver work- as much because it needs doing and I understand what needs to be done and could do it. In fact, I was about ready to do it part-time for an embedded systems customer anyway and they opted to use someone more local to them so the dev could be in office occasionally...)

It does NOT help that the information is not available.

It's not a fallacy- it's just not as simple a thing as many make it out to be getting technical info (it used to be...not with shaders, though...it's become quite a bit more complex...) or source code (Which still really, really requires technical info... <*gives VIA a nasty look*>). It's not a case of the emperor has no clothes here. The only successful situation so far has been with Intel and AMD where they provide info, source in many cases, and some assistance as we get up to speed.

[baryluk]

Reasons are simpler. GPU specifications were published by ATI too late, currently GPUs are really complicated. If they had published this specs say 10 years ago, and added next specs are new products was released, it could be much more easier to have complete and and up-to-date drivers.

For my open-source drivers have one main benefit, which is of very big importance
- they are well integrated and are preinstalled in mainline kernel (no installation required, upgrade using standard distribution mechanisms, no messing with libraries, path, compilation, messing with distirbution packages, ABI, API incompatibilities due to other changes, build problems, compilation errors, separate patches, much wider review for security releated problems, easier debuging of problems when something do not work)

Is this not enough?

[dfx.]

Sadly, with the hardware complexities of a modern GPU, writing software for the things is equally complex. That's not likely to change any time soon.

yep, this is exactly the point i was trying to make (and 'bread'='breed' in my first message, douh). and - it would be sweat to have docs for simpler hw around so learning curve would become flater. it's an endless circle.

[marek]

I don't know about specifics with graphical computation, but there are courses teaching embedded (assembly, C), circuit design, logic circuit design, processor architecture theory, operating system design, etc etc etc.

You only need to know C/C++ and how to use 3D APIs to make games or cool graphics demos like those Emil Persson AKA Humus is publishing (good experience with shader programming is absolutely sufficient) in order to be able to write 3D drivers. Every good 3D graphics application developer (like those working for big game companies) already has the required entry-level skills. You can learn the rest as you go.

There is so few driver developers that we manage to answer all questions on IRC. It is highly unlikely that if there was a good startup documentation, anyone would read it. Also reading a documention does not immediately make a good driver developer out of you. If you already know OpenGL, it is not hard to learn how whole Mesa works, though you may get a few headaches in the process. Again, devs usually learn that as they go, one piece at a time, or more precisely, as they write code. The same for kernel DRM.