New X.Org Release Process Has Been Reached

[Bryce Harrington]

No need to be rude. The guy just said that if a company lives from Linux and they have money, it would be an investment if they hired more developers for X.

Well, really he said the following (wildly paraphrased):

1. "Money means nothing to Canonical and they have an infinite amount of it so they should spend more of it." Canonical is a for-profit company with a budget and the goal to become profitable so it is able to contribute to open source as an ongoing concern. This meme that seems to assume Canonical is some sort of open source charity really needs to die.

2. "They'd reap profits if they invested in X development." I see this claimed a lot, but it hand waves the question of where the profits actually come from. Distros who provide free CDs do not make profit off end users, so simply increasing the quantity of end users via better Xorg support obviously does not make profit. What DOES drive profits for Canonical is increased OEM shipments with ubuntu pre-installed. And in fact, this *is* an area Canonical has hired quite a few engineers to work at. This is one of those things that is completely invisible to you when it's working properly.

3. "The real work is really done by hardcore coders like Keith Packard." (As opposed to people who work at Intel/IBM/Canonical?) In fact, Keith's employer is Intel. Also, this is just in general a really inconsiderate statement given how much of the really cutting edge X development is funded by Intel.

4. "Xorg sucks because it needs 1)graphics design and photography, 2)Gaming, 3)Media center." None of those are Xorg. Not to say these areas don't need improvements, but many, many problems tend to get blamed on the X.org project which in reality aren't X.org's responsibility.

5. "I just spend another evening without success to get vdpau working... because the kernel was updated and I use the latest driver downloaded from nvidia." So you've given $$$ to company A, and are demanding that companies B, C, and D, whom you did not give your $$$ to, foot the cost of making X better for users of company A's products. I see...

Seriously though, Xorg is in better shape than people give it credit for. Sure, Nvidia hardware is not well supported in open source. Complaints on blogs like this one aimed at !Nvidia is just going to earn you snarky replies like mine. If you actually really want to put in $200 to help change that, then find some nouveau developer and ask if you can buy them some video cards.

[Apopas]

1. "Money means nothing to Canonical and they have an infinite amount of it so they should spend more of it." Canonical is a for-profit company with a budget and the goal to become profitable so it is able to contribute to open source as an ongoing concern. This meme that seems to assume Canonical is some sort of open source charity really needs to die.

Are you sure he said that? I understood a totally differnet thing as I mentioned above. Maybe you've heard a lot from your postion than me and you're more biased than me towards people and their claims.

2. "They'd reap profits if they invested in X development." I see this claimed a lot, but it hand waves the question of where the profits actually come from. Distros who provide free CDs do not make profit off end users, so simply increasing the quantity of end users via better Xorg support obviously does not make profit. What DOES drive profits for Canonical is increased OEM shipments with ubuntu pre-installed. And in fact, this *is* an area Canonical has hired quite a few engineers to work at. This is one of those things that is completely invisible to you when it's working properly.

You're wrong here. Good Xorg and good drivers mean better multimedia which leads to easier adoption and more games which leads to higher sales and OEM shipments.

[Bryce Harrington]

You're wrong here. Good Xorg and good drivers mean better multimedia which leads to easier adoption and more games which leads to higher sales and OEM shipments.

No. Again, this is hand-wavy profit modeling. It's not like Blizzard is going, "OMG, it's too hard to program graphics on Linux, we can't support it." Rather, they're saying, "It's too hard to make money selling games on Linux." Technical issues with X multimedia performance is irrelevant, these guys are awesome coders that port games to cell phones and whatnot; game manufacturers simply aren't interested, because they aren't seeing the demand from users. Spend your $200 on stamps and ask game manufacturers to port and support Linux, and that will much more directly solve the lack of games than anything X.org could do.

Not saying that X.org multimedia doesn't need to be improved, but I don't think that is the top issue holding back adoption. Proprietary codecs, patent restrictions, manipulative software monopolies, lack of ports of flagship software... these are the real issues that hold back OEMs and end users.

Kissing leads to foreplay, which leads to sex, which leads to more babies and world overpopulation, so to solve global warming we must ban kissing. Um...

[Apopas]

No. Again, this is hand-wavy profit modeling. It's not like Blizzard is going, "OMG, it's too hard to program graphics on Linux, we can't support it." Rather, they're saying, "It's too hard to make money selling games on Linux." Technical issues with X multimedia performance is irrelevant, these guys are awesome coders that port games to cell phones and whatnot; game manufacturers simply aren't interested, because they aren't seeing the demand from users. Spend your $200 on stamps and ask game manufacturers to port and support Linux, and that will much more directly solve the lack of games than anything X.org could do.

Not saying that X.org multimedia doesn't need to be improved, but I don't think that is the top issue holding back adoption. Proprietary codecs, patent restrictions, manipulative software monopolies, lack of ports of flagship software... these are the real issues that hold back OEMs and end users.

No. When I say adoption I mean from the common users, not from the game companies. How could you persuade a casual user who has a radeon card to switch to Linux with the current state of video drivers? To be able to make the switch, he needs something more, not something less and only with enough switch then the game vendors will realize that there is a a viable market in Linux. Only then Blizzard will say now there is a reason to make Linux ports. So we need good Xorg.

Kissing leads to foreplay, which leads to sex, which leads to more babies and world overpopulation, so to solve global warming we must ban kissing. Um...

Unfortunate example. As far as I know the overpopulation comes from the developing countries of Asia and Africa, so the reason is something else. Hell here in Greece we still enjoy sex and we are just 11 millions. You want to stop overpopulation? Give to everyone meds and proper education, you want to stop M$ monopoly? Give to Linux the most advanced benefits that window$ have and these are proper multimedia.

[perpetualrabbit]

I can understand your irritation. I do not want to belittle the great work that is being done. I just think it is going too slow too compete with OS X anytime soon. Competing with OS X is a stated goal by Mark Shuttleworth.

What I am proposing is a fund raise to improve the linux graphics stack, not just Xorg, in strategic places. By this I mean:
* viable 2D, 3D drivers for ati, nvidia, intel. Fast enough for a 3D desktop. Fast enough for to compete with windows on games is a later phase.
* finishing the `infrastructure? overhaul (i.e. dri2, gallium3d, kms etc)
* a holistic approach to the whole graphics stack. Step back and see what is missing in each subsystem _for a certain goal_ and help those subsystems out. I mean help the people who are working on it out.
* To do this there needs to be a central organisation with means. Canonical is such an organisation. For instance, I cannot send $200 to a random nouveau developer to buy a video card. How do I come in contact with him? But if enough people pay some money to Canonical, Canonical could buy a number of videocards (say 3 of each major type of nvidia card) and lend them out to developers. You could start a kind of public library for hardware, were developers can loan stuff to develop drivers for. Such a library would be a part of the 2 million dollar plan I was proposing.

how many people work on `core? linux graphics?
----------------------------------------------
I don't really know the specifics of who does what in the linux graphics world. I just estimate that about 30 people are pulling the carts so to say. How many work _fulltime_ on nouveau: about 5? How many on KMS: another 5? The radeon driver: 2 at Novell, there were 3 but one was let go. If I extend this reasoning to the other big subsystems: gallium3d, dri2, etc. I get about 30 people. I count about 6 major pieces like gallium, dri, kms, mesa. If each of those has 5 fulltime developers, I get 30. I could be wrong by a factor of 3, but not by a factor of 10.

What is needed to speed things up?
----------------------------------
20 extra dedicated fulltime people make a difference. I think that a good programmer earns about $100k, so that makes 2 million per year. I think there are enough linux users with a day job who are willing to pay about $200 per year, if they know this money is spent on solving the linux graphics problems once and for all. I think there are (tens of) thousands of us.
First there needs to be a holistic view, a plan, for the whole graphics system. Since the users needs are diverse you need to set certain goals, and reach these goals in phases. For each goal you need to identify which parts are weak or missing. Then you assign people to work on those particular subsystems. For instance: you have the normal desktop user. You have gamers. Graphics designers. Teachers who want to set up a cheap multiseat system. And so on.

Goal for 1st year: All PCs with an ati, intel or nvidia card can load a hardware accelerated 3D linux flicker-free desktop. Not the highest framerates yet, but very usable. Subsystems that need help: xorg, dri2, driver teams (radeon, nouveau, intel), gallium3d, kms.

Goal for 2nd year: Playing and developing 3D games on linux is as good as in windows, and watching HD movies works. Needed for this goal: speed up the whole graphics stack. Start a game development platform project, maybe based on design involving parts of Eclipse, Blender, Gimp, open sourced 3D engines. For HD viewing you need a open source vdpau clone, and propers support in {xine, mplayer, vlc, mythtv}. Subsystems involved: for the speedup everyone, for the game IDE each of the projects mentioned, for HD viewing the driver teams again, plus the xine, mplayer, vlc and mythtv communities.

Goal for 3rd year: Adress specialed needs: >8 bit per color channel for photographers, to compete with OS X. Multi seat, multi touch, multi head, rotation are easily configurable out of the box. Involved subsystems: at least Xorg and drivers, probably many others too.

Anyway: set a goal, identify work to be done in each subsystem, identify the projects and people involved, and offer help in a way that makes sense. Could be to pay someone to do it full-time, could be buying hardware for developers and taking care of loaning it to them (the library idea). Could be hiring new people to help out a project, could be starting a whole new project. I think on average you need to pay 20 specialized people, which costs about 2 million, which you can find sponsors for if you set the goals clearly enough.
For this you need a central vision, and a central organisation.

Does this make more sense?

[perpetualrabbit]

Well, really he said the following (wildly paraphrased):

1. "Money means nothing to Canonical and they have an infinite amount of it so they should spend more of it." Canonical is a for-profit company with a budget and the goal to become profitable so it is able to contribute to open source as an ongoing concern. This meme that seems to assume Canonical is some sort of open source charity really needs to die.

I don't say that at all. Improving the linux graphics stack to say OS X level would make Canonical much more able to compete though. Therefore _indirectly_ able to profit more.

[QOUTE]2. "They'd reap profits if they invested in X development." I see this claimed a lot, but it hand waves the question of where the profits actually come from. Distros who provide free CDs do not make profit off end users, so simply increasing the quantity of end users via better Xorg support obviously does not make profit. What DOES drive profits for Canonical is increased OEM shipments with ubuntu pre-installed. And in fact, this *is* an area Canonical has hired quite a few engineers to work at. This is one of those things that is completely invisible to you when it's working properly.[/QUOTE]

The profit is indirect. A better product means better chance at landing OEM contract maybe?

3. "The real work is really done by hardcore coders like Keith Packard." (As opposed to people who work at Intel/IBM/Canonical?) In fact, Keith's employer is Intel. Also, this is just in general a really inconsiderate statement given how much of the really cutting edge X development is funded by Intel.

I did not say any of these things. I know who Keith Packard works for, I know about the good works of Intel, and I know that IBM, Canonical, Novell, Redhat, all do their parts, and roughly even know what that part is. Still, all in all I estimate about 30 people working on the graphics overhaul. And not in a very focused manor, as in say getting an iPhone out the door, and keeping track of getting to work all the subsystems of an iPhone. It seems more like each project to themselves to me.

4. "Xorg sucks because it needs 1)graphics design and photography, 2)Gaming, 3)Media center." None of those are Xorg. Not to say these areas don't need improvements, but many, many problems tend to get blamed on the X.org project which in reality aren't X.org's responsibility.

I never said Xorg sucks. In fact, I think after years of stagnation during the Xfree86 times, X got modularized, the drivers decoupled from the server, code got cleaned up, it builds better etc. Some of these things started in de Xfree86 days btw.
Yes, I said work is needed for photography, games, media center. Lots of pieces of work, some of which are in Xorg, some in lots of other places. Therefore there needs to be a holistic view, and a group (probable paid) to bring it together. Faster than is happening now.

5. "I just spend another evening without success to get vdpau working... because the kernel was updated and I use the latest driver downloaded from nvidia." So you've given $$$ to company A, and are demanding that companies B, C, and D, whom you did not give your $$$ to, foot the cost of making X better for users of company A's products. I see...

Well, read my comments above, again, I said nothing of the sort. I am saying pay a number of developers to reach a certain well-defined level of `working with linux?. Identify which subsystems are involved and help those projects with money, hardware or hire specialists in that field.

Seriously though, Xorg is in better shape than people give it credit for. Sure, Nvidia hardware is not well supported in open source. Complaints on blogs like this one aimed at !Nvidia is just going to earn you snarky replies like mine. If you actually really want to put in $200 to help change that, then find some nouveau developer and ask if you can buy them some video cards.

Well, I counter that with proposing Canonical starts a library where developers can loan not books but hardware. This costs money for hardware, postage and people (1, not fulltime) managing it, designing a website for it. Not that much money for hardware, maybe $10,000 to buy a few specimens of the major types of videocard. $3000 for postage.
One person to set up the website, one person who spends maybe a day per week on average to handle requests. Probably less.

[Apopas]

Very well said perpetualrabbit, very well said. I suppose the Linux foundation could undertake the holistic projec t you mentioned rather than a specific Linux company like Canonical or Redhat.

[kernelOfTruth]

I can understand your irritation. I do not want to belittle the great work that is being done. I just think it is going too slow too compete with OS X anytime soon. Competing with OS X is a stated goal by Mark Shuttleworth.

What I am proposing is a fund raise to improve the linux graphics stack, not just Xorg, in strategic places. By this I mean:
* viable 2D, 3D drivers for ati, nvidia, intel. Fast enough for a 3D desktop. Fast enough for to compete with windows on games is a later phase.
* finishing the `infrastructure? overhaul (i.e. dri2, gallium3d, kms etc)
* a holistic approach to the whole graphics stack. Step back and see what is missing in each subsystem _for a certain goal_ and help those subsystems out. I mean help the people who are working on it out.
* To do this there needs to be a central organisation with means. Canonical is such an organisation. For instance, I cannot send $200 to a random nouveau developer to buy a video card. How do I come in contact with him? But if enough people pay some money to Canonical, Canonical could buy a number of videocards (say 3 of each major type of nvidia card) and lend them out to developers. You could start a kind of public library for hardware, were developers can loan stuff to develop drivers for. Such a library would be a part of the 2 million dollar plan I was proposing.

how many people work on `core? linux graphics?
----------------------------------------------
I don't really know the specifics of who does what in the linux graphics world. I just estimate that about 30 people are pulling the carts so to say. How many work _fulltime_ on nouveau: about 5? How many on KMS: another 5? The radeon driver: 2 at Novell, there were 3 but one was let go. If I extend this reasoning to the other big subsystems: gallium3d, dri2, etc. I get about 30 people. I count about 6 major pieces like gallium, dri, kms, mesa. If each of those has 5 fulltime developers, I get 30. I could be wrong by a factor of 3, but not by a factor of 10.

What is needed to speed things up?
----------------------------------
20 extra dedicated fulltime people make a difference. I think that a good programmer earns about $100k, so that makes 2 million per year. I think there are enough linux users with a day job who are willing to pay about $200 per year, if they know this money is spent on solving the linux graphics problems once and for all. I think there are (tens of) thousands of us.
First there needs to be a holistic view, a plan, for the whole graphics system. Since the users needs are diverse you need to set certain goals, and reach these goals in phases. For each goal you need to identify which parts are weak or missing. Then you assign people to work on those particular subsystems. For instance: you have the normal desktop user. You have gamers. Graphics designers. Teachers who want to set up a cheap multiseat system. And so on.

Goal for 1st year: All PCs with an ati, intel or nvidia card can load a hardware accelerated 3D linux flicker-free desktop. Not the highest framerates yet, but very usable. Subsystems that need help: xorg, dri2, driver teams (radeon, nouveau, intel), gallium3d, kms.

Goal for 2nd year: Playing and developing 3D games on linux is as good as in windows, and watching HD movies works. Needed for this goal: speed up the whole graphics stack. Start a game development platform project, maybe based on design involving parts of Eclipse, Blender, Gimp, open sourced 3D engines. For HD viewing you need a open source vdpau clone, and propers support in {xine, mplayer, vlc, mythtv}. Subsystems involved: for the speedup everyone, for the game IDE each of the projects mentioned, for HD viewing the driver teams again, plus the xine, mplayer, vlc and mythtv communities.

Goal for 3rd year: Adress specialed needs: >8 bit per color channel for photographers, to compete with OS X. Multi seat, multi touch, multi head, rotation are easily configurable out of the box. Involved subsystems: at least Xorg and drivers, probably many others too.

Anyway: set a goal, identify work to be done in each subsystem, identify the projects and people involved, and offer help in a way that makes sense. Could be to pay someone to do it full-time, could be buying hardware for developers and taking care of loaning it to them (the library idea). Could be hiring new people to help out a project, could be starting a whole new project. I think on average you need to pay 20 specialized people, which costs about 2 million, which you can find sponsors for if you set the goals clearly enough.
For this you need a central vision, and a central organisation.

Does this make more sense?

great post perpetualrabbit !

Where's the paypal account for transferring the 200 $$ ?

I don't have the time to mess around with dozens of different driver settings and/or drivers

give me a company who takes care of this since life's to short (for me) to "waste" it with these kind of things - my $$ talks (would do so) so do the best with it to solve the problems that really need to be solved first and are important for most of the users

[thefirstm]

I don't say that at all. Improving the linux graphics stack to say OS X level would make Canonical much more able to compete though. Therefore _indirectly_ able to profit more.

You must mean "regress" rather than improve? The OSX graphics stack is incredibly poor! Have you ever seen how laggy things like menu selections are on OSX? How about on 10.6, where you can't even use OpenGL 3.x because of OpenCL?

[adrianveidt]

I'd like to respond to perpetualrabbit's original question, which is:

So in short, is it money, is it people, is it both?
Or is it lack of leadership and focus?

As I understand the issue, the xorg codebase is so complex and esoteric that it is, in Daniel Stone's words, "unapproachable". Here are quotes from recent mailinglist posts by Stone:

certainly we lack badly for developers, and a lot of the XDC
discussion was on how to make the codebase more approachable so we can
get more testers, casual contributors, and full-on developers involved.
We've got a long way to go.

reference: http://lists.x.org/archives/xorg-dev...er/002338.html

...so we can start getting a coherent API (well, any API would be a start).

reference: http://lists.x.org/archives/xorg-dev...er/002330.html

An API "determines the vocabulary and calling conventions the programmer should employ to use the services. It may include specifications for routines, data structures, object classes and protocols used to communicate between the requesting software and the library." according to wikipedia.

I'm no hacker, but I've developed a half dozen WordPress plugins because that API is well-documented.
If a massive thing like xorg has no API, that is what someone might call a problem. It means, to use the word I used earlier, this codebase is quintessentially esoteric, ie. understood only by the initiated, or by those like Stone and Packard et al. who have worked on it for many years.
Money will not solve that problem, and neither will more developers. Assuming I'm right about what the problem is, I don't know how you take such a large code base and add an API without throwing everything out and starting over.