A First Look At The 2010 Linux Graphics Survey Results

[droidhacker]

I think Michael is busy with XDS proceedings... he's probably (close to) finding himself in France, so he's got more going on than usual. He's got a lot on his plate. Give him time.

Before this turns into a Michael bashing thread, can people please assume good faith on Michael's part, that he accidentally misinterpreted the results based on his pie charts? Give him the benefit of the doubt that he isn't trying to misrepresent the situation. And give him an extra amount of time to fix the article because he is busy with a significant yearly event right now.

Yes, it's wrong and should be fixed, but no need to get up in arms over it.

I don't think that anyone is interpreting this as intentional misrepresentation. Everyone is aware that making errors is part of the condition of being human. To top it off, we now know that Michael isn't a robot.

[kazetsukai]

OK, before I give my opinions on the matter:

What exactly is the problem with these results? I can see they'll rub some camps the wrong way, but that is just how polls and surveys go. This isn't Michael trying to prove something- its a freakin poll.

I had to use multiple choice for certain questions- such as my use case for Linux. I use it for Professional 2D/3D, gaming and (not listed) work.

I guess it the fact that not enough people checked the "open source" box pisses some people off, but that is the choice made by the person filling out this information, not the person that put up the poll.

For the multiple choice, I'll give you that a pie chart didn't make alot of sense. Bar graphs for each with option/total entries makes alot more sense.

And now for my opinion-

But really? "omg not enough people picked open source"? The number of people that did so was very low, and it doesn't matter how you graph it- people picked practicality and functionality over open source. Since the -mass majority- of people that filled out this form reported themselves as 'mainstream users', this should really come at no surprise.

As far as I'm concerned, neither is really negotiable in a driver. Neither the freedoms that Open Source brings, nor the Full Functionality that the hardware provides. A driver should fulfill both to be considered a great driver- and unfortunately, nothing exists that fits that description. I'm stuck with a binary for the time being, I guess.

[allquixotic]

OK, before I give my opinions on the matter:

What exactly is the problem with these results? I can see they'll rub some camps the wrong way, but that is just how polls and surveys go. This isn't Michael trying to prove something- its a freakin poll.

The problem is that a key piece of statistical information is missing from the results. Based on the information available, it is mathematically impossible to determine the number of people who ticked the open source box, either in absolute numbers or by percentages. All we need to know is the total number of boxes that were ticked across all respondents for each question. If that made no sense to you, go back and re-read my previous post in this thread.

I had to use multiple choice for certain questions- such as my use case for Linux. I use it for Professional 2D/3D, gaming and (not listed) work.

Yes, and the multiple answers nature of the questions makes the results somewhere between misleading and plain wrong.

I guess it the fact that not enough people checked the "open source" box pisses some people off, but that is the choice made by the person filling out this information, not the person that put up the poll.

It's not that not enough people checked the box. In fact, by my estimate, as many as 50% of the people checked the box, which isn't half bad. But we can't know for sure until Michael publishes the needed information. It all depends on how many checkboxes in that question the typical person checked -- if it is a high number relative to the number of options, then the 9.38% open source on the pie chart is rather significant. If the average number of checks is low, then the resulting actual % wouldn't be much higher than what it is now. As I said before, I estimate that, given 11 options, most people would pick at least 5, meaning that a conservative estimate of the actual % of people who picked open source is at least 40%. I don't have any agenda to want to game the statistics to make it seem like more people checked it than reality, but at the same time, it is extremely dishonest to say that 9.38% of the people checked the open source box, because it's plain wrong, by a factor of at least 4 or 5, due to the way the statistics were aggregated by Michael.

For the multiple choice, I'll give you that a pie chart didn't make alot of sense. Bar graphs for each with option/total entries makes alot more sense.

Right, and it's not just the visual presentation that's wrong, but the actual data that can be gleaned from it.

But really? "omg not enough people picked open source"?

No, "omg the statistics are misleading and the actual number is by all estimates very much higher". If the actual number of people who checked open source were 9.38%, I would probably say "omg not enough people picked open source"; but I can't even arrive at that conclusion as it stands because I lack the information to draw that conclusion.

The number of people that did so was very low, and it doesn't matter how you graph it- people picked practicality and functionality over open source. Since the -mass majority- of people that filled out this form reported themselves as 'mainstream users', this should really come at no surprise.

No, you have no idea of the number of people who picked open source. If you think you do, then you don't understand statistics.

Think of it this way. The single most popular answer, "3D / OpenGL Performance", is 16.54% of the pie chart. Open source is 9.38%. The portion of the pie chart given to the most popular answer is only 1.8 times higher than the portion allocated to open source. Not even double the frequency. Do you see now why the actual number of people who thought open source is important is much higher than just 9.38%? The problem is that nobody except Michael knows exactly how much higher. The actual number is statistically relevant if it's more than 5% different than 9.38%.

As far as I'm concerned, neither is really negotiable in a driver. Neither the freedoms that Open Source brings, nor the Full Functionality that the hardware provides. A driver should fulfill both to be considered a great driver- and unfortunately, nothing exists that fits that description. I'm stuck with a binary for the time being, I guess.

You're right that in an ideal world we could have our cake and eat it too, but nobody is trying to make a point that disagrees with your "opinion" on this matter. We're just looking at the numbers and saying, "before you misinterpret this, stop and think about the statistics for a minute, and then you'll realize that we can't draw any conclusions from the information available".

[droidhacker]

I had to use multiple choice for certain questions- such as my use case for Linux. I use it for Professional 2D/3D, gaming and (not listed) work.

I guess it the fact that not enough people checked the "open source" box pisses some people off, but that is the choice made by the person filling out this information, not the person that put up the poll.

An example might be better able to demonstrate the flaw in the way the information is processed and presented;

You have a survey with one of those multiple-select options. TEN people completed the survey, ALL of them selected the "open source" option.... but ALL of them ALSO selected all of the other 10 options (there were a total of 11 options in that question). Based on the method used to process and present this information, it would be presented as ~9% open source, when in REALITY, ***EVERY SINGLE RESPONDENT*** (i.e. 100%) selected open source.

Like this: you have your list of results for each question:
1: 10
2: 10
3: 10
4: 10
5: 10
6: 10
7: 10
8: 10
9: 10
10: 10
11: 10
TOTAL QUESTIONS ANSWERED: 110.
proportions:
1: 10/110=9%
2: 10/110=9%
3: 10/110=9%
4: 10/110=9%
5: 10/110=9%
6: 10/110=9%
7: 10/110=9%
8: 10/110=9%
9: 10/110=9%
10: 10/110=9%
11: 10/110=9%
**** ALL INCORRECT.
The correct way to process these numbers isn't to divide by the total number of answers, but by the total number of RESPONDENTS. The CORRECT results (for this example) are:
1: 10/10=100%
2: 10/10=100%
3: 10/10=100%
4: 10/10=100%
5: 10/10=100%
6: 10/10=100%
7: 10/10=100%
8: 10/10=100%
9: 10/10=100%
10: 10/10=100%
11: 10/10=100%
*** And this CAN'T be presented on a pie chart since a FULL PIE represents 100%. It *HAS* to be presented on a bar graph.

You see the flaw? Obviously showing 9% when it should be 100% is very very incorrect. Note that it is a coincidence that the value Michael gave for open-source was close to 9% (it was 9.38%). It would obviously go up for every other option that somebody did NOT select, and would go down for every respondent that did NOT select it. My *guess* is that the real number is probably somewhere in the 50-60% range (this is, of course, a wild guess that is not based in fact).

For the multiple choice, I'll give you that a pie chart didn't make alot of sense. Bar graphs for each with option/total entries makes alot more sense.

More for the reason I've explained than just the matter of it making more or less sense to your way of perceiving it.

But really? "omg not enough people picked open source"? The number of people that did so was very low, and it doesn't matter how you graph it- people picked practicality and functionality over open source. Since the -mass majority- of people that filled out this form reported themselves as 'mainstream users', this should really come at no surprise.

I ABSOLUTELY GUARANTEE 100% that the proportion of people who selected open-source was VASTLY greater than the reported 9.38%. The complaints you've read about this have NOTHING to do with people arguing about whether they think open source is important or not, but by the fact that the numbers are simply INCORRECT.

Just add up the results of that question to confirm:
9.38+7.81+3.68+12.1+16.94+15.52+6.83+16.54+8.34+2. 24+.61 = 100%.
This could ONLY be true IF EVERY respondent checked off PRECISELY ONE option.

Note: If you haven't taken a college level statistics course, I highly recommend that you do. Very useful for data analysis.

As far as I'm concerned, neither is really negotiable in a driver. Neither the freedoms that Open Source brings, nor the Full Functionality that the hardware provides. A driver should fulfill both to be considered a great driver- and unfortunately, nothing exists that fits that description. I'm stuck with a binary for the time being, I guess.

That does depend on your perspective.
Actually, the open source drivers have FAR FAR MORE functionality than the blobs (i.e. KMS). What the blobs have over the open source drivers is 3D performance, which may or may not matter to some particular user. For my use, I don't really care about having the absolute best possible performance. What **I** am looking for in graphics is FLEXIBILITY and STABILITY. I use an AMD 4290 IGP to drive TWO 1920x1200 digital displays. I like good 2D performance, and my 3D needs are VERY modest... to the point of being almost non-existent.

So from MY perspective, the open source Radeon driver *IS UNQUESTIONABLY* a GREAT driver since it FULLY satisfies MY needs. In fact, the blob driver CAN'T do what I require. The blob driver CAN'T (for technical reasons) drive two digital displays on my hardware. You can go ahead and ask Alex (agd5f) if you don't believe me. And note that the REASON why the open source radeon driver CAN do what I need it to do is PRECISELY BECAUSE it is open source

[mutlu_inek]

I think Michael is busy with XDS proceedings... he's probably (close to) finding himself in France, so he's got more going on than usual. He's got a lot on his plate. Give him time.

Before this turns into a Michael bashing thread, can people please assume good faith on Michael's part, that he accidentally misinterpreted the results based on his pie charts? Give him the benefit of the doubt that he isn't trying to misrepresent the situation. And give him an extra amount of time to fix the article because he is busy with a significant yearly event right now.

Yes, it's wrong and should be fixed, but no need to get up in arms over it.

So, is it now time to "get up in arms over it"? It has been quite a while. The results are still wrong. They are still FUD that harms open source, simply due to Michael's lack of basic math skills.

Michael, why don't you do anything about this? You could simply edit the article. It should be relatively little work. You might not be able to undo damage you did, but prevent further damage.

MICHAEL, THIS IS UNACCEPTABLE!

[devius]

My vote goes to droidhacker for helping out in the phoronix statistics department He clearly knows what he's talking about so why not take advantage of that to make phoronix better?

[allquixotic]

BTW, of possible interest:

2009 Linux Graphics Survey

• He used bar graphs instead of pie charts, so we can actually call the pie charts a regression
• 43.37% of the people clicked Licensing / Open-Source as an issue (6000 out of 13836 respondents ticked the box)
• Probably around the same percentage of people ticked the box this year, if not more; there's no basis for believing that the number would have regressed all the way to 9.38% or even close to it.

BTW, Michael is at Oktoberfest now, so he's still "busy" and probably therefore lacking the time to go back over this complicated article. But I won't apologize for him; this is still wrong and needs to be fixed.

I wonder if he reads these comments or whether we need to start showing up at his events and bugging him about it in person...

[Lukian]

Nice chart bro!

http://www.phoronix.com/data/img/results/lgs_2010_results/8.png

Who's feeling the 'multiple' love tonight?

[dotancohen]

You didn't think thoroughly about resolution choices. Come on, today's laptops are often equipped with a 1440 x 900 screen - to which category would You qualify it, less than 1280 x 1024 or less than 1600 x 1200?

1440x900=1296000
1280x1024=1310720
1600x1200=1920000

That would be less than 1600x1200.

[RCL_]

Hmm...

On the diagram "How many display(s) do you have", page 3, there are several "multiple" columns which probably need to be summed up?

Otherwise it looks like that Single display gets unfair advantage.