An Open Letter To Tech Review Sites

[nanonyme]

Now go out and spread THAT word. Forget the benchmarking, tell the world that Linux is EASY TO USE and JUST WORKS and WORKS RELIABLY and DOES EVERYTHING and IS FREE. People need to know this in order to make informed decisions.

Linux is really only free if you build your own computer or buy one with no OS installed, otherwise the vendor probably did compatibility testing and makes you pay for it. (does an average user want to buy a computer without an OS? no)

[Michael]

Well the prepared iso has got one huge problem: the kernel is too new to install fglrx drivers. So you can only test with intel onboard or try installing nvidia drivers. fglrx will fail till an updated driver is available. So the usefullness is restricted. Better provide extra images with different kernels till ati manages to create a new fglrx for 2.6.31. Of course i have got my own selection of iso images with PTS with any kernel which is needed

PTS Desktop Live 2009.3 isn't designed to do graphics testing, hence why download caches for all of the graphics tests (except for Tremulous, I believe) were left out. More work on graphics support will come in future PTS Desktop Live releases.

[Kano]

I did not yet find the position of that dlcache. But it can not be that big. Which is your favorite suite to run from it?

[Michael]

I did not yet find the position of that dlcache. But it can not be that big.

/var/cache/phoronix-test-suite/

It's several hundred megs at present.

[Kano]

Right 569M files are there. I would put em in uncompressed area and add some extra path however. Which suites run with those default files?

[lbcoder]

Linux is really only free if you build your own computer or buy one with no OS installed, otherwise the vendor probably did compatibility testing and makes you pay for it. (does an average user want to buy a computer without an OS? no)

That is a pointless argument since if the vendor is going to ship it with *any* OS, they'll STILL have to do compatibility testing. That should be considered as a component of the fixed assembly cost.

You do, of course, bring up an interesting point; shipping computers without OS.... if the average user was interested in and knew how simple it was to install a common Linux distro, they wouldn't care if it shipped with no OS. And this, of course, leads to a new level of conspiracy theories; WHY IS msevil such a PITA to install and set up? Might it be because they WANT the average user to be afraid of installing an OS? To build a market for pre-installed OS? To force hardware vendors to preinstall THEIR MScrapOS? Because the average user will just use *what it came with*? This is, of course, illegal anti-competitive behavior. Also difficult to prove.

It would be nice to see some freedom-FORCING legislation rather than the more common freedom-BLOCKING legislation (i.e. DMCA)... I would like to see something along the lines of "general purpose computing equipment must not ship with any software included or preinstalled", and "vendors of general purpose computing equipment must not distribute or recommend software of any kind under penalty of being tortured to death".

[jntesteves]

This is a very self-serving article. It is designed to promote this site's testing software. Quite frankly, I am absolutely sick of the poorly executed performance tests -- they don't ever illustrate anything significant and just get in the way of the actual important articles, like updates on the status of the R6/700 open source 3d initiative.

From reading your post one could suppose that you're not really interested on benchmarking articles. Well, guess what, a lot of people care about 'em. That's why there are so many on-line publications that do solely that. And the point of the article is that they should use GNU/Linux for that too. I couldn't agree more, since I too ain't not a little bit interested in all those XP/Vista/Seven tests all around the web.

As you said, we should spread the word about Linux. That means we should tell people to test and use it. That's exactly what this article does. But it wasn't aimed at you. It was aimed at other testing publications just like Phoronix. And they should read it.

...and just get in the way of the actual important articles, like updates on the status of the R6/700 open source 3d initiative.

I too, am here just to know the news about the FLOSS drivers for those AMD chips that I use and love. But what would be the point of writing that they are fully 3D capable (in a short future I hope) if it doesn't come accompanied with some benchmarks? I want to know how their performance compare to other drivers. And that may help in their development too so, why not benchmark? I want to know how ATI compare to nVidia on Linux too. And how Tux3 compares to Btrfs/Ext4/ZFS.


Best regards,

Jonatas Esteves

[deanjo]

and the third is grasping at straws since it takes NO resources (last I checked, Linux was free and would run on the same hardware as msevil).

Your taking the word of "resources" out of context. Resources such as a complete readily available and comprehesive benchmark suite or not having the investment capitol / time / or qualifications to develop such a solution. Not hardware resources.

http://en.wikipedia.org/wiki/Resource_%28project_management%29

[jackflap]

We're actually in the process of building something similar

We've just recently built a proof-of-concept site called 'isitopen' which allows users to submit and rate devices to determine how 'open' they are (i.e. source-code available, proprietary plug-connection, etc.)

Originally we had planned to try and make it a community-driven review site, with wiki-style functionality in order to allow our users write specific reviews regarding Linux-hardware, but things evolved away from that.

We're currently trying to get some feedback to see if anyone is interested in this type of website, and where we should go from here.

Are people mostly interested in performance benchmarks? What other benchmarks or data would you be interested in?

http://www.isitopen.biz/

[maniacmusician]

This is a very self-serving article. It is designed to promote this site's testing software. Quite frankly, I am absolutely sick of the poorly executed performance tests -- they don't ever illustrate anything significant and just get in the way of the actual important articles, like updates on the status of the R6/700 open source 3d initiative.

I don't know that makes sense to you. The benchmarks often provide very relevant real-world data, especially as they're based on tools that are regularly utilized in the typical Linux-based environment. It's easy to infer the day-to-day impact from the benchmarks generated here. That's not to say that better and more varied test profiles couldn't be added, but as a reader, I'm assuming that will happen over time. I agree that the typical average user won't care about benchmarks, but this is an in-depth, technology-oriented review and news site. the "R6/700 open source 3d initiative" is still regularly covered and reported on...the benchmarks only supplement it.

Something I would like to see, and I think you might have been getting at this a little bit, is more explanations of real-world impact of some of the benchmarks instead of only a really concise summary at the end. It is easy for someone like me to infer real-world relevance, but definitely won't be for everyone; Actually, it would even make going through the benchmarks easier for me as well.

Of those 3 excuses, the second one is meaningless, and the third is grasping at straws since it takes NO resources (last I checked, Linux was free and would run on the same hardware as msevil). The first excuse is the frightening one that it is important to overcome...

The second one is poorly worded; what it means to say is "lack of benchmarking tools as well as a lack of up-to-date benchmark results to use as a consistent reference point." I would even say "perceived lack" fits better for what the writer is really trying to express with this open letter. As someone else mentioned, the 3rd is talking about resources in the general sense of "something that is necessary to carry out a task, and is available in a limited quantity, thereby increasing the fundamental value of the resource on a day-to-day basis." So that applies to things like time, an easily decipherable knowledge base, or even the availability of staff with a comprehensive understanding of the process and environment they would need to use.

As for the first, while I agree that the learning curve in terms of initial troubleshooting and setup is much diminished, I don't think that's what Michael was referring to. There is still a shift in the interface paradigm that presupposes a certain level of familiarity with the system in order to be able to use it in an effective and (more importantly) flexible way.

There are already people out there spreading the word that Linux hardware compatibility issues are mostly resolved, and that with a small amount of effort and desire, people can now easily revolutionize the way they can use their computers. However, the point of this article is just something entirely different, addressed at a different audience. This is an appeal to publications that have a good deal of influence over the technology-informed public in terms of presenting them with important and relevant information, and it's saying that Linux is ready to be included in that category (and people want it to be). So it's necessary to address the concerns these publications have about making a move like this; the letter is saying "the existing knowledge base makes the learning curve easier, our established and tested benchmarking tools can help you provide useful data, and the combination of those two is going to greatly decrease the impact it will have on your resources. This is now a practical thing for you to try out, and we think you should, because it will benefit both your readership and the Linux/open-source community at large."

And this, of course, leads to a new level of conspiracy theories; WHY IS msevil such a PITA to install and set up? Might it be because they WANT the average user to be afraid of installing an OS? To build a market for pre-installed OS?

I think that's going a little far. Microsoft is a business, and they only think with profits in mind. The reason their installation interface and process sucks is that they have assumed that an end-user will hardly ever carry it out, so why waste their resources on it? They also assume that even when an end-user performs the installation, the hardware manufacturer will provide CDs with necessary drivers to make it hassle-free. It's not a conspiracy theory, it's just cutting corners and focusing development efforts on areas where they think it will pay off more.

Another factor is that the various teams at Microsoft, at least until the point of Vista, had always been disconnected because they thought it more efficient to work that way. So by the time end-user installations became a concern, it would have taken too much effort for them to coordinate all the teams to carry that out together, and it would have required a number of other structural changes/fixes in their code, which they consider wasting resources. For the first time with Vista (I'm fairly sure about this, but not 100%), they tried to get the teams to work together more to try and produce cohesive results. I presume that since it was the first such attempt, it's probably why Vista came out as poorly as it did, and why it was delayed, etc. Unfortunately (for the open source community), they seem to be getting their act together a lot more for the much-hyped Windows 7.

It would be nice to see some freedom-FORCING legislation rather than the more common freedom-BLOCKING legislation

I somewhat agree, though not to the extent that you proposed...obviously the second example was a joke, but I couldn't tell if you were serious about your first example at all. Because that'd be kind of lame too. It would be a great blessing to even at least get the freedom-restricting legislation done away with. But with the sickeningly rapid advent of consumer capitalism, we have basically taken the hands of large corporations, and wrapped them around our necks, saying "Please do whatever you want. I want you to guide our lives to a better future for mankind." So that change in legislation -- well, short of another social revolution (yeah, right), I don't really see that happening for a long, long time. What saddens me the most is that eventually, people might just stop wanting that kind of freedom.