Last month I said what OpenBenchmarking.org is and how it should change the benchmarking / automated testing landscape once it's released in conjunction with Phoronix test Suite 3.0 "Iveland" early next year. I have also showed off the new graphing capabilities for this software and provided another update at the end of last month. Here now is another update with some more exciting details.
In recent weeks and months there has been quite a bit of work towards improving the responsiveness of the Linux desktop with some very significant milestones building up recently and new patches continuing to come. This work is greatly improving the experience of the Linux desktop when the computer is withstanding a great deal of CPU load and memory strain. Fortunately, the exciting improvements are far from over. There is a new patch that has not yet been merged but has undergone a few revisions over the past several weeks and it is quite small -- just over 200 lines of code -- but it does wonders for the Linux desktop.
Now that the Linux 2.6.37 kernel merge window is closed and this next major release is in the middle of its development cycle, we have new benchmarks to publish looking at the file-system performance of Btrfs and EXT4 compared to earlier releases. The Linux file-system performance is under constant evolution as shown by our five years of Linux kernel benchmarks and more recent file-system-focused articles such as looking at EXT4 and Btrfs regressions in Linux 2.6.36, solid-state drive Linux benchmarks, and even ZFS-FUSE benchmarks, among other similar articles.
LLVM 2.8 was released last month with the Clang compiler having feature-complete C++ support, enhancements to the DragonEgg GCC plug-in, a near feature-complete alternative to libstdc++, a drop-in system assembler, ARM code-generation improvements, and many other changes. With there being great interest in the Low-Level Virtual Machine, we have conducted a large LLVM-focused compiler comparison at Phoronix of GCC with versions 4.2.1 through 4.6-20101030, GCC 4.5.1 using the DragonEgg 2.8 plug-in, LLVM-GCC with LLVM 2.8 and GCC 4.2, and lastly with Clang on LLVM 2.8.
While we have conducted studies related to the Linux kernel performance in the past such as benchmarking up to twelve kernel releases, going out the door this morning are the results from the largest-ever Linux kernel comparison conducted at Phoronix, and very likely the largest ever of its kind regardless of source. Every major Linux kernel release from Linux 2.6.12, which was released in mid-2005, up through the latest Linux 2.6.37 development code was tested. This represents the past five years of the Linux kernel and shows how the performance has evolved over the past 25 stable kernel releases and the most recent 2.6.37 development kernel.
Phoronix Test Suite 3.0 (codenamed "Iveland") has been under heavy development for more than a month and there is still at least three more months left of work before this major release will be christened. Today though it is time to publicly share the first details (aside from those that learned about it in the Augustiner tent at Oktoberfest) for one of the new components to be making up a critical piece of the Phoronix Test Suite 3.0 platform: OpenBenchmarking.org.
With three laptops representing different generations of mobile hardware, we loaded up the past four stable releases of Fedora Linux plus the most recent Fedora 14 Alpha release and then carried out an arsenal of tests looking at how the battery power consumption rate has changed since 2008. If you are concerned at all about running Linux on your battery-powered mobile devices, this article is worth reading.
It's a pity Luca Barbieri or any Mesa / Gallium3D developers are not at Oktoberfest as they are deserving of more than a few Maß of Augustiner. In fact, today a new Gallium3D state tracker was pushed into Mesa and it's perhaps the most interesting state tracker for this open-source graphics driver architecture yet. It's a state tracker that exposes Microsoft's DirectX 10/11 API on Linux! And it's already working and can be hooked into Wine!
Earlier this month we started once again our annual Linux Graphics Survey in which we poll our readers about their choices and opinions concerning graphics cards, display drivers, and other graphics / X.Org related features of the Linux desktop. While this survey is still going on through the end of September -- so you still have time to participate -- here are the results from the first 6,300 people to submit their responses. We are publishing the results so far since there is the X Developers' Summit this week in Toulouse and some of these findings may prove to be useful during those discussions.
Recently when benchmarking the Btrfs and EXT4 file-systems we were left surprised that the performance of the next-generation Btrfs file-system had regressed against EXT4 to the point where the evolutionary file-system is measurably faster in a greater number of disk benchmarks. In fact, even with solid-state drives and Btrfs offering an SSD optimized mode, it still conceded to EXT4. It turns out that in the Linux 2.6.35 kernel, Btrfs regressed. This regression should have been fixed with the Linux 2.6.36 kernel, but recently when benchmarking EXT4/Btrfs against ZFS-FUSE on a 2.6.36 development snapshot we found its performance to still be poor for Btrfs compared to EXT4. To confirm where these two most prominent Linux file-systems are at right now, we have new EXT4 and Btrfs performance results from the Linux 2.6.34, 2.6.35, and 2.6.36-rc3 kernels.
The Q3'2010 update to the Phoronix Test Suite introduces new test profiles, provides new analytics capabilities, supports testing under a more diverse selection of hardware and software, and provides numerous other features for those looking to deploy this leading automated testing platform within enterprise environments.
Last week we reported that a native ZFS implementation for Linux is soon being released that is based upon the work by Lawrence Livermore National Laboratory to bring Sun's ZFS file-system to Linux as a CDDL-licensed kernel module. As said though in that article, there is already a ZFS module for FUSE (File-system in User-space) that is already available and with it not living in the GPL-land of the Linux kernel, it is legally allowed, but it does not come without some performance overhead. Over the weekend though there's been some discussions in the related forum thread and elsewhere about the dependability of ZFS-FUSE and what the level of impact on using FUSE really amounts to in real-world usage. We have tested the ZFS-FUSE -- both the latest stable and Git snapshots -- and have compared this alternate ZFS Linux implementation to that of the native EXT4 and Btrfs.
Earlier this month we delivered benchmarks comparing the ZFS, EXT4, and Btrfs file-systems from both solid-state drives and hard drives. The EXT4 file-system was the clear winner in terms of the overall disk performance while Btrfs came in second followed by Sun's ZFS in FreeBSD 8.2. It was a surprise that in our most recent testing the EXT4 file-system turned around and did better than the next-generation Btrfs file-system, but it turns out that Btrfs regressed hard in Linux 2.6.35 as to be found in Ubuntu 10.10 and other soon-to-be-released distributions. However, regardless of where Btrfs is performing, its speed can be boosted by enabling its transparent zlib compression support.
Prior to the emergence of Btrfs as a viable next-generation Linux file-system, Sun's ZFS file-system was sought after for Linux due to its advanced feature-set and capabilities compared to EXT3 and other open-source file-systems at the time. While ZFS support has worked its way into OpenSolaris, FreeBSD, NetBSD, and other operating systems, ZFS had not been ported to Linux as its source-code is distributed under the CDDL license, which is incompatible with the GNU GPL barring it from integration into the mainline Linux kernel. Next month, however, a working ZFS module for the Linux kernel without a dependence on FUSE will be publicly released.
While we benchmark the latest Linux kernel code on a daily basis at kernel-tracker.phoromatic.com using our automated testing platform built on the Phoronix Test Suite, now that the Linux 2.6.35 kernel was released, we have run a formalized set of kernel benchmarks on a ThinkPad W510 notebook with an Intel Core i7 CPU to see how the Linux 64-bit kernel is running with this high-end notebook under the Linux 2.6.32, 2.6.33, 2.6.34, and 2.6.35 releases.
ZFS is often looked upon as an advanced, superior file-system and one of the strong points of the Solaris/OpenSolaris platform while most feel that only recently has Linux been able to catch-up on the file-system front with EXT4 and the still-experimental Btrfs. ZFS is copy-on-write, self-healing with 256-bit checksums, supports compression, online pool growth, scales much better than the UFS file-system commonly used on BSD operating systems, supports snapshots, supports deduplication, and the list goes on for the features of this file-system developed by Sun Microsystems. In this article we are seeing how well the performance of the ZFS file-system under PC-BSD/FreeBSD 8.1 stacks up to UFS (including UFS+J and UFS+S) and on the Linux side with EXT4 and Btrfs.
As was mentioned in last Friday's article, Which Is Faster: Debian Linux or FreeBSD, tests of FreeBSD atop the ZFS file-system (rather than UFS2+S) are currently underway and those results are expected to be published in full later this week as the ZFS disk performance is compared directly to UFS2+S, UFS2+J, and also Ubuntu Linux with the EXT4 and Btrfs file-systems. Today though we have a few ZFS performance numbers to share as we look at the performance of the new CAM-ATA sub-system on FreeBSD.
In previous articles I have hinted that at Phoronix we are working to take advantage of the Btrfs file-system within the Phoronix Test Suite and Phoromatic to provide an interesting feature that will further expand our automated testing capabilities, but how does this file-system come into play? Well, here is what's being worked on and it should be of terrific value to many people.
Three years ago Intel had released PowerTop, an open-source utility for Linux that would analyze how well your laptop was conserving power and would allow users to easily tune their system for maximum battery life via simple power optimizations. By simply running this utility, some users were able to significantly extend their battery life. However, is this utility still useful and needed with a modern Linux desktop? The most recent release of PowerTop (v1.11) was a year and a half ago, so we are seeing how well PowerTop is still able to reduce the power consumption of Intel notebooks/netbooks running Linux.
Yesterday we reported that Ubuntu 10.10 gained Btrfs installation support and since then we have been trying out this Btrfs support in Ubuntu "Maverick Meerkat" and have a fresh set of Btrfs benchmarks to serve up.
Last month we looked at the cost of running Compiz by means of looking at how the window manager affected the frame-rate of several different games and whether compositing was used. We also tested out several different drivers and pieces of hardware. When Compiz was running rather than GNOME's Metacity it often caused a measurable drop in the OpenGL performance and then we later found this to be the case too with KDE's KWin. Today we are seeing if and how using Mutter, the window manager for the GNOME 3.0 desktop that uses Clutter-based compositing, will affect the performance of several different open-source games.
With MeeGo using Btrfs by default, Canonical making plans for Btrfs in as soon as Ubuntu 10.10, and Novell now pushing Btrfs in openSUSE, among other milestones for this advanced Linux file-system, we decided to see where the Btrfs performance is now at with the Linux 2.6.35 kernel that's currently in development. We compare the Btrfs performance to EXT4 and see how some of the different mount options are affecting the file-system's performance in different benchmarks.
For the past six months we have been monitoring the performance of the very latest Linux kernel code on a daily basis across multiple systems. We have spotted a few regressions -- both positive and negative -- on occasion using our automated daily testing of the Linux kernel, but nothing like what we have encountered the past few days: the Linux 2.6.35 kernel performance has fallen hard. In fact, the performance has fallen very hard in a number of tests and right now, we would consider it a disaster. While the 2.6.35 code has not even seen its first release candidate yet, there are some massive performance drops in a variety of different tests that have yet to be corrected and nothing like we have encountered with previous kernel release cycles especially for a regression that has lived now for about one week.
With its continuing commitment to provide quarterly updates to its industry-leading automated testing and benchmarking platform, Phoronix Media has announced the immediate availability of Phoronix Test Suite 2.6 (codenamed "Lyngen").
Last week we published Arch Linux vs. Ubuntu benchmarks to finally lay to rest that for the overall system performance the speed of the rolling Arch Linux distribution is not too different from that of Ubuntu when running with similar package versions. One of the areas, however, where the performance was different with the "out of the box" experience was the OpenGL gaming where Ubuntu was using Compiz by default where as Arch had Metacity. This surprised many so we published another article entitled The Cost Of Running Compiz where we showed the performance penalties of a compositing window manager with different hardware and drivers. This led some to ask whether the performance of KWin also causes the OpenGL frame-rate to drop, so here are those KDE benchmarks.
Earlier this week we published benchmarks comparing Arch Linux and Ubuntu. There were only a few areas where the two Linux distributions actually performed differently with many of their core packages being similar, but one of the areas where the results were vastly different was with the OpenGL performance as Ubuntu uses Compiz by default (when a supported GPU driver is detected) where as Arch does not. This had surprised many within our forums so we decided to carry out a number of tests with different hardware and drivers to show off what the real performance cost is of running Compiz as a desktop compositing manager in different configurations.
The Kernel-based Virtual Machine (KVM) has been in the mainline Linux kernel since Linux 2.6.20 in early 2007 and over time it has become one of the most widely used virtualization platforms on Linux. Ubuntu uses KVM, Fedora uses KVM, and Red Hat Enterprise Linux has even switched from Xen to preferring KVM, among others. While occasionally we deliver new KVM virtualization benchmarks, we decided to investigate how the performance of KVM virtualization has changed -- if at all -- over the past two years for better or worse.
As we reported this morning, via a blog post and keynote to kick-off the start of the Ubuntu Developer Summit this week for engaging in Ubuntu 10.10 development activities, Mark Shuttleworth announced Ubuntu Light and Ubuntu Unity. Ubuntu Light is a new spin of Ubuntu that is being offered up to OEMs that are looking to offer Ubuntu Linux as part of a dual-boot installation on their PCs. Unity is the new Ubuntu desktop interface that is used by Ubuntu Light.
Last week in a FreeBSD status report we talked about the Chromium web-browser support on FreeBSD improving through a new subscription program whereby most of the FreeBSD patches are being kept closed-source for some length of time before being committed back upstream as open-source and reaching the hands of the non-subscribers. This caused some to question the work, but the developer behind this FreeBSD-Chromium subscription program, Sprewell, has written an editorial that we are now publishing. This details his beliefs concerning the future of open-source software business models.
With the recent release of GCC 4.5 and the forthcoming release of LLVM 2.7 that is expected in the coming days, we have decided to see how the performance of GCC compares to that of LLVM. For this testing we have results from GCC 4.3/4.4/4.5 compared to LLVM with its GCC front-end (LLVM-GCC) and against Clang, which is the native C/C++ compiler front-end for the Low-Level Virtual Machine.
530 software articles published on Phoronix.