There is no shortage of EXT4 benchmarks from comparing this evolutionary file-system's performance on netbooks to how it battles the Btrfs file-system to its performance recession. We have even benchmarked it on USB flash drives and on high-end SSDs. We have also delivered numerous Btrfs benchmarks. In this article though we are finally delivering something that has long been requested and that is Reiser4 file-system benchmarks running directly against EXT4 and Btrfs. We have also thrown in the original ReiserFS file-system for comparison too.
At Phoronix we have been benchmarking the Linux kernel on a daily basis using Phoromatic Tracker, a sub-component of Phoromatic and the Phoronix Test Suite. We launched our first system in the Linux kernel testing farm just prior to the Linux 2.6.33 kernel development cycle and found a number of notable regressions during the past three months. Now with the Linux 2.6.34 kernel development cycle getting into swing, we have added an additional two systems to our daily kernel benchmarking farm. One of the systems is an Atom Z520 system but what makes it more interesting is that the system is using a Btrfs file-system and then the second new system added to the kernel tracker is a 64-bit setup. However, to provide a historical look at the Linux kernel performance, we have ran some fresh benchmarks going back to the Linux 2.6.24 kernel and ending with the recently released Linux 2.6.33 kernel.
In December we wrote that Ubuntu 10.04 already shortened the boot time, which has been a great focus amongst Canonical and Ubuntu developers as they strive for a ten second boot. A lot has changed since that article was published last year, including the introduction of Plymouth and many kernel mode-setting improvements along with the introduction of Nouveau for NVIDIA KMS support. We've ran a new boot performance comparison on two laptops and a netbook as we see how the boot times are looking with Ubuntu 10.04 LTS when compared to Ubuntu 9.10. We have also looked at how the power consumption has changed in the Lucid Lynx for these mobile devices.
This weekend at the Southern California Linux Expo in Los Angeles, Matthew Tippett and I presented a talk entitled Five Stages of Benchmark Loss: PTS and You. In this hour-long talk, we covered Linux benchmarking, what has been learned over the years of benchmarking at Phoronix, the Phoronix Test Suite, and the five stages that users and developers generally go through when they lose out on benchmarking results. For those that were unable to attend this event, here are the slides and recordings.
Last month we published benchmarks of EXT4 comparing this file-system's performance when it was first marked stable in the mainline kernel and then where it is at now in the Linux kernel while testing every major release in between. This article was followed up by a Btrfs versus EXT4 comparison using the Linux 2.6.33 kernel to see how the two most talked about Linux file-systems are battling it out with the latest kernel. After those Linux file-system benchmarks were published, we received a request from Canonical to look at the EXT3 performance too. With that said, we have done just that and have published EXT3, EXT4, and Btrfs benchmarks from Ubuntu 9.10 and a Ubuntu 10.04 development snapshot from an Intel Atom netbook.
Phoronix Media has announced the immediate release of Phoronix Test Suite 2.4 (codenamed "Lenvik"), as the latest update to their open-source testing framework that delivers immediate and measurable advantages to its customers. The Phoronix Test Suite 2.4 software is compatible with a greater number of operating systems, introduces support for mobile platforms, offers a new range of test profiles, and other features to further solidify its premiere position within the computer benchmarking industry.
Earlier this week we published extensive benchmarks of EXT4 that looked at the performance of this Linux file-system under every major kernel release since it was declared stable in the Linux 2.6.28 release. EXT4 has encountered many significant performance losses over time as its developers batten up the data security, but there have been some improvements too. At the same time though the developers working on the still-experimental Btrfs file-system continue to move along and push forward changes with each kernel cycle. Just last month we delivered Btrfs comparative benchmarks using the Linux 2.6.32 kernel, but already out of our own personal interest and requests from readers, we have new tests atop the latest Linux 2.6.33 kernel.
Over the past week there has been a lot of talk about the EXT4 file-system following the announcement that Google is migrating their EXT2 file-systems to EXT4. Their reasons for this transition to EXT4 are attributed to the easy migration process and Google engineers are pleased with this file-system's performance. However, as we mentioned in that news post last week and in many other articles over the past weeks and months, EXT4 is not as great of a contender as it was in the past, well, for some tests at least. The performance of the EXT4 file-system commonly goes down with new kernel releases and not up, as kernel developers continue to introduce new safeguards to address potential data loss problems that initially plagued some EXT4 users. For our latest EXT4 benchmarks we have numbers that show this file-system's performance using a vanilla 2.6.28 kernel (when EXT4 was marked as stable) and then every major kernel release up through the latest Linux 2.6.33 release candidate.
For those that may have forgot, at the start of December we launched the Phoronix Kernel Test Farm to begin benchmarking the Linux kernel on a daily basis using the automated tools that we provide via the Phoronix Test Suite and Phoromatic. Towards the middle of December we then unveiled the Phoromatic Tracker, which exposes these test results in real-time to the public. Well, it's now been a month of monitoring the kernel's performance and the entire Linux 2.6.33 kernel development cycle thus far, with many interesting findings.
With 2009 having come to an end, here are the twelve most popular articles and news stories that were published on Phoronix in 2009. Over 230 original articles and 880 news stories were published on Phoronix.com in 2009.
Last month Phoromatic went into public beta, which is our remote test management software for the Phoronix Test Suite that allows a wealth of possibilities including the ability to easily build a benchmarking test farm. At the start of this month, we in fact announced that the Phoronix Kernel Test Farm went live and it would be benchmarking the latest mainline Linux kernel on a daily basis. This was followed by the addition of a system in our test farm to benchmark the latest Fedora Rawhide packages on a daily basis. We had not intended to begin pushing out these results publicly through a new web-site until next year, but we have already collected some interesting metrics that are documenting active regressions within the Linux 2.6.33 kernel. As a result, this morning we are rolling out Phoromatic Tracker, the public interface to our test farm.
We have published articles containing EXT4 benchmarks many times now going back to our original real world benchmarks of EXT4 to when Ubuntu 9.04 received EXT4 support and when we ran a variety of file-system benchmarks on an Intel X25-E SSD. We had also thrown in EXT4 numbers when benchmarking Btrfs (and again with Btrfs 0.19) along with NILFS2 benchmarks. Each time has been with a different kernel and the performance of the different Linux file-systems continue to change as each file-system matures and picks up different features. Though with the Linux 2.6.32 kernel the EXT4 performance had changed a great deal due to a change that provides better data integrity on writes but at a significant performance cost. To see how this changes the Linux file-system landscape, atop the latest Linux kernel we have a fresh set of benchmarks for EXT3, EXT4, XFS, ReiserFS, and Btrfs.
Only two weeks have passed since the launch of Phoronix Test Suite 2.2, but it is already time to push out the first alpha release for Phoronix Test Suite 2.4! Phoronix Test Suite 2.4 (codenamed "Lenvik") Alpha 1 is what is running on our kernel test farm that launched yesterday and there is already a great deal of changes to be found in this release. In fact, the patch already between 2.2 Bardu and 2.4 Lenvik Alpha 1 is nearly 24,000 lines of code! There is some code reorganization, but a lot of new code has been introduced. Here is some of what can be found already in this development release.
This year with the Phoronix Test Suite we have delivered four major updates to this leading, widely adopted, multi-platform testing software that has brought dozens of new test profiles and literally hundreds of significant changes. These changes ranged from features to autonomously track performance regressions within any code-base, the ability to not only compare frame-rates within OpenGL tests but image quality comparisons too, support for mobile platforms, and so much other major work to further drive automated testing and benchmarking not only on Linux but OpenSolaris, *BSD, and Mac OS X too. In 2009 we also launched PTS Desktop Live, our own operating system for carrying out standardized benchmarks in an easy-to-use and repeatable manner from a live Linux environment, and also Phoromatic, which is designed for the enterprise world and allows the Phoronix Test Suite to be easily deployed across many systems and then managed from a central interface. The year is not over yet, nor is our work on ensuring that the Phoronix Test Suite is the most powerful and robust testing/benchmarking platform. With that said, as of this morning our Phoronix kernel test farm is now alive!
With the Linux 2.6.32 kernel being released in a few days, we found it time to benchmark this newest kernel release that brings new drivers, kernel mode-setting improvements, virtualization enhancements, and more.
Yesterday marked the release of Phoronix Test Suite 2.2 and it was the best version yet with the addition of many new exciting and useful features. While this release was gratifying, there are much greater plans for the Phoronix Test Suite going into the next decade. It has already been shared that Windows support is coming, but there are other huge features coming too as soon as Q1'2010. Up to this point, most of the tests and the design of pts-core (the Phoronix Test Suite engine) have been focused on quantitative benchmarks with many of the tests spitting out a frame-rate, time, or some other measurement. However, now being supported in the Phoronix Test Suite is the ability to produce abstract results, such as screenshots used for image quality comparisons. The Phoronix Test Suite can now track the image quality of various test profiles (such as OpenGL games) across hardware configurations, drivers, and more. All of this is still leveraged upon the existing Phoronix Test Suite framework and our design philosophies so that even image quality comparisons can be carried out autonomously, the ability to compare many results side-by-side, support for carrying out these tests remotely via Phoromatic, and the ability to share your abstract results with others via Phoronix Global. Now not only can you be sure you are satisfied with the quantitative frame-rate of the hardware you have -- or are about to purchase -- but you have a plethora of options for looking at the qualitative performance too.
Continuing in the tradition of providing feature-rich, quarterly updates to the Phoronix Test Suite, Phoronix Media has announced the immediate availability of Phoronix Test Suite 2.2 (codenamed "Bardu"). Phoronix Test Suite 2.2 continues to expand the capabilities and feature set for this open-source testing framework with the introduction of many new features, a new graphical user-interface, numerous new test profiles and suites, and a public beta of the Phoromatic remote test management system.
There's the Phoronix Test Suite, Phoronix Global, and PTS Desktop Live as our family of free software products to provide extensive benchmarking and automated testing capabilities atop Linux, OpenSolaris, BSD, and Mac OS X operating systems. The Phoronix Test Suite has more than 120 test profiles and 50 test suites with new suites and tests continuing to come in through its extensible architecture, but for organizations with multiple test systems or entire testing farm(s) devoted to performance monitoring and regression tracking, they have had to strap the Phoronix Test Suite atop their own management systems or hack away at simple scripts to deploy our testing software across an array of systems. Today though we are announcing the public beta launch of Phoromatic. Phoromatic is a remote test management system that allows controlling any number of PTS-powered systems through a single web-based interface, which also allows all of the test results to be viewed from a central source. This article provides a first-look and guide for some of the possibilities of Phoromatic from those looking to build a benchmarking test farm or for individuals simply wanting to benchmark computers across the world. We also share in this article what may be coming next to our Linux-focused benchmarking empire.
After a three month development period following the release of Phoronix Test Suite 2.0, the first beta release of Phoronix Test Suite 2.2 "Bardu" is now available for all of your testing needs on Linux, Mac OS X, OpenSolaris, and BSD platforms. Phoronix Test Suite 2.2 Beta 1 carries more than 200 changes since the release of 2.0 Sandtorg with many new prominent features being introduced, new test profiles added, and greater usability enhancements. In this article, we will go over some of the key improvements to be found in Phoronix Test Suite 2.2.
Last weekend a few Phoronix benchmarks were underway of the Linux 2.6.32-rc5 kernel when a very significant performance regression was spotted. This regression caused the PostgreSQL server to run at about 18% of the performance found in earlier kernel releases. Long story short, in tracking down this performance regression we have finally devised a way to autonomously locate performance regressions within the Linux kernel and potentially any Git-based project for that matter. Here are a few details.
Earlier this month at the Red Hat Summit where Red Hat Enterprise Linux 5.4 was released with support for the Kernel-based Virtual Machine. At this Red Hat event, virtualization -- particularly KVM -- and cloud computing were the most talked about topics. But how is KVM performing these days? With new virtualization refinements going into almost every new Linux kernel release, we have published a new set of KVM benchmarks using the Linux 2.6.31 kernel, to provide updated numbers against our KVM benchmarks from last year and our Core i7 virtualization numbers. This time around, we are also using a Phenom II processor for testing out the AMD-V technology.
Last December we had looked at the Ubuntu 9.04 home encryption performance after this feature appeared in a development snapshot as an alternative to those looking for security some of their data but are not looking for completely encrypting the hard drive due to the performance impact or other reasons. The home encryption feature ended up being disabled in Ubuntu 9.04 unless a special boot parameter was used, but it has now reappeared in Ubuntu 9.10.
A few weeks back Con Kolivas returned to the Linux kernel scene after parting ways with kernel development for two years. Con, who has contributed a great deal to the Linux kernel in the past particularly with CPU schedulers, returned and introduced BFS. BFS (not to be confused with the file-system of the same name) is a new scheduler for the Linux kernel that's designed for optimal performance on hardware that's more common with a majority of Linux desktop users, not massive data centers running dozens (and in some cases, hundreds) of CPUs. The BFS scheduler is designed to offer "extremely low latencies for excellent interactivity", according to Con Kolivas. In this article we have a set of benchmarks comparing BFS and the current default scheduler within the Linux kernel, the Completely Fair Scheduler.
Last Friday we published Mac OS X 10.6 benchmarks and then on Monday they were joined by Ubuntu 9.10 vs. Mac OS X 10.6 benchmarks. One of the requests that has come up since publishing those articles are to carry out a set of tests comparing the performance of LLVM and LLVM-GCC. With Apple's Snow Leopard release, some parts of the operating system were built using LLVM-GCC for optimized performance, although this compiler is not yet matured. In this article we have a set of 12 benchmarks comparing GCC to LLVM-GCC.
Beyond pushing out a new graphical boot screen just before the feature freeze went into effect for Ubuntu 9.10, Canonical released the first public version of their own app store, previously codenamed AppCenter, but now known as the Ubuntu Software Store (or software-store as its package is called). Canonical does have some grand plans for the Ubuntu Software Store and in this article we have some screenshots of what it looks like currently and how it functions along with some of their plans for the future.
When benchmarking development releases of Fedora in particular, they often end up being much slower than the final build and perform lower when compared against some of the other leading desktop distributions. As we have mentioned in previous articles, this is generally due to the debugging support enabled within the development builds of Fedora. To see just what the performance cost is, we have compared the Fedora 11 performance of the normal kernel against the kernel-debug package. Additionally, we also compared the performance when disabling SELinux and system auditing support.
Have you considered embracing Linux in any of your articles? If you have, but have not acted on such thoughts, why is that? Is it the Linux learning curve? The "lack of benchmarks"? Simply the lack of resources on the part of your editors and product evaluators? After speaking with several editors from fellow publications, these seem to be most of the excuses at hand. However, at Phoronix Media, we have the solutions to these problems -- and they are free and easy to adopt. I would invite you to think how using Linux to complement your Windows testing in hardware reviews could benefit your publication by providing more thorough reviews to fulfill the needs of more readers, wooing more hardware companies with having another feature to offer, and generate additional page views from having more content.
Phoronix Media has released version 2.0 (codenamed "Sandtorg") of the Phoronix Test Suite, which encompasses hundreds of updates to its flagship testing and performance profiling software. Phoronix Media has also released PTS Desktop Live 2009.3 (codenamed "Gernlinden"), which is the first Linux-based desktop operating system designed exclusively for carrying out automated tests using the Phoronix Test Suite from a live environment. Phoromatic, a web-based remote test management system, has also entered closed beta testing.
In less than two weeks we will be officially releasing Phoronix Test Suite 2.0 "Sandtorg" and this by far is the biggest upgrade ever to our flagship testing and benchmarking software. While the Phoronix Test Suite is most often associated with Linux, this open-source software is also compatible with Mac OS X, OpenSolaris, and BSD operating systems too, all of which offer new improvements with Phoronix Test Suite 2.0. In this article we have detailed some of the major highlights of Phoronix Test Suite 2.0 and how we seek to drive innovation into PC benchmarking and performance profiling.
Since we began benchmarking Btrfs a few months ago we have found it to not deliver any spectacular file-system performance results on Linux. This next-generation Linux file-system that has often been compared to Sun's ZFS has not really performed that well, granted it's still very much under development. Btrfs is far from being the performance king and even its SSD mode has had little positive effect. Just weeks ago we delivered EXT4, Btrfs, and NILFS2 benchmarks, but now there is a new release of Btrfs available. Committed to the Linux 2.6.31 kernel was Btrfs v0.19. Does this release bring any performance improvements? Yes and no.
521 software articles published on Phoronix.