1. Computers
  2. Display Drivers
  3. Graphics Cards
  4. Memory
  5. Motherboards
  6. Processors
  7. Software
  8. Storage
  9. Operating Systems


Facebook RSS Twitter Twitter Google Plus


Phoronix Test Suite

OpenBenchmarking.org

Autonomously Finding Performance Regressions In The Linux Kernel

Michael Larabel

Published on 22 October 2009
Written by Michael Larabel
Page 2 of 2 - 44 Comments

This bisect module starts by analyzing the result file and comparing the selected test(s) that illustrate a significant performance drop (or gain). Once that has been done and the good/bad revisions tested on the running test system, git-bisect is automatically called and handled by this Phoronix Test Suite module. With each bisect, the test with any specific test arguments is called and this module then determines whether that revision resulted in a "good" or "bad" run. The bisect module continues until the commit causing the problem is found. Now and into the future we will also work to expand these capabilities and provide more features during the bisecting process.

While this module was written to locate a Linux kernel performance regression, it can be used with any Git-based package that is suitable for testing by the Phoronix Test Suite (read: we can just as easily track down performance regressions in the X Server, Mesa, and drivers). Of course, an external build/install script needs to be written for setting up the environment, but this module should be very extensible. In Phoronix Test Suite 2.2 and future releases, the capabilities of the bisect module and our analytical options within this GPLv3 code should expand. Of course, this is for finding a performance regression from the past. For properly monitoring the performance of system code as its developed, one could also run Phoromatic, which will have the capability of automatically calling test runs after a Git commit is made and to then report the results back to the central repository in real-time rather than debugging regressions well after the fact.

Did the Phoronix Test Suite end up being successful in tracking down this PostgreSQL performance regression in the Linux kernel? Oh yes it did. The very significant drop in PostgreSQL's performance in the Linux 2.6.32 kernel with default options can be attributed to this lone Git commit that is for a fix to address cache flushing in ext4_sync_file for the EXT4 file-system. This commit improves data integrity in the event of a power loss or other problem, but carries a high disk performance penalty. After the Phoronix Test Suite module reported that this was the faulty commit, it was manually confirmed too. When asked about the commit, Red Hat's Eric Sandeen shared:

"Hey, thanks for doing the digging :)

It is required for safe behavior with volatile write caches on drives.

You could mount with -o nobarrier and it would go away, but a sequence like write->fsync->lose power->reboot may well find your file without the data that you synced, if the drive had write caches enabled.

If you know you have no write cache, or that it is safely battery backed, then you can mount with -o nobarrier, and not incur this penalty.

-Eric"

Many kernels were built in the process of tracking down this PostgreSQL performance drop triggered by this default EXT4 change, but fortunately through the Phoronix Test Suite it required very little user intervention once the bisect module was running thanks to a combination of PTS Bardu and Git. Rather than having to use git-bisect manually and building out the software each time, the Phoronix Test Suite can cover it and just tell you if/where a performance regression occurs all while you are sitting back and watching. Look for this capability in Phoronix Test Suite 2.2 "Bardu" along with major GTK user-interface improvements, verification of statistical significance in benchmark results, and many other new features. You can expect to read more on these automated revision-traversing testing capabilities at Phoronix as its features are brought forward and extended to drive a new level of benchmarking in open-source software.

About The Author
Michael Larabel is the principal author of Phoronix.com and founded the web-site in 2004 with a focus on enriching the Linux hardware experience and being the largest web-site devoted to Linux hardware reviews, particularly for products relevant to Linux gamers and enthusiasts but also commonly reviewing servers/workstations and embedded Linux devices. Michael has written more than 10,000 articles covering the state of Linux hardware support, Linux performance, graphics hardware drivers, and other topics. Michael is also the lead developer of the Phoronix Test Suite, Phoromatic, and OpenBenchmarking.org automated testing software. He can be followed via and or contacted via .
Latest Linux Hardware Reviews
  1. Intel Xeon E5-1680 v3 & E5-2687W v3 Compared To The Core i7 5960X On Linux
  2. Intel 120GB 530 Series SSD Linux Performance
  3. Btrfs/EXT4/XFS/F2FS RAID 0/1/5/6/10 Linux Benchmarks On Four SSDs
  4. AMD's Windows Catalyst Driver Remains Largely Faster Than Linux Drivers
Latest Linux Articles
  1. Mesa Git Yields Performance Improvements For Newer AMD GPUs
  2. Apple OS X 10.10 vs. Ubuntu 14.10 Performance
  3. Mesa 10.5-devel Brings Some Intel Haswell HD Graphics Changes Over Mesa 10.3
  4. NVIDIA vs. Nouveau Drivers With Linux 3.18 + Mesa 10.4-devel
Latest Linux News
  1. IMP Launches As Another Open-Source Computer Attempt
  2. Git 2.2.0 Released With 550+ Changes
  3. GNOME 3.15.2 Released
  4. Quantum OS Aims For A Linux Desktop With QML, Wayland & Material Design
  5. New Open-Source, Linux Benchmarks To Feast On
  6. FreeBSD Plans For The Next Ten Years
  7. Qt 5.4 Planned For Release On 9 December
  8. Meizu's Ubuntu Phone Not Expected Until Early Next Year
  9. DragonFlyBSD 4.0 Drops i386 Support, Improves Graphics
  10. Expensive "Free/Libre Software Laptop" Uses A NVIDIA GPU
Latest Forum Discussions
  1. Updated and Optimized Ubuntu Free Graphics Drivers
  2. Hurrican SDL Port
  3. Roadmap to Catalyst 14.10 ?
  4. how to configure module phoromatic ?
  5. PulseAudio 6.0 Is Coming & Other Linux Audio Plans For The Future
  6. Debian Developer Resigns From The Systemd Maintainership Team
  7. Cant get working Kaveri APU - A10-7850k
  8. Script for Fan Speed Control