1. Computers
  2. Display Drivers
  3. Graphics Cards
  4. Memory
  5. Motherboards
  6. Processors
  7. Software
  8. Storage
  9. Operating Systems


Facebook RSS Twitter Twitter Google Plus


Phoronix Test Suite

OpenBenchmarking.org

Autonomously Finding Performance Regressions In The Linux Kernel

Michael Larabel

Published on 22 October 2009
Written by Michael Larabel
Page 2 of 2 - 44 Comments

This bisect module starts by analyzing the result file and comparing the selected test(s) that illustrate a significant performance drop (or gain). Once that has been done and the good/bad revisions tested on the running test system, git-bisect is automatically called and handled by this Phoronix Test Suite module. With each bisect, the test with any specific test arguments is called and this module then determines whether that revision resulted in a "good" or "bad" run. The bisect module continues until the commit causing the problem is found. Now and into the future we will also work to expand these capabilities and provide more features during the bisecting process.

While this module was written to locate a Linux kernel performance regression, it can be used with any Git-based package that is suitable for testing by the Phoronix Test Suite (read: we can just as easily track down performance regressions in the X Server, Mesa, and drivers). Of course, an external build/install script needs to be written for setting up the environment, but this module should be very extensible. In Phoronix Test Suite 2.2 and future releases, the capabilities of the bisect module and our analytical options within this GPLv3 code should expand. Of course, this is for finding a performance regression from the past. For properly monitoring the performance of system code as its developed, one could also run Phoromatic, which will have the capability of automatically calling test runs after a Git commit is made and to then report the results back to the central repository in real-time rather than debugging regressions well after the fact.

Did the Phoronix Test Suite end up being successful in tracking down this PostgreSQL performance regression in the Linux kernel? Oh yes it did. The very significant drop in PostgreSQL's performance in the Linux 2.6.32 kernel with default options can be attributed to this lone Git commit that is for a fix to address cache flushing in ext4_sync_file for the EXT4 file-system. This commit improves data integrity in the event of a power loss or other problem, but carries a high disk performance penalty. After the Phoronix Test Suite module reported that this was the faulty commit, it was manually confirmed too. When asked about the commit, Red Hat's Eric Sandeen shared:

"Hey, thanks for doing the digging :)

It is required for safe behavior with volatile write caches on drives.

You could mount with -o nobarrier and it would go away, but a sequence like write->fsync->lose power->reboot may well find your file without the data that you synced, if the drive had write caches enabled.

If you know you have no write cache, or that it is safely battery backed, then you can mount with -o nobarrier, and not incur this penalty.

-Eric"

Many kernels were built in the process of tracking down this PostgreSQL performance drop triggered by this default EXT4 change, but fortunately through the Phoronix Test Suite it required very little user intervention once the bisect module was running thanks to a combination of PTS Bardu and Git. Rather than having to use git-bisect manually and building out the software each time, the Phoronix Test Suite can cover it and just tell you if/where a performance regression occurs all while you are sitting back and watching. Look for this capability in Phoronix Test Suite 2.2 "Bardu" along with major GTK user-interface improvements, verification of statistical significance in benchmark results, and many other new features. You can expect to read more on these automated revision-traversing testing capabilities at Phoronix as its features are brought forward and extended to drive a new level of benchmarking in open-source software.

About The Author
Michael Larabel is the principal author of Phoronix.com and founded the web-site in 2004 with a focus on enriching the Linux hardware experience and being the largest web-site devoted to Linux hardware reviews, particularly for products relevant to Linux gamers and enthusiasts but also commonly reviewing servers/workstations and embedded Linux devices. Michael has written more than 10,000 articles covering the state of Linux hardware support, Linux performance, graphics hardware drivers, and other topics. Michael is also the lead developer of the Phoronix Test Suite, Phoromatic, and OpenBenchmarking.org automated testing software. He can be followed via and or contacted via .
Latest Linux Hardware Reviews
  1. 13-Way Low-End GPU Comparison With AMD's AM1 Athlon
  2. ASUS AM1I-A: A Mini-ITX Board For Socketed Kabini APUs
  3. Mini-Box M350: A Simple, Affordable Mini-ITX Case
  4. Overclocking The AMD AM1 Athlon & Sempron APUs
Latest Linux Articles
  1. How Much Video RAM Is Needed For Catalyst R3 Graphics?
  2. Ubuntu 12.04 LTS vs. 14.04 LTS Cloud Benchmarks
  3. Ubuntu 12.04.4 vs. 13.10 vs. 14.04 LTS Desktop Benchmarks
  4. AMD OpenCL Performance With AM1 Kabini APUs
Latest Linux News
  1. Linux 3.15 Lands Some DRM Graphics Driver Fixes
  2. AMD Is Disabling DPM Support For RV770 GPUs
  3. ReactOS Working On A Community Windows OS
  4. Borderlands Is Being Considered For Linux
  5. Mesa 10.0 & 10.1 Stable Get Updated
  6. Getting Hit By The Variable Performance Of The Public Cloud
  7. Git 2.0 Test Releases Begin With Many Changes
  8. Wine 1.7.17 Works On Its Task Scheduler, C Run-Time
  9. The Improv ARM Board Still Isn't Shipping; Riding A Dead Horse?
  10. Debian To Maintain 6.0 Squeeze As An LTS Release
  11. Wasteland 2 Is Finally Released For Linux Gamers
  12. FreeBSD Advances For ARM, Bhyve, Clang
Latest Forum Discussions
  1. The GNOME Foundation Is Running Short On Money
  2. Updated and Optimized Ubuntu Free Graphics Drivers
  3. Catalyst 14.3 Beta
  4. Suggestions about how to make a Radeon HD 7790 work decently?
  5. Radeon 8000M problematic on Linux?
  6. Linux Kernel Developers Fed Up With Ridiculous Bugs In Systemd
  7. After Jack Keane, RuseSoft will briing Ankh 3 to Linux through Desura
  8. Suspected PHP Proxy Issue