1. Computers
  2. Display Drivers
  3. Graphics Cards
  4. Memory
  5. Motherboards
  6. Processors
  7. Software
  8. Storage
  9. Operating Systems


Facebook RSS Twitter Twitter Google Plus


Phoronix Test Suite

OpenBenchmarking.org

Autonomously Finding Performance Regressions In The Linux Kernel

Michael Larabel

Published on 22 October 2009
Written by Michael Larabel
Page 1 of 2 - 44 Comments

Last weekend a few Phoronix benchmarks were underway of the Linux 2.6.32-rc5 kernel when a very significant performance regression was spotted. This regression caused the PostgreSQL server to run at about 18% of the performance found in earlier kernel releases. Long story short, in tracking down this performance regression we have finally devised a way to autonomously locate performance regressions within the Linux kernel and potentially any Git-based project for that matter. Here are a few details.

For some background information on the test that led to this work, the PostgreSQL performance under the Linux 2.6.32-rc1 through the Linux 2.6.32-rc5 (and still in Git master) is in shambles when compared to earlier kernels (tested Linux 2.6.30 and 2.6.31 for comparison). This regression was found using the same kernel configuration with the default options on a clean Ubuntu 9.10 installation. To ensure this wasn't an isolated incident, first this regression was confirmed on an entirely different system -- a dual AMD Opteron system compared to an Intel Atom "nettop" system.

Normally these benchmarks would end up being published on Phoronix as-is and left up to other interested users or developers to track down the problem due to our limited resources and most of this work being done by a lone individual. With the Phoronix Test Suite maturing though, a realization was made. For months it has been a goal to essentially build a Phoronix Test Suite module that would layer our performance profiling / benchmarking software directly atop Git and its git-bisect command. With the Phoronix Test Suite 2.2 "Bardu" work -- and the motivation to track down this significant drop in disk performance -- this was now quite possible with just a few hundred lines of code to create the module that ties the components together.

To cut to the chase, to be introduced with Phoronix Test Suite 2.2 Beta 1 next week will be this module that leverages the Phoronix Test Suite and Git together to create one powerful testing pair. The bisect module in the Phoronix Test Suite makes it possible to track down the triggering chance for any type of regression that can be tested by the Phoronix Test Suite. With the extensible Phoronix Test Suite architecture, you can write a test profile for just about any part of the system, if the included 110+ tests and 40+ test suites do not cover a problem that's encountered.

All that this PTS module needs to know is the starting and end points that return a different state, an accessible repository (Git, currently) to be tested, a Phoronix Test Suit result file that illustrates a performance regression, and an external build script to be written that automatically builds and installs the said code. With that information, the Phoronix Test Suite and the bisect module are able to autonomously locate the performance regression. Right now only Git is supported, but we are working on support for other source code management systems as well. There's even an experimental version of our bisect module that even automates the reboot process and recovery of the Phoronix Test Suite upon reboot.

<< Previous Page
1
Latest Linux Hardware Reviews
  1. 13-Way Low-End GPU Comparison With AMD's AM1 Athlon
  2. ASUS AM1I-A: A Mini-ITX Board For Socketed Kabini APUs
  3. Mini-Box M350: A Simple, Affordable Mini-ITX Case
  4. Overclocking The AMD AM1 Athlon & Sempron APUs
Latest Linux Articles
  1. Ubuntu 12.04.4 vs. 13.10 vs. 14.04 LTS Desktop Benchmarks
  2. AMD OpenCL Performance With AM1 Kabini APUs
  3. A Quick Look At GCC 4.9 vs. LLVM Clang 3.5
  4. Are AMD Athlon/Sempron APUs Fast Enough For Steam On Linux?
Latest Linux News
  1. Mesa 10.0 & 10.1 Stable Get Updated
  2. Getting Hit By The Variable Performance Of The Public Cloud
  3. Git 2.0 Test Releases Begin With Many Changes
  4. Wine 1.7.17 Works On Its Task Scheduler, C Run-Time
  5. The Improv ARM Board Still Isn't Shipping; Riding A Dead Horse?
  6. Debian To Maintain 6.0 Squeeze As An LTS Release
  7. Wasteland 2 Is Finally Released For Linux Gamers
  8. FreeBSD Advances For ARM, Bhyve, Clang
  9. Ubuntu 14.04 LTS "Trusty Tahr" Officially Released
  10. Ubuntu 12.04 LTS vs. 14.04 LTS Server Benchmarks
  11. QEMU 2.0 Released With ARM, x86 Enhancements
  12. Running The Unity 8 Preview Session On Ubuntu 14.04 LTS
Latest Forum Discussions
  1. Suggestions about how to make a Radeon HD 7790 work decently?
  2. The GNOME Foundation Is Running Short On Money
  3. Updated and Optimized Ubuntu Free Graphics Drivers
  4. Radeon 8000M problematic on Linux?
  5. Linux Kernel Developers Fed Up With Ridiculous Bugs In Systemd
  6. After Jack Keane, RuseSoft will briing Ankh 3 to Linux through Desura
  7. Suspected PHP Proxy Issue
  8. Change installation destination from home directory