Btrfs Did Regress Hard In The Linux 2.6.35 Kernel

This morning we published benchmarks of ZFS, EXT4, and Btrfs when running these three popular file-systems off a high-performance OCZ Vertex 2 solid-state drive and also a 7200RPM notebook hard drive. To a fair amount of surprise, the EXT4 file-system ended up beating out Btrfs on the SSD in a number of tests, which was a change from our tests when using the EXT4/Btrfs modules on previous kernels. It was even more surprising since Btrfs has an SSD-optimized mode where as EXT4 does not. However, it turns out this is part of a major performance regression for this file-system in the Linux 2.6.35 kernel.

Over the weekend, another Btrfs tester reported in on regressions that he found where the affect of using the Linux 2.6.35 kernel with this file-system resulted in as much as a 10x performance hit under write-heavy workloads. This 2.6.35 regression was tracked down to this Git commit that was described as "Shrink delayed allocation space in a synchronized manner is more controllable than flushing all delay allocated space in an async thread."

Chris Mason, the Btrfs lead developer, quickly responded that Btrfs is not being aggressive enough in allocating chunks of data, which is making the flusher come in and start data I/O. Chris is now working to reproduce and address this issue. No fix has yet been committed, but it's expected that it will land within the Linux 2.6.36 kernel.

Other users that have been trying Btrfs on Ubuntu 10.10, which adds Btrfs installation support and is using the Linux 2.6.35 kernel, have also been having some troubles. Here's an open bug on Launchpad where carrying out a Btrfs Ubuntu installation takes about two hours to install with the root Btrfs file-system where as on the same system with an EXT4 file-system it takes just about 15 minutes. We can confirm this problem too with Ubuntu 10.10 at present.

It's a pity that our daily Linux kernel performance tracker hadn't caught this regression on the day it was pulled into the Linux kernel, but the Btrfs test system running daily benchmarks of the Linux kernel happened to crash and it's not yet back online. Needless to say, we will have new Btrfs benchmarks once this regression is worked out.