File-System Benchmarks With The Linux 2.6.34 Kernel

[liangsuilong]

It is quite strange that ext4 becomes more and more slower when a new kernel goes out. I rememeber that ext4 is much faster than ext3 in kernel-2.6.2x. I can feel ext4 is much faster than ext3 in daily use, especially using computer for a long time.

[walkeer]

Thanks for nice test, but I dont really understand why are you testing FS on SSD, while >95% servers and desktops still have normal rotation HDD? Unless there is TRIM support in linux, it doesnt make much sense to use SSD in any server for heavy FS usage.

[liangsuilong]

Will Micheal test these file-system on HDD again? I am interested in it.

[Veerappan]

Thanks for nice test, but I dont really understand why are you testing FS on SSD, while >95% servers and desktops still have normal rotation HDD? Unless there is TRIM support in linux, it doesnt make much sense to use SSD in any server for heavy FS usage.

http://en.wikipedia.org/wiki/TRIM#Operating_system_support

Yes, it's wikipedia (the greatest source in the universe... </sarcasm>), but it says 2.6.33 kernels and higher support TRIM. I found a few posts going back to early last year that mentioned people were already committing patches for preliminary TRIM support.

[Michael]

Will Micheal test these file-system on HDD again? I am interested in it.

Not at this time, unless there is sufficient interest (particularly from premium subscribers).

[Jimmy]

It is quite strange that ext4 becomes more and more slower when a new kernel goes out. I rememeber that ext4 is much faster than ext3 in kernel-2.6.2x. I can feel ext4 is much faster than ext3 in daily use, especially using computer for a long time.

As I understand it, ext4 became safer and as a result became slower. Much of the safeties can be turned off with mount options to gain performance back. If your disks are battery backed you can gain the performance without much worry about your data by changing a few mount options.

Remember, Phoronix benchmark articles typically focuses on the "out of the box" experience. As such potential optimizations are omitted. It is generally assumed by distros and kernel developers that your system will crash or loose power at any moment. They also assume that you will come after them with torches and pitchforks if your data is lost. So, the safer options very often become the defaults.

[discordian]

Interesting - for the most part, BTFS performs well, if not exceptionally so. But there are just a couple of tests - the database ones - where it lags hugely behind everything else. What makes it so much worse than the others at that particular usage pattern?

I`d wager that databases were primary developed with EXT3/4 (alike) FS in mind and are spending much CPU time organising the accesses optimally for those. Afterall File IO is by far the biggest bottleneck for databases so any effort to raise performance will be in this area.
more modern FS begin to organise their data similar to databases themself, so not only is some of the effort by Postgre/mySQL wasted, it might actually hinder performance.

Just my guess. When Btrfs and other filesystems get more popular I`m sure the database software will be optimized for them.
Except for such special-purpose issues, Btrfs looks great for the general case.

[movieman]

As I understand it, ext4 became safer and as a result became slower. Much of the safeties can be turned off with mount options to gain performance back. If your disks are battery backed you can gain the performance without much worry about your data by changing a few mount options.

Ext4 still writes metadata before data, doesn't it? Battery backup won't help if it crashes between writing the metadata and writing the actual data.

[erm67]

The xfs filesystem used for the benchmark was using lazy counters or not?

It really makes a difference for metadata intensive tasks like kernel compile or high concurrency benchmarks.

the lazy counters option is relatively new and is default only since the 3.1.0 release of xfstools; it should be used when creating the xfs filesystem so the question is was the xfs filesystem created with a recent version of mkfs.xfs?
from xfs FAQ:

Work on the userspace packages has been just as busy. In mkfs.xfs the lazy superblock counter feature has now been enabled by default for the upcoming xfsprogs 3.1.0 release, which will require kernel 2.6.22 for the default mkfs invocation.

it is possible to check if it is enabled using xfs_db:

laptop / # xfs_db -r /dev/sda6
xfs_db> version
versionnum [0xb4a4+0xa] = V4,NLINK,ALIGN,DIRV2,LOGV2,EXTFLG,MOREBITS,ATTR2,L AZYSBCOUNT

[Jimmy]

Ext4 still writes metadata before data, doesn't it? Battery backup won't help if it crashes between writing the metadata and writing the actual data.

I think the idea is that power loss is the most likely "crash" type event and battery backup eliminates that. The other ways to crash (eg a hardware lockup) are rare enough that the risk seems worthwhile for the performance gain. Frequently, people brag about the long uptimes they can achieve with Linux so it would seem that having a low crash environment is possible if not likely for some. Certainly backups should be happening with a higher frequency than crashes. I can see the risk/performance trade off working for people once power loss is eliminated.

Having said that, there are lots of people (like your typical home user) who care very much about their data, don't have their systems protected by batteries, and don't have elaborate file backup systems in play. These users also find ways of making crashes more likely. For example running out and buying a GPU from either ATI or NVidia and using the speedy binary only drivers (Don't flame, either vendor will significantly increase your likelihood of crashing. Its not like I'm expressing which vendor I believe is better at it ).

So the question becomes: Will ext4 eat my valuable data if I push the crash monsters out of the shadows? Will it eat it better than other file systems? Maybe.

According to the man page, data=ordered is the default:

"This is the default mode. All data is forced directly out to the main file system prior to its metadata being committed to the journal."

In ordered mode, only the metadata is protected by journaling meaning that the file system's internal data should always be in good order, but the actual file data could for example be corrupted by an incomplete write.

But then how do we end up with zero length files when we

fd = open("foo.new")/write(fd,..)/close(fd)/ rename("foo.new", "foo")<<<CRASH>>>>

if the file data is written before the meta data? By using a kernel older than 2.6.30 or using the non-default noauto_da_alloc mount option on 2.6.30 or later. (See mount(8) under Mount options for ext4, auto_da_alloc / noauto_da_alloc. Also the description for this commit)

From what I can tell, data is written before meta data by default for ext4 in recent kernels. So, while your data may still get eaten, it shouldn't be from metadata being written before data unless you have something like data=writeback,barrier=0 in your mount options.

Disclaimer: I'm not a file system engineer. I could be wrong. This is a "near as I can tell" type post.