EXT4 Lets Us Down, There Goes Our R600/700 Mesa Tests

[energyman]

yeah yeah, backups. Some of you do life backups of everything at once. Right?

Fact is, data loss always appear at the most inconvinient point. Another fact is that ext4 is utterly broken.

No backups can help you with that.

[SunnyDrake]

From times when i used smartdrive tool in dos to make drive writes @virtual@ this became play of tradeoff between performance/reability.
This topic is quite interesting as it shows weak point in fs's that phoronix avoided in articles. It will be nice to see new article where fs's data loss/corruptions tested based on kernel lockup,powercut,real data write under heavy cpu load, memory usage, cpu usage, performance,amount of hardware write calls in stock settings and fail-safe settings.

[deanjo]

yeah yeah, backups. Some of you do life backups of everything at once. Right?

Fact is, data loss always appear at the most inconvinient point. Another fact is that ext4 is utterly broken.

No backups can help you with that.

Lets not forget even the most prudent "backup" person has at some point had a moment of laziness at which Murphy's Law will kick into play.

[energyman]

yeah. Or the enlightening moment you realize that your backup solution has been writing garbage to the medium for the last couple of month. Or when the controller you want to use to replay the backup fries the disk.

backup only reduces the chance of catastrophic loss. But you can never reach 0.

[deanjo]

backup only reduces the chance of catastrophic loss. But you can never reach 0.

Yup chances are that a simple rsync to a usb keychain after each test ran would have been just as good as any of the other presented solutions in the long run.

[phtpht]

Learn to 'sync' when you want your data safe. File systems and block devices are at liberty to do anything but write the data until you 'sync'. But yeah right, blame ext4.

[movieman]

But yeah right, blame ext4.

Sensible filesystems don't require you to sync if you want your data to actually be written to the disk in a safe manner, so if ext4 doesn't do that, it's simply broken as a general-use filesystem.

More to the point, the only reason to use ext4 is performance, and if you're now going to have to sync every time you write anything to disk, your performance is going to be worse than a reliable filesystem like ext3. So what's the point?

I've been running ext4 on my netbook because it's now the default for Ubuntu, and so far I'm amazed that nothing bad has happened after the times I've had to do a hard power down due to NFS lockups. I'm still wondering what may have been silently corrupted without my knowledge.

[phtpht]

Sensible filesystems don't require you to sync if you want your data to actually be written to the disk in a safe manner, so if ext4 doesn't do that, it's simply broken as a general-use filesystem.

More to the point, the only reason to use ext4 is performance, and if you're now going to have to sync every time you write anything to disk, your performance is going to be worse than a reliable filesystem like ext3. So what's the point?

Not true. Since the dawn of times drives have used in-memory "write-back" caches. Your operating system has one, your RAID controller has one and hell even most modern consumer drives have one:

$ cat /sys/block/sda/device/model 
WDC WD6401AALS-0  
$ cat /sys/block/sda/device/scsi_disk\:0\:0\:0\:0/cache_type
write back

So what do you think that happens when you cut off power while the data is still in cache but not on disk? Data lost.
When and how often a cache flushes its contents to the drive depends solely on its logic. If you want to guarantee your data on the disk, you will use sync, period. Transactional databases do that on end of each transaction and every sane person does that as well. Note that this has NOTHING to do with the file system used.

However, on that matter. Some file systems are more jerky wrt cache and some are more intelligent. EXT3 is more jerky. Normally you can ask a filesystem to synchronize the contents of one file (or a portion of) to the disk. Not with ext3. It can't flush the contents of a single file; when you call sync, it flushes everything instead. Therefore it's more likely that your data gets on the disk eventually. But that's also why each sync takes sooooooo looooooong on ext3 and that's why people have GIVEN UP on doing that and that's why IT SEEMS that ext3 is more reliable (unless you use Ubuntu which uses unsafe defaults).

EXT4 is more mature in this manner. That means it works with the cache as good as the other filesystems like XFS or JFS have worked for decades now. But because of EXT3 history, people forget to goddamn sync their data when they want them on the disk. Then they whine that if their game freezes they lose their goddamn bookmarks with EXT4.

So you want to either use a sissy filesystem like EXT3 and forget about my lecture here or you use a real one and learn how to use it properly. The balance between reliable and fast is more in your hands, you control when and how the cache gets flushed. If you do it right you will gain MORE performance as opposed to your opinion.

And by the way, the article says that they also lost data which were already on disk and which they did not touch for some time. That means they either hit a filesystem bug (it's immature, but not weird in its cache usage); or they screwed up the hardware; or they use some insecure mount option (it's ubuntu you know).

[energyman]

XFS has been critized for its bad behaviour on crashs. There was even the rule that with XFS you must use a PSU or be prepared for catastrophal data loss.
Everybody ridiculed it for that behaviour.

And ext4 copied that crap. Zero lenght files. Too much data on the fly. Everything that is bad about XFS got copied.

You want a fs that cares about your data?

[ 269.960844] reiser4[postdrop(3406)]: disable_write_barrier (fs/reiser4/wander.c:235)[zam-1055]:
[ 269.960849] NOTICE: md1 does not support write barriers, using synchronous write instead.

user reiser4

[phtpht]

XFS has been critized for its bad behaviour on crashs. There was even the rule that with XFS you must use a PSU or be prepared for catastrophal data loss.
Everybody ridiculed it for that behaviour.

And ext4 copied that crap. Zero lenght files. Too much data on the fly. Everything that is bad about XFS got copied.

Urban legends. All XFS does it truncates files that would on other FS's have bullshit written into them. The idea being that you're more likely to notice the former.

You don't need a PSU. You need to backup your data while you have it all right. And that's a general rule with any file system.

And it has precisely nothing to do with cache usage. If you're complaining about too much data in the cache you can always shrink it or disable it totally. You have ample options and knobs to do whatever you please. Using EXT3 that ignores most of them does at most bring false sense of being in control and secure.

EXT3 is a pathetic caricature, a mild improvement over EXT2, but nothing more, it was brought to this world to make countless EXT2 disks migrate easier to journaling but it took its toll. Every time I did some real work on EXT3 it took freaking more time than on XFS or even reiser; yet EXT3 has been the frigging default for almost all distro's.

You all EXT3'ers have been actually using EXT2 for all the time. (EXT2 is not bad, but it's simple and plain and has its limits.) Now that EXT4 brings real changes and features and pushes finally the default configuration to be on par with other FS's you either mindlessly whine or turn all the cool stuff off effectively retarding it to EXT2 again.

You want a fs that cares about your data?

[ 269.960844] reiser4[postdrop(3406)]: disable_write_barrier (fs/reiser4/wander.c:235)[zam-1055]:
[ 269.960849] NOTICE: md1 does not support write barriers, using synchronous write instead.

user reiser4

If your block driver doesn't support write barriers then you're screwed. You can not hope to have a functioning journaling FS without barriers.

You can always mount almost any system with -o sync to achieve the abovementioned effect. Your write performance will suffer horribly. The old Zaurus handhelds had synchronous writes hardcoded in the kernel. By the time it copied your photos onto a USB flash you could make a ragu for dinner.