Btrfs Seems To Finally Have Failed Me On A Production System

[bulletxt]

Michael, if you continue loosing money you should eventually stop.

This probably ins' t a good business and I don't see this changing in the future.
Make the entire website pro, if people think yours is a good service they will eventuslly pay. Otherwise amen.

[profoundWHALE]

Wrong! Even latest Win 8 could easily hit BSOD if ntfs driver faced corrupted file system. Should there happen to be bad sector on drive - it goes really nuts! Corrupted filesystem structures? Get this cool BSOD. Now! And while ntfs driver BSODs, scandisk/chkdisk often fails to find any issues at all, giving a lot of lulz to owner who is completely unable to mount troublesome volume in windows. Windows just BSODs on mount attempt and ... you can't access your data at all using windows.

Their stupid fast boot causes so many headaches for me. If anything happens to the system while trying to shut down, it's essentially, "screw you!" and you have to reinstall. Of course, you might think that they've either ironed that out or that it doesn't happen often. It's happened to me around 3 times with Windows 8-8.1 and 2 times already with Windows 10.

[SystemCrasher]

That was 20 years ago. Not since then...

Wrong! Even latest Win 8 could easily hit BSOD if ntfs driver faced corrupted file system. Should there happen to be bad sector on drive - it goes really nuts! Corrupted filesystem structures? Get this cool BSOD. Now! And while ntfs driver BSODs, scandisk/chkdisk often fails to find any issues at all, giving a lot of lulz to owner who is completely unable to mount troublesome volume in windows. Windows just BSODs on mount attempt and ... you can't access your data at all using windows. Needless to say, usual users admit EPIC FAIL in such situation and forced to seek help of some gurus around. While I'm not hardcore data recovery guru, I sometimes bother myself to be "in-situ data recovery specialist" and managed to recover data in about 5 such dumbass cases, by simply mounting "faulty" volume under NTFS-3G fuse driver in Linux and copying all or almost all data to another drive, so drive could be re-formatted and is getting back into order. This simple, cheap and dirty trick saved some users a lot of woes :P. And I really do not get why scandisk and chkdisk fails to find errors while driver dares to crashes on mount. That's not what I would call good filesystem toolkit.

Actually, M$ haves surprisingly low code quality: as long as things are not ideal, you can expect worst. Windows still sometimes BSODs when you just insert flash drive, failing to handle debounce at initial connection properly and likely facing some silly race condition. Despite claims they "rewrote whole kernel", this bug persists for about 15 years or so, he-he-he. So it seems they "rewrite" their kernels using copy and paste (why its ok on its own, bold claims and lies about "rewrite" are not ok!).

As for btrfs...
1) fsck mostly a stub to keep boot process happy since btrfs on its own not really supposed to face fatal faults unless underlying hardware failed, etc. It is similar to ZFS in this regard. I guess best idea to boot from livecd (looking on errors it seems your kernel modules gone nuts so I wouldn't expect such OS to work properly). And then use "btrfs check" I guess, if it refuses to mount. Or if it mounts, btrfs scrub could be an option. I suspect faulty kernels maybe wrote some garbage data to disk or so. Yet it's not a good reason to hang indefinitely - filesystem should deal with faulty data and at least return errors and fail requested operations, returning I/O error or so.

For more details on checking btrfs: https://btrfs.wiki.kernel.org/index.php/Btrfsck

Note: it is better to use really recent kernel and tools for recovery. Bugs in FS tools and kernel are not fun, after all :P.

2) If everything has failed BUT you still need to get some valuable data back, you can try "btrfs restore" - a non-destructive attempt to read data and salvage what is actually readable. See https://btrfs.wiki.kernel.org/index.php/Restore for more details.

[Michael]

That systems production task is just benchmarking the latest Git kernel every morning.

Why would you upgrade to an RC kernel on a "production" system? I've been running btrfs on my Arch HTPC for about two years, the only problem I ever had was after upgrading the kernel at one point. Had to run the zero log command from a rescue disk and then btrfsck. Everything running smoothly after that.

I'm even using snapshots quite heavily, and it's saved me countless times when I messed things up. Just boot back into a previous snapshot and to a working state.

[arokh]

Why would you upgrade to an RC kernel on a "production" system? I've been running btrfs on my Arch HTPC for about two years, the only problem I ever had was after upgrading the kernel at one point. Had to run the zero log command from a rescue disk and then btrfsck. Everything running smoothly after that.

I'm even using snapshots quite heavily, and it's saved me countless times when I messed things up. Just boot back into a previous snapshot and to a working state.

[eydee]

NTFS has disk errors all the time, and many times Windows cannot repair the files that are damaged so you are forced to reinstall the entire operating system just to repair a few files.

That was 20 years ago. Not since then...

[mmstick]

Hahaha btrfs is the future they say... hahahahahaha losers all the way. When was the last time ntfs did something like this? 30 years ago? Who are the amateurs working on btrfs, so I know never to use any of their products.

NTFS has disk errors all the time, and many times Windows cannot repair the files that are damaged so you are forced to reinstall the entire operating system just to repair a few files.

[lovenemesis]

Seems like the deadlock issue that hit kernel 3.19 stable release users:

https://btrfs.wiki.kernel.org/index.php/Gotchas

Try use btrfs-zero-log with a older/higher kernel.

One of my Fedora box got hit by this, 4 others remained fine before patched kernel push to update repo.

[cataphract]

As outlined in the earlier post, it's what I do now and - given current circumstances and resources - it's much quicker and easier just wiping the disk and reloading the OS.

You can use btrfs-image to create an image of the filesystem with zeroed data that you can include in a bug report.

[Michael]

Writing an article is _not_ a bug report. Raise one properly and help get _your_ problem sorted

As outlined in the earlier post, it's what I do now and - given current circumstances and resources - it's much quicker and easier just wiping the disk and reloading the OS.