Casefolding For Bcachefs File-System Posted

[Mitch]

well, you're actually accusing me of not being genuine here, but if https://wiki.debian.org/Btrfs is the "list" you're referring to, then it's you not being genuine here.

* firstale, what you're referring to as "a list of btrfs problems" is just a documentation wiki page.
* secondly, is not at all regarding "problem related to btrfs".

The only few problems listed there are either:

* interaction with other tools (ie: bcache, LVM) that are directly attributable to them
* regarding VERY old kernel version

all the rest are good practices, advices and features documentation.

these are just some "problems" copied and pasted from the page:

* "Many people have experienced upwards of six years of btrfs usage without issue"
* "Does btrfs really protect my data from hard drive corruption? Yes"
* "Does btrfs work on RaspberryPi? Yes, possibly improving filesystem I/O responsiveness."



what does it mean "it isn't for some environments"?
There are environments where XFS is not fit, or environments where not even Linux is fit.
So what?

Should we take the XFS documentation page, call it a list of problems, and post everytime there's a filesystem article on Phoronix?

Lol, I know. Every time I hear someone say Btrfs is so unusuable and slow and corrupts data, it comes off to me as if someone were to say "Linux, the OS that can't game". Software sometimes changes a LOT, and improves a LOT. Btrfs still has gaps, but those are pretty well-understood and well-documented. I use Btrfs as my primary FS on every install and I face no problems. I also have a Btrfs RAID10 array that works flawlessly (I know it's not the normal RAID10, but it works great).

[oleid]

Yes, I do have a link, but I don't believe you could be genuinely interested, as searching for it is as easy as looking for "debian btrfs" in the engine of your preference and hitting the first result.

Debian wiki is my first result. But that's not what you're describing. The next few results don't look like a bug tracker as well. You must be aware that the order of search results highly depend on the search engine and your native language.

And you are right on the kernel version while it can be old for some, you must realize that it actually isn't for some environments;

We use certain elder (and unsupported) kernels at work in embedded devices. But they don't use btrfs, so I guess they don't count.

[oleid]

I haven't had Btrfs wreck any data in a long, long time now, but I can make Btrfs hard crash quite easily on 5.15 under Ubuntu 22.04 LTS. A hard crash at the filesystem level is still pretty bad, regardless of whether any data gets wrecked or not, IMO.

How can you make it crash easily?

[curfew]

How can you make it crash easily?

I used to nuke my system a few times after I went aggressive on the undervolting (on a laptop).

The system would work just fine during my daily tasks but would crash during a system upgrade. I guess the high number of reads and writes across the disk, combined with not having enough power, made the disk controller or whatever fail. Anyway, the end-result was a kernel panic with an error message related to the filesystem / Btrfs. There would be lots of zero-sized files, and my only way to recover was reinstalling packages on top the existing OS. (Which on Arch is pretty easy but still takes some time.)

I'm not blaming Btrfs, as that was likely a physical device fault, but it sure did look like a Btrfs failure when overlooking the circumstances.

[billyswong]

Unless I'm missing something the reason some games may crash on btrfs is precisely because btrfs doesn't do case folding, if the game is sloppy with path resolution it can't find files then shits itself. More microsoft baking a ticking compatibility timebomb into ntfs than anything btrfs has done wrong.

I think it is unfair to blame NTFS. The case insensitivity is at least as old as the original FAT filesystem in DOS era. Apple computers at that time also followed suit. ISO 9660 the first filesystem format for CD-ROM also followed that tradition. Nobody in Microsoft would let NTFS break the compatibility.

According to Wikipedia, there are older OSes/filesystems that are also case insensitive. Namely DECTape (and also Level-D, RT-11 etc) for PDP computers since 1964 and ODS-1 (and various -2, -5 etc) for PDP-11 and later on the VMS operating system since 1975. Bill Gates probably designed the FAT filesystem with them in mind.

Nowadays we take ASCII with uppercase and lowercase letters for granted but in the earliest days, the Morse code for telegraph is case insensitive.

[stormcrow]

F2FS compression does not save space, unless you manually make the compressed files immutable.

Also it has a relatively short file extension limit... I can't remember what it is exactly, as its not documented, and I had to find it in the source when I ran into it.

As for the crashes, some games are unhappy without casefolding (which is on by default in Windows). But this may not be the issue, as its not enabled in ext4 by default, and IIRC proton takes care of it anyway.

The point is that it does support compression. The OP said it didn't. The reasoning it does what it does and why it does it are sound if you actually read the documentation re: transparently compressed files with recoverable space must be write-once (immutable). This is a flash media quirk it's working around, that is, it's minimizing the write amplification related to rewriting previously compressed files when they can be still potentially written to. If that file has lost the area of disk where it would normally reside were it uncompressed, the entire file has to be rewritten elsewhere doubling the write wear (filesystems still avoid unnecessary fragmentation, because it has consequences even on flash media). Compression on flash media isn't always about conserving space. It's about conserving the number of IOPS needed to write data. Fewer writes means less wear on the cells whereas the number of cells holding data is less of a concern because of how cells are grouped. It's counter intuitive, but true.

[WereCatf]

How can you make it crash easily?

Use mergerfs to combine a couple of Btrfs disks, then point Radarr at it and BOOM -- dmesg shows a crash in the Btrfs module within the kernel and the system has to be hard-rebooted, because no normal filesystem operations work anymore. Another way of doing it is with LXD, but it's a bit lengthier process, so I can't be arsed to go through it here.

[Democrab]

There's not any reliability problems with btrfs outside of raid 5/6, iirc the steam desk uses btrfs for the root fs but ext4 for the game data partition.

Some games however, don't run on XFS/btrfs and want an ext4 fs. It's also not the fastest FS right now

Even btrfs raid5/6 is mostly fine these days provided you're aware of the best-practices (such as using raid1/1c3/1c4 for metadata) and research what exactly the known issues with it are. I've had an array running fine for about 3 years now but more recently the 6.2 kernel update included some pretty big relevant changes to the btrfs raid56 code, I'd say it's fine for home lab usage at the very least especially as most home labs in my experience mostly deal with redownloadable data and a small portion of data that would be genuinely unrecoverable if lost which should have at least two other copies if you're actually following 3-2-1 regardless...Although it will be quite slow sometimes (At least on 7200rpm drives) so be prepared for that.

I'm even getting a lot of mileage out of btrfs on my desktop thanks to winbtrfs making it probably the best solution for dual-booters to share data between Linux and Windows, I grabbed a cheapo 6TB Seagate drive for the vast majority of games that don't really benefit from fast storage on which I've made two subvolumes, one for Linux and one for Windows, so both Windows and Linux think they've got completely unique installs of the same games despite dedupe ensuring minimal redundant data is actually being stored and additionally I have a 2TB Toshiba drive which I use for the user libraries (ie. The "Documents", "Music", "Pictures", "Videos" folders in /home/username and in Windows' user folder) once again shared between both OS' and has snapshotting enabled because that can be so bloody handy for those kinds of files. Other than that, I just use F2FS for my SSDs.

Use mergerfs to combine a couple of Btrfs disks, then point Radarr at it and BOOM -- dmesg shows a crash in the Btrfs module within the kernel and the system has to be hard-rebooted, because no normal filesystem operations work anymore. Another way of doing it is with LXD, but it's a bit lengthier process, so I can't be arsed to go through it here.

Out of curiosity, if you're only combining btrfs formatted disks wouldn't you just btrfs' inbuilt JBOD handling rather than MergerFS? Does MergerFS have additional features you're using or is there a drive with an alternative fs there?

[WereCatf]

Out of curiosity, if you're only combining btrfs formatted disks wouldn't you just btrfs' inbuilt JBOD handling rather than MergerFS? Does MergerFS have additional features you're using or is there a drive with an alternative fs there?

In JBOD, you lose all the drives if you lose one drive. With mergerfs, they're all still individual disks, so losing one disk results in only that disk's contents being lost. With Btrfs, you'd probably use RAID1 for metadata in such a setup, but when I experimented with Btrfs JBOD-like, the performance was terrible. I'll admit it may have just been my system or something I did wrong, but I went with mergerfs instead and haven't bothered trying again.

EDIT: Now that I think about it, I'm unsure how Btrfs JBOD-like handles files that don't fit on a single disk, like e.g. that disk is getting full and there's no room to hold the full file. Does it spread the file over multiple drives, then? Mergerfs handles it so that each and every file is fully on a single drive, so you can't lose files due to them being spread over multiple drives. If Btrfs doesn't do that, then that's a big plus for mergerfs in such a scenario.

[kreijack]

[...]
EDIT: Now that I think about it, I'm unsure how Btrfs JBOD-like handles files that don't fit on a single disk, like e.g. that disk is getting full and there's no room to hold the full file. Does it spread the file over multiple drives, then? Mergerfs handles it so that each and every file is fully on a single drive, so you can't lose files due to them being spread over multiple drives. If Btrfs doesn't do that, then that's a big plus for mergerfs in such a scenario.

In BTRFS there is no an official JBOD disks arrangement. There are two profiles which may be used as JBOD:
1) SINGLE (that is the one used in your link), in this profile each block-group (which is the allocation unit of btrfs) is allocated to the disk with the most free space. If you have two disk of the same size, the first block group is allocated on the first disk, the second block-group on the second disk, the third block group on the first disk... So a file may span different disk even if the filesystem is near empty.
2) RAID0, in this profile the data is spread over the disk in 64kb unit (i.e. the first 64kb is write in the first disk, the second in the second and so on)...

In both the cases, there is no guarantee that the filesystem survives to the lost of a disk.