Bcachefs File-System Might Be In Shape For Upstreaming In The Linux Kernel In 2019

[phoronix]

Phoronix: Bcachefs File-System Might Be In Shape For Upstreaming In The Linux Kernel In 2019

Bcachefs lead developer Kent Overstreet provided an update to the kernel developers on his file-system over the weekend, following his Patreon post that updated followers on the latest for this "next-gen" Linux file-system born out of the block cache code...

http://www.phoronix.com/scan.php?page=news_item&px=Bcachefs-2019-Upstream-Maybe

[Naib]

This is really good news!
if you look at the git repo, Linus has been tagging versions every so often.

Out of the so called "next gen filesystems" this is the one I am most interested in

BTRFS is just turning into a mess
ZFS has some annoying licensing issues
stratis a dbus wrapped filesystem is just ....

When I built my present system about 1-2years ago I considered BTRFS or going ZFS but neither ticked all the boxes so I went Ext4. Now BCacheFS development seems to be doing things right. Ensure the low-level (what some perceive as boring) is rock-solid. This provides a solid foundation to add all the funky things. This is where BTRFS went wrong, they went in to show the bells and whistles without architecting the file-system.

I really want BcacheFS to succeed and if it is being submitted for inclusion, this will increase its exposure


Oh and those questioning my statement about BTRFS:

https://forums.gentoo.org/viewtopic-t-1081990-highlight-bcachefs.html

Btrfs has been in development for over ten years now, and while the basic stuff is being considered stable, many of the more advanced features are in development, not really finalized yet and some are even unstable - check this out: https://btrfs.wiki.kernel.org/index.php/Status. If you want an even more sceptic analysis of its status, look here: https://github.com/mosteo/btrfs-status

The show stopper for enterprise use is in many cases that Btrfs doesn't support triple parity, while ZFS has been offering this feature since ages. Another show stopper for using btrfs is that it just plainly sucks with VM images and databases, in fact the official recommendation for both is to switch COW off then, because otherwise your file might fragment in literelly ten thousand of pieces or more. But what's the point in using VM images/database storage on a COW file system, when you cannot use the COW features then? For both usage types you are way better of with XFS, or ext4 - in that order.

Also another problem with the still unstable state of btrfs is that putting it on a server means that you should be able to roll out the latest and greatest kernel version every release cycle, because it may contain important bug fixes and performance optimizations. The thing is if your preferred Linux distribution doesn't do backports, than choosing Btrfs might not be such an exactly brillant idea, because when you are running into issues the developers will tell you to switch to the latest and greatest kernel.

Facebook by the way has been testing it in production, but mostly for the deployment and management of system partitions, and not much more. Btrfs on Red Hat has never been a first class citizen, but yes, they lost faith in it completely.

If you want to get real insights what using btrfs in production use meaned, just look at CoreOS. They rolled it out as their default file system in 2014, and the complaints and problems about it skyrocketed to such a boiling point, that they ditched it entirely back then and switched to ext4 + overlayfs early in 2015. The most typical complaints were out of disk space errors, metadata rebalancing problems, and generally slow performance compared to other file systems (https://lwn.net/Articles/627232/). Rebalanciing problems are even an issue in 2018: https://www.spinics.net/lists/linux-btrfs/msg74892.html

Russel Coker from Australia is writing in his personal blog about his experience with Btrfs on his own machines, which he started bloggin about in 2012 and you might also want to check here: https://etbe.coker.com.au/ In the first few years he had a major hiccup almost every six months or so, were he had to ask the mailing list for help, but then it became more reliable and it slowed down.

I also used it several times on my own, and it always ran okay for some weeks then, when problems started to occur and the fun stuff happened. And RAID1 is now nothing fancy in particular. I always had a backup of my data, just in case, and I always needed it.

Having said that, Btrfs is fine to play around with in my opinion if you are using it with throwaway data only, or you get a regular backup of your data. I would never, ever use it on a production level system.

Synology though does use it since over two years as standard file systems, and they seem do be quite happy with it.

To make it short: if you really need a COW file system on Linux, then take ZFS. If you have issues with it on a Linux machine due to CCDL vs GPL, then switch over to FreeBSD. Btrfs has been a nice idea, but is still to much construction area, and a convoluted mess in reality which I would not trust my data with. And BcacheFS might be what could become the new king, but give it at least two or three years, because they are still lacking some basic features like scrubbing. The thing though is that many people, just like myself, have lost their faith in Btreefs completely, so time is not such a pressing issue for BcacheFS at the moment as some might think.

And by the way Oracle recently in February this year completely unexpected relicensed another cron jewel from Sun, dtrace, under the GPL. Maybe ZFS might follow, who knows?

[DrYak]

BTRFS is just turning into a mess

And still somehow openSUSE has managed to make it stable and usable in production, and it is their default filesystem.
Maybe because they actually test it to work kinks out, instead of just adding an option in the installer drop menu.

And BTRFS has always worked for me, no major data loss. Ever.

But maybe that's because I've taken time to understand what's specific here.

I really want BcacheFS to succeed and if it is being submitted for inclusion, this will increase its exposure

I am interested by it too.
I specially like the way Kent is solving some of the RAID5/6 problems (regarding read-modify-write cycles)., by leveraging the "layered storage".
Basically, storing multiple copies (RAID1), until there is enough to make a simple RAID5/6 write instead of read-modify-write, and then garbage collect the superfluous copies. (Dropbox' Magic Pocket also uses the same approach).

But he's a single guy and still hasn't talked some of the most difficult and critical part of a modern FS.

Oh and those questioning my statement about BTRFS:

Another show stopper for using btrfs is that it just plainly sucks with VM images and databases, in fact the official recommendation for both is to switch COW off then, because otherwise your file might fragment in literelly ten thousand of pieces or more. But what's the point in using VM images/database storage on a COW file system, when you cannot use the COW features then? For both usage types you are way better of with XFS, or ext4 - in that order.

Indeed, why the hell are you storing databases (a thing that does its own journaling) or a VM (which usually internally runs its a filesystem with its own safety features), above a COW system ? Are you running your mdadm RAID6 on SCSI/SAS enclosures that do their own RAID6 internally ? (and then complaining about the slowdown of matroska-shaped read-modify-writes ?)

Note that the same problem affects ZFS (And they have specific countermeasures against it) and that recent versions of BTRFS have autodefrag for the exact same reasons (group writes together in single long extent to avoid excessive fragmentations).

either flag the file for nocow (chattr +C) or use a inplace-modification FS like EXT4, XFS, etc.

I always had a backup of my data, just in case, and I always needed it.

Having said that, Btrfs is fine to play around with in my opinion if you are using it with throwaway data only, or you get a regular backup of your data. I would never, ever use it on a production level system.

Been using BTRFS in production on multiple system for years, never needed to bring out the backup.

Synology though does use it since over two years as standard file systems, and they seem do be quite happy with it.

Probably the same situation as with openSUSE : a company that took time to test and understand how to make BTRFS best work for them.

And by the way Oracle recently in February this year completely unexpected relicensed another cron jewel from Sun, dtrace, under the GPL. Maybe ZFS might follow, who knows?

Yeah, keep dreaming. They'll GPL it on the day BCacheFS is out and stable and starts to eat their market. Until then, it's going to stay kernel-incompatibly licensed.

[bobpaul]

https://forums.gentoo.org/viewtopic-...-bcachefs.html [Quo,te] And by the way Oracle recently in February this year completely unexpected relicensed another cron jewel from Sun, dtrace, under the GPL. Maybe ZFS might follow, who knows?

[/QUOTE]

This is completely impossible. Oracle doesn't own all the copyrights for OracleZFS let alone OpenZFS. OracleZFS isn't compatible with OpenZFS. Nobody cares about OracleZFS anymore. Relicensing OpenZFS would require Oracle to commit to relicensing, but it would also require tracking down over 10 years of contributors. ZFS has been open source since 2005.

A lot of ZFS contributors wouldn't be ok relicensing to GPL as that would make it incompatible with BSD projects. CDDL limits to the file scope, so while it's copyleft, it can still live in BSD licensed kernels. Meanwhile a lot of other contributors would object to licensing it under BSD. If it were BSD licensed it could be used in both GPL and BSD kernels, but it could be used in closed source kernels, too, which a lot of contributors would object to.

The earliest you can expect ZFS to be relicensed is approximately 150 years from now. Pick a version you like and then tally up all the contributors for that version and wait. It takes 70 years after the deaths of all contributors for the copyrights to expire. Note that you'll be stuck on whatever kernels supported that particular version.