Bcachefs File-System Is Working On Going Upstream In The Linux Kernel

[jwilliams]

Bcachefs is a copy-on-write file-system that supports native compression, encryption, caching, snapshots, multiple devices, and other modern capabilities.
http://www.phoronix.com/scan.php?pag...Upstream-Start

That statement in the article is misleading. For one thing, Bcachefs does NOT currently support snapshots, and if you read the status, they are a long way from being ready. Encryption is apparently not even started yet.

Probably should have phrased it as "supports, or has plans to support..."

[Vistaus]

Which is currently providing only $1754 per month. So I doubt he considers that support for full-time work.

Depends. While $1754 may not sound like a lot for a full-time job, it depends on his IRL job. I mean: let's say he's only making $1400/month in his IRL job and he makes ends meet with it, then $1754 is more than enough to replace his IRL job.
I earn less per month than what he gets from Patreon donations and while I'm far from rich and I also have my cat to take care of, I do make ends meet and my apartment is plenty big, it has more space than I need. So nothing to complain about, really. So for him, $1754 may just be enough.

[DrYak]

I might be getting things wrong, or maybe the patreon page and the website are both out of date, but the among the main features that attract user to new-gen FS like BTRFS, ZFS and BCacheFS :

- only checksum and encryption are currently finished in BCacheFS (and encryption hasn't been tested/reviewed much, and only features 1 single choice of AEAD - though a good modern one (ChaCha20-Poly1305), at least)
( -Well, tiered caching works too. But that used to already be the case back in the BCache era, before Kent even started to work on a FS. He has been merely bettering the system with "disk group storage" - more generic, with "caching" being a feature that comes naturally for free from it).

For the rest:

- snapshots, the most important key feature for all these next-genFS and the one which is arguably the most complex to get right, HAS BARELY STARTED BEING WORKED ON.
- compression doesn't really work in practice (BCacheFS does compress the data with either zlib or lz4, but ends up using the original allocation size on disk anyway so you only get the speed boost of lz4, but none of the space advantage).
- RAID1 isn't quite there yet (BCacheFS can already write 2 copies, but is still unable to use the alternative copy for recovery)
- RAID5/6 (you know the things for which Btrfs gets regularily bashed for not being "stable yet", and only recently had patches able to use alternative blocks for recovery) hasn't been even started in BCacheFS.

In short, the "caching" bit is the only main thing that BCacheFS currently has to show off (and currently could be implemented with BCacheFS's ancestor on any other filesystem).
All the other things (compression, RAID1/5/6 and - most important - *snapshots* !) aren't there to show.

Isn't it a bit premature to think about kernel inclusion ?
(Though Kent has managed to get a corporate sponsor according to his feeds, so at least that requirement for in-kernel inclusion is now finally fulfilled)

Specially regarding snapshotting : it's the most complex "next gen" feature to get right. It's not finished yet.
Kent thinks that the on-disk bit format is ready, but until the snapshotting feature is finished and considered stable, I'm not ready to follow his confidence.
I get it from the way he develop things that he probably intends to try to make use of the generic copy-tiering system with some reference counting/modified garbage-collecting to keep some older copy versions around. So in his mind, the necessary building blocks (copy counts management system) is already there.
But I am still afraid he's going to get bitten in the ass by some unforseen tiny details, and that he'll definitely need to change stuff ondisk before the set of key features is ready. Meaning that the current BCacheFS format won't be compatible with the final feature complete version.

He is taking jabs at BTRFS having been rushed "too fast without proper planning in advance" leading to too much code bloat. But he wants to mainline a next-gen FS before having the *snapshotting* feature complete ?
Yes, he makes big promises that one day, once it's implemented, they will by fast and lightweight and nice.
(Like the thing which is provided - currently in production - by other CoW FS like BTRFS, or even in a slightly less advanced manner by log-structured systems (UDF, F2FS, etc.). Unlike the slowness of block-level snapshotting like LVM and most virtual machine disk images).
But it's not there *yet*.

When at the same time his "btrfs" favourite punching-bag only has the "RAID5/6" feature currently considered unstable (and patches already available to make it leverage checksuming and alternative blocks in recovery. Making the final stabilisation actually possible within a near future). Including the efficient snapshots that can be relied on (used by containers systems such as LXC and Docker ; used by openSUSE's snapper system doing exactly what he advertises as a use case, etc.).


Kent IS doing a great work. And file systems ARE complicated (I wouldn't even dream being capable of making such one my self).
But I sincerely think that his moves toward mainlining are a bit premature.

[caligula]

Depends. While $1754 may not sound like a lot for a full-time job, it depends on his IRL job. I mean: let's say he's only making $1400/month in his IRL job and he makes ends meet with it, then $1754 is more than enough to replace his IRL job.
I earn less per month than what he gets from Patreon donations and while I'm far from rich and I also have my cat to take care of, I do make ends meet and my apartment is plenty big, it has more space than I need. So nothing to complain about, really. So for him, $1754 may just be enough.

The income and cost levels aren't universal. $1754 is A LOT in the poorest parts of Africa (in Uganda it's 2.5 times the annual GDP) while in the Bay Area / Silicon valley it's a joke.

[starshipeleven]

The income and cost levels aren't universal. $1754 is A LOT in the poorest parts of Africa (in Uganda it's 2.5 times the annual GDP) while in the Bay Area / Silicon valley it's a joke.

Given the statement on Patreon about his close encounter with a "moose" I'm speculating that he is probably not in the poorest parts of Africa.

[tg--]

He's living in San Francisco, where the current average rent for a one-bedroom apartment is about 3200 USD, so I think it's fair to say, that he won't be working on bcachefs exclusively for a while. At least not on the patreon money.

Still, looking forward to see it develop, I too have been burned by btrfs too many times and would really love a replacement with the same capabilities.

[jwilliams]

I might be getting things wrong, or maybe the patreon page and the website are both out of date, but the among the main features that attract user to new-gen FS like BTRFS, ZFS and BCacheFS :

- only checksum and encryption are currently finished in BCacheFS (and encryption hasn't been tested/reviewed much, and only features 1 single choice of AEAD - though a good modern one (ChaCha20-Poly1305), at least)

Where are you getting that encryption is finished?

The status page at https://www.patreon.com/bcachefs says, "Encryption: Not yet started"

[jwilliams]

So for him, $1754 may just be enough.

Nope, it says right on his Patreon page that he is looking for $3000 per month.

[oiaohm]

That's pretty darn sweet, but I still fail to see it gaining traction. RH pushes Stratis, SUSE pushes Btrfs, Ubuntu pushes ZFS. That's freedom of choice for ya.

Not quite on with RH. Stratis is a stop gap from description "devicemapper subsystem, and the XFS filesystem". https://lwn.net/Articles/747633/ Now XFS is starting to be extended to include the extra features. The interesting part is the Xfs lead developer is attempting to avoid the defects of both Btrfs and ZFS.

[oiaohm]

xfs, which is reliable and robust but still fundamentally a classical design - it's designed around update in place, not copy on write (COW). As someone who's both read and written quite a bit of filesystem code, the xfs developers (and Dave Chinner in particular) routinely impress me with just how rigorous their code is - the quality of the xfs code is genuinely head and shoulders above any other upstream filesystem. Unfortunately, there is a long list of very desirable features that are not really possible in a non COW filesystem, and it is generally recognized that xfs will not be the vehicle for those features.

https://lwn.net/Articles/747633/
Sorry XFS is a data copy on write filesystem. XFS includes in current form data copy on write. What has been generally recognised as requiring a full COW file system turns out to be wrong. Hybrid between data copy on write and update in place metadata has been demoed. The hybrid prototype seams to show that it can do all the functionality of full copy on write file systems without the worst overheads and issues of full copy on write file systems.

Data copy on write file system is a lot simpler item to create than a full copy on write file system. Data copy on write allows effective duplication reduction.

XFS might be the first copy on write file system with all features that does not have strange issues. This is while keeping update in place metadata system. Basically lets not reinvent the wheel if we don't have to.

https://en.wikipedia.org/wiki/Ext3cow
Note what XFS is doing is not the first of either by using data cow with update in place metadata to create a copy on write file system with most of the functionality.

Really we might end up hating ZFS because its method that btrfs and other attempted to follow might have been completely the wrong way.