Bcachefs Submitted For Review - Next-Gen CoW File-System Aims For Mainline

[cynic]

I don’t like unnecessary reimplementations of existing features, but I have to admire releasing something that’s so feature-complete. I hope it allows you to mark certain files to always be cached, because that’s something that seems to be missing from plain bcache and other caching solutions.

I'm sure you already know this, but just in case... for such need I use vmtouch.

[Old Grouch]

I really, really like the NILFS2 capability for 'continuous snapshotting'

NILFS2 is a log-structured file system (LFS) supporting continuous snapshotting. In addition to versioning capability of the entire file system, users can even restore files mistakenly overwritten or destroyed just a few seconds ago. Since NILFS2 can keep consistency like conventional LFS, it achieves quick recovery after system crashes.

NILFS2 creates a number of checkpoints every few seconds or per synchronous write basis (unless there is no change). Users can select significant versions among continuously created checkpoints, and can change them into snapshots which will be preserved until they are changed back to checkpoints.

There is no limit on the number of snapshots until the volume gets full. Each snapshot is mountable as a read-only file system concurrently with its writable mount, and this feature is convenient for online backup.

Unfortunately, as far as I can see, neither bcachefs nor btrfs offer this, although my naïve view is that 'it shouldn't be too difficult' - being able to roll back to an arbitrary point in time is amazing, and goes a long way to meeting (and in fact bettering) the need for file versioning (which VMS Files-11 ODS-2 or ODS-5 offered).

[sobkas]

Well there is fs/bcachefs:

https://evilpiepirate.org/git/bcachefs.git/tree/fs/bcachefs?h=bcachefs-for-upstream

[some_canuck]

There is a growing problem for anything based around RAID5 and RAID6 ideas as drives grow in size the rebuild time is going up. Simpler duplicate the file solutions have reduced rebuild times.

zfs i/o is broken down into several groups: sync read, sync write, async read, async write, scrub/resilver. resilver has the lowest priority as to not impact performance on an active pool... you can turn it up if you want, at the detriment of all other i/o... theoretically a full 3tb resilver at sata speeds should take 2 hours.

[uid313]

Maybe it would have less bugs if it was written in Rust?

Now that the Linux kernel finally has (some) support for Rust it would be amazing to get a file system written in Rust.

[zexelon]

Maybe it would have less bugs if it was written in Rust?

Now that the Linux kernel finally has (some) support for Rust it would be amazing to get a file system written in Rust.

Not an expert opinion here so take it with a grain of salt, but I don't think changing the language would have any affect on the "major bugs" that a file system could run into. Yes Rust is a "safer language" from a memory management side of things perhaps that c/c++ but that has no affect on logic bugs.

The write hole issue mentioned usually as soon as Btrfs is mentioned is a logic issue. You could use any language you wanted and this issue would still exist.

That said, it would be very interesting to see a FS built in Rust, even just to see what could be better/worse, you never know until you try!

Probably the simplest thing to start with would be a re-write of say the FAT32 file system in Rust and see what the benefits could be on a smaller scale first.

[evert_mouw]

Not an expert opinion here so take it with a grain of salt, but I don't think changing the language would have any affect on the "major bugs" that a file system could run into. Yes Rust is a "safer language" from a memory management side of things perhaps that c/c++ but that has no affect on logic bugs.

The write hole issue mentioned usually as soon as Btrfs is mentioned is a logic issue. You could use any language you wanted and this issue would still exist.

That said, it would be very interesting to see a FS built in Rust, even just to see what could be better/worse, you never know until you try!

Probably the simplest thing to start with would be a re-write of say the FAT32 file system in Rust and see what the benefits could be on a smaller scale first.

Chech out REDOX... encryption, CoW

6 years ago: https://github.com/redox-os/tfs
currently: https://github.com/redox-os/redoxfs

more info: https://news.itsfoss.com/redox-os-0-7-0-release/

[zexelon]

Chech out REDOX... encryption, CoW

6 years ago: https://github.com/redox-os/tfs
currently: https://github.com/redox-os/redoxfs

more info: https://news.itsfoss.com/redox-os-0-7-0-release/

Yes, I have been watching REDOX for a while also, I think I even tried it out a while ago in a VM. It has some really interesting takes on things. I am undecided on their "everything is a URL" idea. I think it has some really interesting potential, way to early to tell though.

Their file system is in the same boat, just way to early.

I really hope REDOX can get traction though, their unified approach is an excellent counter point to the chaos that is the Linux ecosystem.

As always software compatibility is king, they need to make sure it is at a minimum Linux compatible, and if they can get full compatibility on Linux drivers without sacrificing to much on their "OS ideals" they could become a really unique system!

[evert_mouw]

Agreed. Their approach is promising!

[blackiwid]

What I miss a bit is either deduplication online so that a copy on a filesystem is automatically create only a hardlink basically. I don't even talk about blocklevel just file level would be enough.

And or alternatively some sort of tagging that get's represented by folders. I mean you could do the tagging with some bash scripts that create some folders and links according to the command, but yes deduplication would be a key to categoriese data in more than 1 way so for music you could have some sort of by-year / by-genre / all folders by example, with mp3s you have the internal tagging and many tools that list it that way, but that is not true for everything, for video files as example there are to much different tagging systems and many players don't support it or don't build you lists of genres and stuff like that. But the same with documents no matter txt / pdf or odf.