Btrfs Async Buffered Writes Slated For Linux 6.1 - 2x Throughput Improvement

[zexelon]

This is just another way of saying you don't have a source for your claim that Tectonic is their largest filesystem.



Only if you are willing to dismiss all users using containers as not "regular". Also as I noted already, several major distributions are using it by default and regular users are therefore using on bare metal systems as a result. Instead of conceding this point, you made a silly comparison to illegal drugs instead.

Okay lets start with a definition, you and I are clearly using words in the English language, but we are putting different meanings to them. Please define for me what you mean when you use the term "user", or to match what I say, "regular user". What is a "regular user"?

The very first line in the document "Tectonic is Facebook’s exabyte-scale distributed filesystem." Facebooks services run in containers sure, and they use a btrfs file system, but until you can show me how this file system is "exabyte-scale" then btrfs cant hold a candle to Tectonic (backed by xfs as explained in that document).

For future readers who make it this far, here is the document again: https://www.usenix.org/system/files/fast21-pan.pdf

First and foremost though, please define "regular user".

[RahulSundaram]

The very first line in the document "Tectonic is Facebook’s exabyte-scale distributed filesystem"

Sure and that is not the part that is being disputed here. The whitepaper you linked only claims that Tectonic is capable of supporting large storage, it doesn't offer any comparative usage metrics to Btrfs within the organization and therefore doesn't support your earlier claim of it being their largest use case. Large capacity != large use case. I don't think any of this is a word definition issue so let's not play that game.

[zexelon]

Sure and that is not the part that is being disputed here. The whitepaper you linked only claims that Tectonic is capable of supporting large storage, it doesn't offer any comparative usage metrics to Btrfs within the organization and therefore doesn't support your earlier claim of it being their largest use case. Large capacity != large use case. I don't think any of this is a word definition issue so let's not play that game.

Then you would suggest we are not debating and only arguing? If that is the case lets end this I have more valuable ways to spend my time.

If you can show me any usage metrics for btrfs in Facebook that would also help your case. Put your stats where your words currently are. I have provided much more detailed information and back up documentation in this discussion. I was under the impression it was a debate, but since it appears to only be an argument lets not continue.

If you wish to continue, define "regular user" if you do not define this it shall be assumed you only wish to argue. If you wish to continue find some Facebook usage stats for btrfs.

Otherwise I yet again state (refer to post Linux Gaming again)... stop using Meta/FB as justification for using btrfs. Their usage of btrfs is not at all comparable.

[zexelon]

SystemD should be in the top 3.

Oh darn I totally blanked on SystemD! Thats a good one! I have updated my list to include 3.5 now

[King InuYasha]

Do you have anything that actually supports your container view? So far all you've given is a link to a cluster filesystem running on top of XFS for shared data. That doesn't prove or disprove what they're using as their root file system nor does it prove or disprove your container position. Who is to say that they're not using BTRFS for root and XFS+Tectonic for shared data drives? That seems to be either already be the case or one of their goals based on all the data I've seen.

Last year Phoronix reported on Facebook using CentOS for their servers and Fedora for their desktops and BTRFS root was one of the deciding factors in regards to Fedora. Facebook is also part of the CentOS Hyperscale SIG that maintains a BTRFS enabled kernel, COW enabled DNF, and other things of that nature. They're literally backing BTRFS on CentOS. Are they backing a BTRFS root? Hell if I know. All the stuff I come across in regards to their CentOS SIG and BTRFS is generic, nothing root specific. All the root stuff is around Fedora...but it isn't a stretch to think they'd want all the BTRFS features like snapshots and whatnot for their CentOS servers.

I wish I gave enough of a shit to run their either their GNOME or KDE installer long enough to see if they offer BTRFS as a root file system for CentOS Stream (since BTRFS isn't an officially supported CentOS file system). Granted those are nearly a year old so who knows how much has changed in the meantime or even if BTRFS root was one of their goals with those releases. As a curious person it'd be nice to see those ISOs get some updates and for the Hyperscale SIG to clarify their stance on BTRFS root.

As one of the contributors of the CentOS Hyperscale SIG (particularly the one that put together those images), I can exclusively (note sarcasm) confirm that we do offer Btrfs-on-root by default. I backported all the enhancements from Fedora and re-enabled all those capabilities for CentOS for those spins. It's basically required for CoW-enabled-DNF, for example.

While I don't work for Meta, I work with them in the SIG and help build solutions to demonstrate the technologies they are developing and providing for the larger community.

As for why the ISOs haven't received updates in a year, I've been working on porting everything over to CentOS Stream 9 and retooling so we can build them in a more automated fashion. To build those ISOs for CentOS Stream 8, I have to build them manually in a specially crafted environment. It sucks and I hate it. I'm trying to do it better with CentOS Stream 9.

If you're interested in learning more about what we're doing in the Hyperscale SIG, feel free to come talk to us!

[zexelon]

As one of the contributors of the CentOS Hyperscale SIG (particularly the one that put together those images), I can exclusively (note sarcasm) confirm that we do offer Btrfs-on-root by default. I backported all the enhancements from Fedora and re-enabled all those capabilities for CentOS for those spins. It's basically required for CoW-enabled-DNF, for example.

While I don't work for Meta, I work with them in the SIG and help build solutions to demonstrate the technologies they are developing and providing for the larger community.

As for why the ISOs haven't received updates in a year, I've been working on porting everything over to CentOS Stream 9 and retooling so we can build them in a more automated fashion. To build those ISOs for CentOS Stream 8, I have to build them manually in a specially crafted environment. It sucks and I hate it. I'm trying to do it better with CentOS Stream 9.

If you're interested in learning more about what we're doing in the Hyperscale SIG, feel free to come talk to us!

Thanks for chiming in! This is very interesting, I may have to spool up CentOS with btrfs on one of our servers here and see how it holds up.

[Danny3]

Poke the maintainer? Or bump it yourself?
I mean, if the interface is clean and the code doesn't do any revolutionary new stuff, then it should be an easy port, right?

I don't know to do either.

I don't know who the maintainer is and definitely can't bump it myself.

[King InuYasha]

I don't know to do either.

I don't know who the maintainer is and definitely can't bump it myself.

Just send an email to the linux-btrfs mailing list and ask for someone to upgrade zstd in the kernel.

[zilexa]

Using MergerFS on BTRFS makes little sense -- MergerFS is a potential source of corruptions which underlying BTRFS would not be able to fix.

Theoretically everything is a potential source of corruptions. What's your point?
For home users 9 out of 10 times a raid solution makes no sense. Main goal is to unionize drives. Makes no sense to use raid for that while you can simply do it on user level, with MergerFS. Much cheaper and easier solution. It does not introduce extra risks.

[coder]

They should never have any corporate code, business intelligence or the like on their local file system. If you are any mid-sized or larger company and allow this, you are doing it wrong... very very wrong regardless of the file system your employee is using.

You can use drive encryption. Git lets you work offline, so you can take your laptop somewhere that might not have good wifi or cell coverage.

As far as what root filesystem the employee is using, there are some admin-related reasons why BTRFS might be nice. You can make snapshots before installing updates, for instance. Makes it easy to rollback, if something breaks. You can also check the filesystem to ensure nothing is corrupted. Along with validating package versions, this is needed to ensure build consistency across machines.