Btrfs Async Buffered Writes Slated For Linux 6.1 - 2x Throughput Improvement

[_ReD_]

Honestly I am just enjoying the back and forth aggressiveness that so wonderfully permeates the Linux community.

So, just a troll.
Ok then, as long as you admit it...

I have learned a TON in this thread about Facebooks internal systems, so that's been fun!

All is well that ends well. Good for you.

[zexelon]

So, just a troll.
Ok then, as long as you admit it...

Hmmm I would not identify typically as a troll, when the conversation started it was more about the idea that Facebook is not a great example use case for btrfs for the "regular user", but yah I will fully admit by around post 60 when the "Linux Community Spirit" fully engaged that it turned into trolling... I mean second only to some southern baptist religions perhaps, there is nothing like the "Linux Community" for faith based zealotry.

I am sure we will cross paths in some future Nvidia thread also...

[binarybanana]

Yeah, about that... that's a pain point right there!
BTRFS sorely needs a "fix in place" tool that works.

Having to "btrfs restore*" for every small problem is far from optimal. (* which implies: restore to another disk, reformat, restore again... and still deal with the aftermath.)
Conversely, a quick pass of an fsck-alike, which tends to consistently do the right thing (as on ext4) would be much more convenient, in a pinch.

Agreed. If btrfs restore can make sense of what's written on disk an fsck.btrfs-like tool should be able to fix things. However, the things I listed are far from "small problems". I never had any issues with random crashes, sudden power loss, etc. over the years. But yeah, it can be problematic on hardware that lies about write barriers/flushing.

[swoorup]

I corrupted all my files, when simply resizing EXT4 using GParted. Was disappointed and wasted quarter of the day reinstalling and configuring my system....

[_ReD_]

However, the things I listed are far from "small problems". I never had any issues with random crashes, sudden power loss, etc. over the years. But yeah, it can be problematic on hardware that lies about write barriers/flushing.

True, it's been my experience as well.
In the last few years, even with [nobarrier,commit=600 & eatmydata libs] —which is a no-no for data integrity, yet makes things fly on slow media— the impact of random-crashes/sudden-power-loss/yanking-of-devices has consistently been negligible.

In my rant though, I had in mind the "double mount" scenario you cited, which bits me every f. time I forget to unmount or to disable the f. "automount" (eg. on someone else's computer) and boot a VM straight from (the now auto-mounted) removable media.

[skeevy420]

Note that you speak specifically about raid5 and not btrfs raid, there are close to zero work being done on raid5 due to there being close to zero interest from the users to use raid5. BTRFS with raid1 (and10) has been stable for decades and is also where the large use base is.



The zstd maintainer in the Linux Kernel is Nick Terrell terrelln@fb.com who AFAIK is also the zstd dev upstream at FaceBook. So that he have still not pushed a newer version of ztstd to the kernel (the zstd code is not for btrfs alone) looks like FB themselves not seeing 1.5 as being stable enough yet for inclusion.

Coincidentally, that's his Phoronix username

[reza]

I use BTRFS on individual drives with duplicate metadata, and unionize the drives with MergerFS.

How do you use BTRFS with mergerfs? Can you still take a "btrfs subvol snapshot" from it?
(I use btrfs and layer an overlayfs on a snapshot, but when I want to create btrfs subvol snap, it doesn't allow it claiming it's not a BTRFS fs.

[Developer12]

This is 2nd-hand, as it came by way of a report regarding defective RAID cards, but Facebook indeed doesn't use BTRFS for raid or any kind of redundancy of any kind.

They use cheap raid cards to produce a volume that is resistant to drive failure and bit errors, and then only layer BTRFS on top persumably for snapshot capability.
They are placing *zero* faith in BRTF's raid (any of them, including raid10) and zero faith in it not exploding the moment it gets metadata corruption in the wrong spot.

That is still the long-standing issue with BTRFS, irrespective of RAID5/6. Get a bit error in the wrong spot or a drive failure at the wrong time and there's a good *chance* your array will survive. But there are new horror stories every *month* from people who were not so lucky. While this has improved over the years, it's still a common and persistent problem, masked by the reliability and error checking built into modern HDDs and SSDs.

This is, incidentally, among the kinds of faults injected into ZFS during continuous testing. ZFS can also boast a FAR better track record in terms of reliability. For example as per [1] during all the years Fishworks supported clients using ZFS they lost client data in exactly *ONE* incident, when one of their own support team edited kernel data structures on a running system. Not a single other time.

[1] https://youtu.be/WsvJT6i_atw?t=3804

[zilexa]

How do you use BTRFS with mergerfs? Can you still take a "btrfs subvol snapshot" from it?
(I use btrfs and layer an overlayfs on a snapshot, but when I want to create btrfs subvol snap, it doesn't allow it claiming it's not a BTRFS fs.

MergerFS runs in userspace. It is unaware and independent from the underlying filesystem. So you just run it.

I don't understand what you mean, of course I can still snapshot subvolumes. It has absolutely nothing to do with MergerFS. It's like asking can I still copy files to a usb stick or vice versa.

Read MergerFS manual.
Feel free to also read step 2a and 2b of my guide:
https://github.com/zilexa/Homeserver...pools-creation

But you don't need anything special to run MergerFS on any particular filesystem, nor does MergerFS needs to be configured specifically for a certain filesystem. It's completely independent of the underlying filesystem.

[stevea]

Sorta off topic - but I'd REALLY like to see some new filesystem benchmarks, esp btrfs.