Linux 5.16.5 Released To Fix Up Btrfs' Botched Up Defragging

[Anux]

Mind that you never had problems with btrfs but you are using the vocabulary like "broken raid 5/6", "corrupting files", etc.

I got that from reading news not from my experience and even if i take all news with a grain of salt, most don't.

MD-raid had the exact same raid write hole issue until about 2017-2018 (and yes, people were losing data), yet somehow all the dramatization about write hole problem were always related to btrfs, never to MD-raid.

My understanding of write hole is, if you lost power on a non battery backed softraid you potentially loose data and mdadm has a journal and a write intend bitmap to mitigate that problem but its impossible to guarantee no data loss. If its the same situation for BTRFS than the reporting seems actually botched and I wonder why no one reported about BTRFSs mitigation technics and compared them to mdadm.

Overall I'm considering the btrfs drama similar to systemd: both are technically superior solutions to what Linux had before, yet people tend to get very dramatic around them.

Most people are fine with their solution and don't need the technical superiority (I highly doubt btw) because their needs are allready satisfied with existing "inferior" solutions. People have a fear of change.

[gfunk]

1. who's actually defragging an SSD?
2. should I defrag my SSD?

[Anux]

Rust-implemented filesystem - when?

https://blog.carlosgaldino.com/writi...h-in-rust.html there you go

[mdedetrich]

That bug is from already closed and is an older one. But it was big in the news back then.
The point is: how can you be sure you're not hit by a bug in the future given they already happened. The point is you can make a bucket list of reasons you won't touch filesystem XYZ based on fixed bugs no matter what filesystem you choose. And you cannot be sure that zfs will not get issues like this one in future. Period.

You posted a bug that was predominantly a Linux specific issue and try to pass that off as ZFS being as bad as btrfs? Nice try.

And yes there is a reason why ZFS is fundamentally better than BTRF's and that is due to its origins. ZFS was created by some of the most technically competent programmers at Sun to be an ultimate filesystem and because of this great effort was made into making sure the original design was sound and proper.

BTRFs on the other hand was originally created because Linux didn't have a CoW filesystem that they could use without possible legal issues.

You see the difference here? One filesystem was created to solve a technical problem where as the other one was created to solve a legal one, the former almost always results in better software. For the same reasons, a lot of features of BTRF's and BTRF's itself was rushed. There is a whole history to this but as an example the reason why the infamous BTRF's RAID 5/6 write hole hasn't been solved yet is because they didn't even consider proper support for RAID 5/6 it in the original on disk format. RAID5/6 was just tacked on like a marketing gimmick to try and get people to adopt BTRFS and compete against ZFS. It got so bad that at one point it really pissed off Linus as to why untested and beta like software was being pushed into the Linux kernel.

So to conclude, I am not claiming that ZFS is without bugs (thats absurd). What I am however claiming is that ZFS (especially on FreeBSD) has a proven track record along with correct priorities and none of these bugs which can be classified as massively critical which BTRF's has/had.

[dfyt]

I have run BTRFS on and off over the last couple years and now as my root and home for the last 3 months. You can say what you want about it being tested for power failures etc. I have them regularly. Around 5x now my KDE toolbar gets stuffed up during a power failure where the desktop "resets" to defaults, plus having Steam games fail validation to the point I am SOOO tempted to go back to XFS. In 8 years never not once had any issues XFS and power failures - same with ZFS. IMO the ONLY reason I use BTRFS is for it's zstd compression, heck even dupremove doesn't work properly on btrfs. I would not recommend BTRFS for anything valuable and ensure you have a FULL system backup.

Not a day after my son installed XeroLinux we had a power failure - same issues desktop reset. He mentioned Manjaro never gave me that (we had XFS back then). Not to mention the major performance issues between distros with BTRFS. It's a very hit and miss scenario.

I can take PopOS 5.15K btrfs and Arch 5.15K btrfs copy the SAME folder and get almost 5 mins on the one and 40 seconds on the other. Use XFS and you will get virtually the same from an old Ubuntu 18.04 vs now.

The moment XFS supports zstd compression BTRFS is gone.

[oleid]

You posted a bug that was predominantly a Linux specific issue and try to pass that off as ZFS being as bad as btrfs? Nice try.

Well, I don't care why the filesystem is broken if it is broken on my system, right?

Another nice one: https://utcc.utoronto.ca/~cks/space/...amageIsForever

The layers you need to add to wrap a filesystem to make it available on a platform are also part of the filesystem, from a users point of view.

And yes there is a reason why ZFS is fundamentally better than BTRF's and that is due to its origins. ZFS was created by some of the most technically competent programmers at Sun to be an ultimate filesystem and because of this great effort was made into making sure the original design was sound and proper.

You should consider a job in the ad industry.

BTRFs on the other hand was originally created because Linux didn't have a CoW filesystem that they could use without possible legal issues.

Solaris not having a COW filesystem was the very same reason ZFS was created.
By the time Btrfs was started, ZFS was Solaris-only.

Also, to quote https://lwn.net/Articles/342892/

At the same time that Chris was figuring out the technical design of btrfs, he was also figuring out how to fund the
development of btrfs in both the short and the long term. Chris had recently moved from SUSE to a special Linux
group at Oracle, one that employs several high-level Linux storage developers, including Martin K. Petersen,
Zach Brown, and Jens Axboe.

Doesn't sound like incompetent noobs invented the thing.
To quote https://lwn.net/Articles/342892/ further:

When ZFS first got started, the outlook for file systems in Solaris was rather dim as well. Logging UFS was already nearing the end of its rope in terms of file system size and performance. UFS was so far behind that many Solaris customers paid substantial sums of money to Veritas to run VxFS instead. Solaris needed a new file system, and it needed it soon.

Jeff Bonwick decided to solve the problem and started the ZFS project inside Sun.
His organizing metaphor was that of the virtual memory subsystem - why can't disk be as easy to administer and use as memory? The central on-disk data structure was the slab - a chunk of disk divided up into the same size blocks, like that in the SLAB kernel memory allocator, which he also created. Instead of extents, ZFS would use one block pointer per block, but each object would use a different block size - e.g., 512 bytes, or 128KB - depending on the size of the object. Block addresses would be translated through a virtual-memory-like mechanism, so that blocks could be relocated without the knowledge of upper layers. All file system data and metadata would be kept in objects. And all changes to the file system would be described in terms of changes to objects, which would be written in a copy-on-write fashion.

In summary, btrfs organizes everything on disk into a btree of extents containing items and data. ZFS organizes everything on disk into a tree of block pointers, with different block sizes depending on the object size. btrfs checksums and reference-counts extents, ZFS checksums and reference-counts variable-sized blocks. Both file systems write out changes to disk using copy-on-write - extents or blocks in use are never overwritten in place, they are always copied somewhere else first.

So, while the feature list of the two file systems looks quite similar, the implementations are completely different. It's a bit like convergent evolution between marsupials and placental mammals - a marsupial mouse and a placental mouse look nearly identical on the outside, but their internal implementations are quite a bit different!

In my opinion, the basic architecture of btrfs is more suitable to storage than that of ZFS. One of the major problems with the ZFS approach - "slabs" of blocks of a particular size - is fragmentation. Each object can contain blocks of only one size, and each slab can only contain blocks of one size. You can easily end up with, for example, a file of 64K blocks that needs to grow one more block, but no 64K blocks are available, even if the file system is full off nearly empty slabs of 512 byte blocks, 4K blocks, 128K blocks, etc. To solve this problem, we (the ZFS developers) invented ways to create big blocks out of little blocks ("gang blocks") and other unpleasant workarounds. In our defense, at the time btrees and extents seemed fundamentally incompatible with copy-on-write, and the virtual memory metaphor served us well in many other respects.

In contrast, the items-in-a-btree approach is extremely space efficient and flexible. Defragmentation is an ongoing process - repacking the items efficiently is part of the normal code path preparing extents to be written to disk. Doing checksums, reference counting, and other assorted metadata busy-work on a per-extent basis reduces overhead and makes new features (such as fast reverse mapping from an extent to everything that references it) possible.

You see the difference here?

Not really. Both filesystems were created because there was no COW filesystem for that platform.
The technical approach of Btrfs seems to be more appropriate to filesystems.

So to conclude, I am not claiming that ZFS is without bugs (thats absurd). What I am however claiming is that ZFS (especially on FreeBSD) has a proven track record along with correct priorities and none of these bugs which can be classified as massively critical which BTRF's has/had.

Well, the main reason you don't find the original bugs of ZFS is that it is a long time ago before it was even ported to FreeBSD and Linux and all those issue reports were interal issues in SUN's bug database made by paying customers. Even the old issue trackers of opensolaris are archived and hard dig into, e.g. here https://illumos.org/opensolaris/bugdb/bug.html#!6554564

[pal666]

With most kernel subsystems it's best to wait until the .10 stable release really

it's best to stick to sane distro and its kernel. fedora currently has 5.15.18(and it doesn't force autodefrag btw). but i wouldn't be surprised if all those crazy zol people extend their poor judgement to distro/kernel selection too and are affected by this bug

[pal666]

What I am however claiming is that ZFS (especially on FreeBSD) has a proven track record

track record of 1.5 users is of little interest

[pal666]

And yes there is a reason why ZFS is fundamentally better than BTRF's and that is due to its origins. ZFS was created by some of the most technically competent programmers at Sun to be an ultimate filesystem and because of this great effort was made into making sure the original design was sound and proper.

it's a reason zfs cult lives in imaginary world. zfs was created before invention of cow-capable btrees and its design was suboptimal and obsolete, according to zfs devs in 2009. and it was about solaris zfs, linux zfs is an abomination with converter from linux to solaris written by some people from internet

BTRFs on the other hand was originally created because Linux didn't have a CoW filesystem that they could use without possible legal issues.

btrfs was created by oracle employee because cow-capable btrees were invented(he participated in development of non-cow btree fs before). oracle can't have legal issues with oracle zfs, the more excuses you post, the more stupid you look

[pal666]

1. who's actually defragging an SSD?

probably same people who use zol

2. should I defrag my SSD?

no