XFS Working Towards Online Repair, Many Underlying Improvements

[starshipeleven]

APFS:

It's not designed for real data use but for single-drive systems (aka MacBook and iMac, no surprises) so it's very very meh. I would say it is Apple's equivalent of XFS, a traditional filesystem with some spice added.

BTRFS:

your opinion is bullshit, btrfs is enterprise-grade but apparently there is very low interest in RAID5/6 development since large parts of that usecase has been taken over by cluster filesystems or has its niche filled already by the usual suspects mentioned below.

HAMMER/HAMMER2:

more bullshit, this filesystem is a network/cluster/whatever filesystem. Linux has like 3-4 similar filesystems that are already in production and work fine. That's why there is zero interest in porting it.

BCACHEFS: I really like this also, I think this FS has had time to settle and really has the potential to be a replacement to BTRFS and be a great desktop filesystem. Perhaps more..

You write this bs only because you hate btrfs, you have no idea of the code quality, or how "enterprise grade" it is. Or even how usable is right now (hint: not much).

ReFS/WinFS/???:

It's trash, but it's better than NTFS in some specific cases for servers. It loses to btrfs, and loses badly to ZFS.

XFS+STRATUS:

Meh, it's at least a way to use and configure all the stuff offered by Linux in a sane way, there is A LOT of stuff that is a massive pain to operate together.

In the enterprise nobody cares what filesystem your server or desktop uses.. they care about large storage platforms and so ZFS on the enterprise level does not really compete with BTRFS or XFS. It competes with NetApp's WAFL and Dell EMC.

Technically correct, while the filesystem is capable, there is no appliance using it, and I strongly suspect none is giving a shit about getting into that market anyway (see above about RAID5/6)

ZFS is a free open source alternative to those proprietary and EXPENSIVE black box systems

appliances using ZFS at similar scales, with enterprise support aren't particularly cheap either.

As open source advocates it makes a lot of sense to have something to compete with them.

Maybe? I mean who is making their own appliances at this scale, why should I care about competition in a VERY enterprise market doing stuff most people don't really need.

ZFS isn't a threat from some long dead company

Correct, Oracle isn't long dead.

[k1e0x]

It's not designed for real data use but for single-drive systems (aka MacBook and iMac, no surprises) so it's very very meh. I would say it is Apple's equivalent of XFS, a traditional filesystem with some spice added.

your opinion is bullshit, btrfs is enterprise-grade but apparently there is very low interest in RAID5/6 development since large parts of that usecase has been taken over by cluster filesystems or has its niche filled already by the usual suspects mentioned below.

more bullshit, this filesystem is a network/cluster/whatever filesystem. Linux has like 3-4 similar filesystems that are already in production and work fine. That's why there is zero interest in porting it.

You write this bs only because you hate btrfs, you have no idea of the code quality, or how "enterprise grade" it is. Or even how usable is right now (hint: not much).

It's trash, but it's better than NTFS in some specific cases for servers. It loses to btrfs, and loses badly to ZFS.

Meh, it's at least a way to use and configure all the stuff offered by Linux in a sane way, there is A LOT of stuff that is a massive pain to operate together.

Technically correct, while the filesystem is capable, there is no appliance using it, and I strongly suspect none is giving a shit about getting into that market anyway (see above about RAID5/6)

appliances using ZFS at similar scales, with enterprise support aren't particularly cheap either.

Maybe? I mean who is making their own appliances at this scale, why should I care about competition in a VERY enterprise market doing stuff most people don't really need.

Correct, Oracle isn't long dead.

My point here is ZFS changed the landscape and what people thought about file systems. For example, APFS is cow, snapshots, clones, compression, some dedup abilities. It's clearly inspired by ZFS tho it's lacking the volume management and checksums. And go read the Stratus whitepaper. They lament they can't use ZFS in it.. well that sure sucks for RedHat but *you* aren't bound by that limitation.

Oracle ZFS in my book isn't any different than NetApp. It's a closed source black box. They are the 3rd or 4th largest storage vendor next to NetApp, EMC, and DDN. We can classify them all together.

Raid 5/6 are depreciated even if people wanted to use them. due to the size of the disks now and un-correctable errors occurring on re silver. (6 is ok for home use) You need triple parity at a minimum. Raid 7-3. Let me flush this out a little more so you have raid 5, if you lose a disk during resilver an un-correctable error means you are toast. With drive sizes at 12~++TB now resilver takes some time and puts strain on the disks. resilver times can be in upwards of days for some arrays. in 6's case if you have an un-correctable error in the same place *and/or* another failure during resilver you're toast. so... neither of those are acceptable risk for enterprise anymore. Triple parity is a minimum now. I suppose if 5 and 6 don't work for BTRFS 7 doesn't either but.. idk.. doesn't matter. . nobody is using it so. Best wishes to them but.. ya.

God I hate how people think they need an appliance or a cloud service to do anything in IT anymore. You don't. Take a rack load it up with FreeBSD ZFS and Gluster and you have a multi petabyte storage array. What is more open source? ZFS or NetApp? That is what we are talking about here. The minutia of the CDDL is irreverent because I can actually READ and change ZFS code.

[SystemCrasher]

Clearly the "imperfect" in your dictionary is way below the common standard.
A RAID 5 implementation that occasionally eats up your data is not imperfect, it's JUNK.

You see, after years of fiddling with various stuff I got idea I don't believe in "should never happen" thing, to begin with. Universe is complicated, it always manages to provide. Where you think it should run smoothly, you can have it other way for arcane reasons nobody foreseen. I've seen quite arcane FS faults. ZFS included. So well, things could fail. If that really matters, there should be plan B. And if that fails and it still matters, you better off with plan C. And whatever else it takes.

1) A lazy ZFS user would pick FreeNAS or FreeBSD instead of believing Canonical commercials

This implies I can't even enjoy by advanced techs on my workstation or laptop - so your techs fail to serve my goals, to begin with. FreeNAS isn't desktop. FreeBSD as desktop? Nope, thanks. I had enough on servers and been more than happy to get rid of it. At the end of day, FreeBSD is crappy desktop, to extent last company scrapped efforts as hopeless. Sorry, I like Linux and going to use Linux. It does the trick. Even as desktops, embedded systems and so on. I'm enjoying it, it works for me - for both fun and profit. Esp compared to windows and macos that gone vendorlocked spyware/malware ways, to put it straight. I like daring experiments, tinkering, DIY, trying new ways/approaches and so on - including fiddling with low level things on the edge of HW and SW. Proprietary systems are extremely hostile to this kind of activity, I had enough of vendorlocks.

2) See 1)
3) See 1)
4) See 1)

And so it fails to deal with my use cases. Four times in a row. Cool.

5) (not interested in every advanced feature.)

This feature eventually boosts my workflows speed hell a lot and opens some new venues. I appreciate it. As simple as that. It like "hinted dedup" where I can do full dedup cheap-n-fast, without paying online dedup price. Be it new VM respin or building OS image for embedded, CoW-copy of that allows to fiddle with plenty of "versions" of it cheap-n-fast, saving time and space without enjoying costs of "real" online/offline dedups. Personally I think its very neat and logical for CoW semantics.

6) I DO want enterprise grade heavy truck for my dedicated NAS server, and I'm happy with naive ext4 on my workstation's SSD. "versatile" here, imo, is a common PR word that means "good at nothing".

For me world doesn't ends on NAS or servers. So I need something sane for my workstation or laptop, embedded systems and more. Somehow btrfs shown it flexible enough to fit the bill in many different occasions. Bringing its benefits ofc.

"versatile" here, imo, is a common PR word that means "good at nothing".

Hum, let us just assume I'm not all about NASes unlike you, and strongly prefer to use Linux as OS. Therefore we seem to have wildly different priorities and goals. So your approaches are unlikely to work for me.

[SystemCrasher]

Just to make sure it's clear.. The internal de-dupe is by a flash controller, not all SSD(and unlikely in USB sticks afaik) are going to do this optimization, but afaik there isn't a way of knowing, it's not something you see in marketing or documented for products, so if you want to be on the safe side, I would be cautious with relying on DUP single disk SSD with BTRFS, like the wiki advises you about.

I know all that. Furthermore I know storages and their firmwares could be "strange" so dodging all possible failure modes is something like "mission impossible". Yet, I gave DUP a try and it dodged quite some bullets under various circumstances.

And if someone cares about data integrity they better to avoid such HW, because:
1) Any imperfection "amplifies" and hits much more data than you would expect otherwise.
2) You have control over FS-level (de)duplication but have no control over FW attitude. At which point you can only wait and see if this feature would backstab. Or not.
3) Such feature ensures data recovery would be far more costly or even impossible if you fail to find lab that reversed all this cool stuff and can do that "manually". So if things turn badly and you had no backups...

That's unrelated to BTRFS de-dupe feature or whatever third-party solution.

Yes, you're about HW/FW level behavior. And I'm well aware some HW can shot your legs in truly arcane ways. Still not a reason to make universal overlygeneric conclusions.

BTW, with SSDs these days they have pretty decent endurance that you don't need to worry about wear from writes.

Oh lol, esp cheap TLC crap, barely surviving like maybe 100-200 write cycles... :P.

I generally see BTRFS as more useful for archive/backup or system data(personal data should be getting backups anyway if valuable).

I had good results using that as rootfs as well. It could improve systems reliability. It could give early warning of incoming problems before they actually hit at their full. For me it dodged nearly all bullets. Somehow I like this.

XFS has been fine for me, it's experienced multiple power cuts or kernel panics without data corruption.

I had at least the following isues with it:
1) Slow metadata performance under some use cases I care, to extent it can take minutes to remove some hierachies, etc. Btrfs isn't fastest but I never stumbled on pathological cases like that "in the wild".
2) Inconsistent files state on crash, it does no full journal. Well, not sure if one can achieve something like that with XFS+cow. But I've recently had few files screwed by XFS. So I would take its data integrity mumbles with grain of salt for now.
3) Damaged file content, like tail full of zeroes.

Btrfs designed the way it rather unlikely to face files with damaged content, CoW on data+metadata implies recovery could be like ignoring some few new changes to both data and metadata, as they both CoW, slightly older state remains - and the only disadvantage is a bit older state of things, but it still consistent. On crash things would be in either "new" or "old" state, but I never seen files in intermediate/inconsistent/illegal states like it eventually happens with XFS. It plagued XFS for many years - and I still had that like in 5.4 kernel or so. Wasn't 5.4 recent enough to get rid of this crap behavior already?

suggests I'll run into issues from CoW/BTRFS if I don't plan properly, especially since I want to use those advanced features which don't always play well with each other(defrag with de-dupe and compression I think is one example).

Furthermore, I can imagine with advances of XFS heading comparable direction it can get comparable set of "habits" either. CoW is quite a departure from "classic" approaches - and everyone who implements it and uses its benefits have to live with its habits. These habits could be "unusual" - but are logical enough after all.

more bullshit, this filesystem is a network/cluster/whatever filesystem. Linux has like 3-4 similar filesystems that are already in production and work fine. That's why there is zero interest in porting it.

Hum? Are you sure on this? HammerFS is CoW filesystem of DragonFly BSD and I dont remember it being advertised as "networked". Nearly the only CoW design any BSD devs managed to code on their own ever. Though your point it virtually untested likely holds: DF BSD is anything but popular, while e.g. btrfs runs on facebook servers. Guess it gives very decent idea about real world usage and resulting "test coverage".