Btrfs RAID 5/6 Support Is "Mostly OK" With Linux 4.12

[caligula]

Sun didn't have to deal with a design supposed to support as much features as btrfs, so they could avoid many annoyances. Really, there would be no point in making something that is not significantly better than ZFS, so the development time being longer shouldn't be so strange.

The guy's also totally ignoring the fact that Sun worked for years on these later features: https://en.wikipedia.org/wiki/ZFS#De...elease_history
There's around 5 years of development time between the oldest and latest zpool version. Let's see... for instance, zpool ver3 introduced RAID6. It wasn't there from the start. How is that possible?!

[gbcox]

The guy's also totally ignoring the fact that Sun worked for years on these later features: https://en.wikipedia.org/wiki/ZFS#De...elease_history
There's around 5 years of development time between the oldest and latest zpool version. Let's see... for instance, zpool ver3 introduced RAID6. It wasn't there from the start. How is that possible?!

That wasn't Cantrill's point, and he is quite familiar with ZFS development.

[gbcox]

And base this on exactly 0 evidence.

Never say these words, they usually attract Chtulhu.

If people can't fucking read and understand a single wiki page with all features then they deserve what they get. btrfs won't be the first nor the last that blows up in their hands.

That "unstable" warning in red near the btrfs RAID5/6 field a few lines below might still catch enough attention.
Really we are talking about a wiki of a filesystem aimed squarely at veteran linux users.

So you ran a filesystem clearly marked as unstable, without a file checker and without scrubbing? And then complain that you just found out it had these fundamental issues last year? I really hope you aren't in charge of anything at your job.

So, you jumped from a hype train to the next, with basically nothing to prove his claims apart from the fact that you think development is too slow.

Sun didn't have to deal with a design supposed to support as much features as btrfs, so they could avoid many annoyances. Really, there would be no point in making something that is not significantly better than ZFS, so the development time being longer shouldn't be so strange.

That wasn't Cantrill's point; nor was it Overstreet's. Like I said, if you like it, that's fine. Use it. As far as running BTRFS I knew exactly that it was experimental - but I expected at some time they would go production. Then the parity issue happened and it was clear it wasn't going into production any time soon. I decided to cut bait.

[Teaspoon]

Every time I see mdraid mentioned in one of these discussions, the same "but if the filesystem doesn't manage the raid itself then it can't self-heal" argument comes up. Could that be changed, though? The XFS tools, for example, are mdraid-aware and can detect the raid geometry to make sensible decisions automagically. What if one of the checksumming filesystems could be mdraid-aware in a way that let them request alternative versions of a block when the checksum doesn't check out?

It seems crazy for every new filesystem to include its own half-baked raid implementation when something as mature and capable as mdraid is already available.

[finite9]

It's because of reliability of the hardware itself. SAS disk are, supposedly, made out of more reliable components (which is not true most of the time, but shh it's marketing voodoo ).

On a protocol level SAS drives are reporting errors immediately to the controller while most consumer-grade SATA drives have a variable report time or don't report them at all (!). This is called SCT ERC - SMART Command Transport Error Recovery Control. You can check if your drive supports it with smartctl -a | grep SCT.

I just migrated an md-raid1 to btrfs raid1. I had previously been burnt by using consumer grade equipment with raid arrays (2-port pcmcia sata controller and wd MyBook drives...<shudder>... rebuilds every single week), so I bought fancy pants WD RE3 drives which had SCT ERC. They are still running solidly without a single error after 9 years in service. Then a few years after that I bought some Seagate Constellation ES enterprise class drives, that were 4k devices. They are also still running strong with zero errors after 6 ish years. These drives, although enterprise class, _do not have_ SCT ERC. These drives usually cost double the amount of a consumer drive which is super annoying, because the manufacturer is basically charging a hefty premium for a firmware feature, that has nothing to do with build quality. So moving to consumer grade drives that are half the price, whilst still being able to "do" raid is an effective strategy in my eyes.

So now I have 2 newer Seagate Firecudas, that are nothing special, but have a 5 yr warranty, and I run btrfs raid1 on them.

My original reason for using md-raid was to be able to do in software, that which had previously required a more expensive hardware solution. Isn't btrfs a similar motivation *for everyone* that used md-raid for the same reason?? Because for me btrfs is a logical evolution of the same mindset from hardware raid to md-raid and now to btrfs raid.

[duby229]

And base this on exactly 0 evidence.

Never say these words, they usually attract Chtulhu.

If people can't fucking read and understand a single wiki page with all features then they deserve what they get. btrfs won't be the first nor the last that blows up in their hands.

That "unstable" warning in red near the btrfs RAID5/6 field a few lines below might still catch enough attention.
Really we are talking about a wiki of a filesystem aimed squarely at veteran linux users.

So you ran a filesystem clearly marked as unstable, without a file checker and without scrubbing? And then complain that you just found out it had these fundamental issues last year? I really hope you aren't in charge of anything at your job.

So, you jumped from a hype train to the next, with basically nothing to prove his claims apart from the fact that you think development is too slow.

Sun didn't have to deal with a design supposed to support as much features as btrfs, so they could avoid many annoyances. Really, there would be no point in making something that is not significantly better than ZFS, so the development time being longer shouldn't be so strange.

I just wanted to say the part I bolded is exactly what's wrong with btrfs. Exactly. You say those words like they are somehow a good thing, but the real truth is that's how they screwed themselves beyond the point of no repair.

[niner]

As I said above, I know of no one who uses this on production systems.

We do use btrfs on all our production servers (because snapshots) and for backups (because send/receive).

[caligula]

It seems crazy for every new filesystem to include its own half-baked raid implementation when something as mature and capable as mdraid is already available.

I had the impression that you could achieve faster perf at higher level. For instance on MD RAID-1, to be sure the data integrity is ok on both disks, you'd need to read each block from both disks and compare on every read. With Btrfs RAID-1 and checksumming, you can split the load 50%-50%, recalc the checksums and rely on periodic scrubs.

[torsionbar28]

These drives usually cost double the amount of a consumer drive which is super annoying, because the manufacturer is basically charging a hefty premium for a firmware feature, that has nothing to do with build quality.

You're wrong about this. In addition to lots of firmware differences, Nearline enterprise drives like the WD RE series have quite a few physical build quality differences. 1. The spindle is supported on both ends, where consumer drives it's only supported on one end. 2. The actuator magnets are much larger/stronger than consumer drives. 3. The platters have windage plates to reduce turbulence and therefore reduce vibration, consumer drives do not. 4. The drive motors are beefier, designed for 24/7 duty cycle, consumer drive motors are smaller and designed for a much lower duty cycle. 5. The media surface is more durable and designed for 550 TBW per year of writes, while consumer drives are rated for 180 TBW per year.

All this is for 7200 rpm nearline drives. When you step up to 10k or 15k online drives, the hardware differences are even more pronounced.

[numacross]

I just migrated an md-raid1 to btrfs raid1. I had previously been burnt by using consumer grade equipment with raid arrays (2-port pcmcia sata controller and wd MyBook drives...<shudder>... rebuilds every single week), so I bought fancy pants WD RE3 drives which had SCT ERC. They are still running solidly without a single error after 9 years in service. Then a few years after that I bought some Seagate Constellation ES enterprise class drives, that were 4k devices. They are also still running strong with zero errors after 6 ish years. These drives, although enterprise class, _do not have_ SCT ERC.

I think in case of the Constellation ES the price premium is mostly due to the esoteric self-encryption, which might be required in some uses. The lack of SCT ERC is unusual, but it might mean that the drive has a fixed short timer on error reporting, which is fine too. Without an exact model number to look up the specc sheet it's hard to tell.

These drives usually cost double the amount of a consumer drive which is super annoying, because the manufacturer is basically charging a hefty premium for a firmware feature, that has nothing to do with build quality. So moving to consumer grade drives that are half the price, whilst still being able to "do" raid is an effective strategy in my eyes.

Yeah, product differentiation like that is common since it's simpler to make one piece of hardware and price by features available than to make multiple separate things. IC manufacturers do this all the time (I'm looking at you Intel with your earlier VT-x/d bullshit).

My original reason for using md-raid was to be able to do in software, that which had previously required a more expensive hardware solution. Isn't btrfs a similar motivation *for everyone* that used md-raid for the same reason?? Because for me btrfs is a logical evolution of the same mindset from hardware raid to md-raid and now to btrfs raid.

Yes that is true and I agree. However I'm using ZFS more since being able to tell exactly how much space is left on the filesystem is a nice thing to have Don't get me wrong, I like btrfs' too, but it is lacking in a few aspects I require. I'm sure they'll get there eventually.