Btrfs Updates Sent In For The Linux 4.17 Kernel

[k1e0x]

What I have, and what Btrfs calls "RAID1", is that every bit of data is in 2 different devices. So any disk can fail or be removed without losing any data. And I can use 100% of my disks for that, without wasting most of them as ZFS would do.

And if I ever need more space I can, say, replace the 500 Gb disk with a 3 Tb one, and I'll get 1250 Gb of extra parity-backed space, with nothing going to waste.
I could even just add more SATA ports and add more disks of whatever size and it'll use them as well as possible.



For backups I have my RPi with two 1 Tb in a Btrfs RAID1. FreeNAS won't run there, and even if it could, I use that RPi for other things that FreeNAS would make cumbersome or impossible to do.

If you had a tank and I told you here use this car it's better because it has the feature of being faster.. would I be right? No because it depends what your using the tank for.

[k1e0x]

You also don't have a raid1, words matter and raid1 means a very specific thing. What you have is an array of logical volumes's in a mirror. I'm sorry its not your fault, its the designers fault.. but still.

Why does this matter? Lets say you told someone you had an array of 4 disk in raid 1. Most normal people would believe that to be 4 disks with the exact same data on each 4x parity because they would be going by the standard definition of that term, not the btrfs equivalency term for single parity array.

[ssorgatem]

You also don't have a raid1, words matter and raid1 means a very specific thing. What you have is an array of logical volumes's in a mirror. I'm sorry its not your fault, its the designers fault.. but still.

Why does this matter? Lets say you told someone you had an array of 4 disk in raid 1. Most normal people would believe that to be 4 disks with the exact same data on each 4x parity because they would be going by the standard definition of that term, not the btrfs equivalency term for single parity array.

That's why I say "btrfs RAID1".

Also, I don't know what do you mean with "array of logical volumes with a mirror". There's only one volume, across 4 devices; the all data extents are guaranteed to be in 2 devices at any given time, but there's no "mirroring".

[k1e0x]

That's what I say "btrfs RAID1".

Also, I don't know what do you mean with "array of logical volumes with a mirror". There's only one volume, across 4 devices; the all data extents are guaranteed to be in 2 devices at any given time, but there's no "mirroring".

Yeah.. i know.. its hard.. and you wonder why enterprise has trouble accepting btrfs. Linux people do this all the time btw, redefining things its highly annoying. KVM anyone?

[ssorgatem]

Also, name it what you want... but the fact remains that it's a very useful feature for non-rich home users that ZFS lacks altogether.

[k1e0x]

Also, name it what you want... but the fact remains that it's a very useful feature for non-rich home users that ZFS lacks altogether.

A name is NOT what you want it to be if you want to be understood by other people who do not know your technology. Think for fucks sake. If what your designing isn't raid1 don't call it raid1. It shows naivety by the designer thinking that WAS the same thing. I'm a sysadmin with 40 years computer experience.. I know *a bit* about storage.. and ya.. it really does seem like a cute home user file system. I wouldn't use it myself because however because usually when I write stuff to disk I want to get it back and your failure rates are exceptionally high in that configuration.. but your not gonna listen to shit so do your own thing and rock on man.

[ssorgatem]

A name is NOT what you want it to be if you want to be understood by other people who do not know your technology. Think for fucks sake. If what your designing isn't raid1 don't call it raid1. It shows naivety by the designer thinking that WAS the same thing. I'm a sysadmin with 40 years computer experience.. I know *a bit* about storage.. and ya.. it really does seem like a cute home user file system. I wouldn't use it myself because however because usually when I write stuff to disk I want to get it back and your failure rates are exceptionally high in that configuration.. but your not gonna listen to shit so do your own thing and rock on man.

lol

For someone that worried about the "proper" use of words, your spelling could surely use some looking into.

Well, here's the thing, this is the definition of "RAID1" on Wikipedia:

RAID 1 consists of data mirroring, without parity or striping. Data is written identically to two drives, thereby producing a "mirrored set" of drives. Thus, any read request can be serviced by any drive in the set. If a request is broadcast to every drive in the set, it can be serviced by the drive that accesses the data first (depending on its seek time and rotational latency), improving performance. Sustained read throughput, if the controller or software is optimized for it, approaches the sum of throughputs of every drive in the set, just as for RAID 0. Actual read throughput of most RAID 1 implementations is slower than the fastest drive. Write throughput is always slower because every drive must be updated, and the slowest drive limits the write performance. The array continues to operate as long as at least one drive is functioning.^[11]

Well, Btrfs' "RAID1" actually does that, data "is written identically to two drives". It just doesn't have the 2 drive limitation, and more drives can be added to the array without increasing the parity.

A RAID1 with more than 2 disks and data written to more than two drives is actually further away from the standard definition of RAID1 than Btrfs' RAID1 is.

Sure thing, it's not your regular RAID1 implementation, but that doesn't make it "actually not RAID1".

[ssorgatem]

Also... why would the failure rates be "exceptionally high"? lol

[waxhead]

Well, Btrfs' "RAID1" actually does that, data "is written identically to two drives". It just doesn't have the 2 drive limitation, and more drives can be added to the array without increasing the parity.

A RAID1 with more than 2 disks and data written to more than two drives is actually further away from the standard definition of RAID1 than Btrfs' RAID1 is.

Sure thing, it's not your regular RAID1 implementation, but that doesn't make it "actually not RAID1".

While BTRFS "RAID"1 resembles "normal" RAID1 it is not. I'll give a shot at explaining:

In BTRFS terms the same data is duplicated on two drives (in a 2 disk setup), but the layout of the data e.g. where it resides on the disk may be very different. In a three disk BTRFS "RAID"1 you will have still only have two copies of the data, in a four disk setup you will still only have two copies of your data on different drives.

But let's concentrate on a two disk setup.
Let's pretend that you HAD three disk in "RAID"1 and rebalanced it to a two disk "RAID"1. On a "normal" RAID1 you would expect the data to be mirrored, e.g. if you seek to block 123456789 on a disk A you will find the same data as you will have on disk B. For BTRFS this may be very different. While it probably is mostly the same if you start off with two disks you have no guarantee. A corruption for example may cause the data to be relocated elsewhere on the disk.

The main confusion is that while BTRFS "RAID"1 works very much like a "normal" RAID1 from a users perspective (duplicate the data), it will not MIRROR the data. Think of this as having two puzzles where one is assembled, and the other one is in a box. You still have the same pieces, and they make take up the same space, but their layout is very different.

BTRFS "RAID"1 also (until kernel 4.14 I think) required at least three storage devices to not enter a permanent read only mode. This because it had to be able to create two copies of the data on two different devices always.

Think of BTRFS different data/metadata profiles roughly like so:

SINGLE: Just like your average filesystem. No redundancy
DUP: Two copies of the data on the same device, makes it possible to survive / recover from corruption on a single device.
RAID1: Should really be called 2COPIES: Keeps two copies of the data on two different devices
RAID10: Should really be called 2COPIES mirrored: Keeps two copies of the data, each copy spread over half of the available storage devices
RAID5: Should really be called STRIPE + Single parity : Spread the data over all, but one of the available devices. Writes parity information on the leftover device.
RAID6: Should really be called STRIPE + Dual parity : Spread the data over all, but two of the available devices. Writes parity information on the two leftover devices

[ssorgatem]

While BTRFS "RAID"1 resembles "normal" RAID1 it is not. I'll give a shot at explaining:

In BTRFS terms the same data is duplicated on two drives (in a 2 disk setup), but the layout of the data e.g. where it resides on the disk may be very different. In a three disk BTRFS "RAID"1 you will have still only have two copies of the data, in a four disk setup you will still only have two copies of your data on different drives.

But let's concentrate on a two disk setup.
Let's pretend that you HAD three disk in "RAID"1 and rebalanced it to a two disk "RAID"1. On a "normal" RAID1 you would expect the data to be mirrored, e.g. if you seek to block 123456789 on a disk A you will find the same data as you will have on disk B. For BTRFS this may be very different. While it probably is mostly the same if you start off with two disks you have no guarantee. A corruption for example may cause the data to be relocated elsewhere on the disk.

The main confusion is that while BTRFS "RAID"1 works very much like a "normal" RAID1 from a users perspective (duplicate the data), it will not MIRROR the data. Think of this as having two puzzles where one is assembled, and the other one is in a box. You still have the same pieces, and they make take up the same space, but their layout is very different.

BTRFS "RAID"1 also (until kernel 4.14 I think) required at least three storage devices to not enter a permanent read only mode. This because it had to be able to create two copies of the data on two different devices always.

Think of BTRFS different data/metadata profiles roughly like so:

SINGLE: Just like your average filesystem. No redundancy
DUP: Two copies of the data on the same device, makes it possible to survive / recover from corruption on a single device.
RAID1: Should really be called 2COPIES: Keeps two copies of the data on two different devices
RAID10: Should really be called 2COPIES mirrored: Keeps two copies of the data, each copy spread over half of the available storage devices
RAID5: Should really be called STRIPE + Single parity : Spread the data over all, but one of the available devices. Writes parity information on the leftover device.
RAID6: Should really be called STRIPE + Dual parity : Spread the data over all, but two of the available devices. Writes parity information on the two leftover devices

The thing is, nowhere in the definition of RAID1 does it specify that the data has to be mapped in the same way in both disks; onyl that it be in both disks.
Also, the definition of RAID1 talks about 2 disks, not "one copy of the data in every disk". So Btrfs' approach of still having 2 copies of the data in different devices regardless of the amount of devices doe snot contradict the definition of RAID1 in any way.

Your problem is that you are trying to compare a block-level RAID1, that can only work in one way because it doesn't have knowledge of the data or the files in it, and a filesystem-level RAID1, which does not need to do useless and wasteful "mirroring" of data layouts, since it's aware of the data.

So it is not "normal" RAID1 because it's not a block-level RAID, but a filesystem level one. Still RAID1, though.

Also, RAID1 has never required more than 2 devices that I can remember, and I've been using it for years.
You could (can?) lock it into read-only if you mount it degraded and read-write with only one device and start writing data to it... but if you got to that point... what did you expect?