Announcement

Collapse
No announcement yet.

Btrfs Adds Degenerate RAID Support, Performance Improvements With Linux 5.15

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • #11
    Originally posted by jacob View Post
    Is RAID5 on Btrfs still broken?
    RAID5/6 still has the write hole, but if you use RAID1c3 or RAID1c4 for metadata and remember to scrub immediately after an unclean unmount you should be good at least in theory. I would not not yet use it myself (and I have tested , working backups).

    http://www.dirtcellar.net

    Comment


    • #12
      Originally posted by waxhead View Post

      RAID5/6 still has the write hole, but if you use RAID1c3 or RAID1c4 for metadata and remember to scrub immediately after an unclean unmount you should be good at least in theory. I would not not yet use it myself (and I have tested , working backups).
      Have been using RAID5 on multiple workstation servers for almost 8 years now. No data loss! However I've got a UPS.

      I do have RAID1 for metadata and RAID5 for data and scrub quarterly in a year. Also use space_cache=v2.

      Comment


      • #13
        The last time I tried BTRFS RAID5/6 arrays I tested by purposefully failing drives and it completely broke. Totally unreliable in my experience.

        Comment


        • #14
          Originally posted by linner View Post
          The last time I tried BTRFS RAID5/6 arrays I tested by purposefully failing drives and it completely broke. Totally unreliable in my experience.
          A phoronix benchmark would use a tantan gosund amazon: "Smart Power Strip Work with Alexa Google Home, Smart Plug Mini WiFi Outlets Surge Protector with 3 USB 3 Charging Port for Cruise Ship Travel Multi-Plug Extender,10A (Black)"
          Overrun with opensource software to power off the sata drives he wants to fail for FS benchmark of raid.

          or use a digital loggers Iot Relay( 3 plugs power on/off together)
          or use a digital loggers Web Power Switch Pro Model ( most plugs you can turn on/off separate)



          Note about how to overrun to take control:

          Don't care about no Alexa, flash Tasmota and take control of your devices
          Reviewed in the United States on May 2, 2020
          Verified Purchase
          I am using this to control the lights and power supply for my new 3D printer. For my last printer, I used solid state relays controlled directly by my Raspberry Pi running Octoprint but wasn't 100% satisfied that there weren't occasional glitches...
          Because I run a bit of an IoT house, I have a Raspberry Pi running an MQTT broker and Node Red to handle lights and vacations and publishing a weather abstract to my personal website. The WP9 appealed to me because each of the outlets was individually controllable, as was the set of (3A!) USB ports. It is listed as a supported device on the Tasmota website, so I was confident I could bend it to my will without too much trouble. Sure enough, Tuya Convert worked it's voodoo and I was able to reprogram the WP9 to be controllable over MQTT from my RPi broker. I was able to make the USB power be independent of the the pushbutton (they are now always on), because they are sufficiently powerful to power the Raspberry Pi which runs my printer (don't want them randomly resetting). I can reprogram the button to turn on one of the outlets (the one with the lights) via Node Red. I can also use Node Red to monitor the progress of the print and turn the printer power off (shhhh, quiet) 20s after the print is done and the lights over the printer 20 minutes later. It's like magic, but it's not. As the immortal Arthur C. Clarke once said: Any sufficiently advanced technology is indistinguishable from magic. I like it so much that I will buy another for my other printer.
          Good: Using the Open-Source work of hundreds of people far cleverer than I, you can take control of this device and make it do what you want without routing your on/off requests through Alexa, Google or a server in China.
          Bad: Cannot think of a single negative.



          Comment


          • #15
            Originally posted by mppix View Post

            Afaik, yes, and it is pretty much discouraged for SSD and large HDD so not sure if even makes sense to fix as even a correct implementation has a decent chance of full data loss.
            Mdraid or lvm can do it for those that really need/want it.
            Would there ever someone create something RAID5/6-like filesystem/array for SSD? In my understanding the concept of RAID5/6 is to maximize the available storage volume, and keep things transparent and operational when there is one/two-drive-loss. if SSDs are put into an old style RAID1/10/5/6 array, all drives will likely wear out at the same rate and fail, in time interval, too close together for even hot spare to save the game. I don't understand why people talk as if RAID6 is outdated for SSD but RAID1/10 are still okay. From what I learnt so far they are all the same in face of the new properties of SSD.

            Comment


            • #16
              Originally posted by linner View Post
              The last time I tried BTRFS RAID5/6 arrays I tested by purposefully failing drives and it completely broke. Totally unreliable in my experience.
              When was last time? That was my experience as well with 4.x kernels. Last time I did some test with it was something >5.2 kernel, but I can't remember which one. On real hardware it worked fine , but if I used a USB array it (as expected) failed horribly.

              http://www.dirtcellar.net

              Comment


              • #17
                Originally posted by mppix View Post

                Afaik, yes, and it is pretty much discouraged for SSD and large HDD so not sure if even makes sense to fix as even a correct implementation has a decent chance of full data loss.
                Mdraid or lvm can do it for those that really need/want it.
                RAID5/6 is fine for ZFS which fixes the RAID 5/6 write hole. Also the main issue about RAID 5/6 (i.e. rebuilds potentially causing a cascading failure with the RAID system) is almost a non issue for modern enterprise NAS drives so its pretty much FUD at this point.

                Originally posted by waxhead View Post

                When was last time? That was my experience as well with 4.x kernels. Last time I did some test with it was something >5.2 kernel, but I can't remember which one. On real hardware it worked fine , but if I used a USB array it (as expected) failed horribly.
                I wouldn't touch BTRFS if you are intending to use RAID5/6 with a 10 foot pole. Its one of the main reasons why I am still with ZFS (also ZFS has an actual history of being reliable, BTRFS does not).

                BTRFS seems fine for datacenters that only care about RAID 10 and local systems where you just have a single drive. I wouldn't use it for anything else
                Last edited by mdedetrich; 01 September 2021, 07:02 AM.

                Comment


                • #18
                  On the topic of btrfs, as it's timely:

                  I have a 4 disk RAID1 setup. On Monday I had some corruption on one of my disks (not enough to bring it out of the raid set), and for the most part scrub took care of it, but having naively followed the advice of the ArchLinux wiki, my postgresql data directory (/var/lib/postgres/data) was chattr +C, and as a consequence of how btrfs implements No_Cow there is no checksums available, so the database got corrupted and postgres refuses to read from it (toast errors).

                  Now presumably I know that the unreliable copy is /dev/sdd3 - that's where all of the detected corruption was, there should be a way to recover my chattr +C files off the working disk; but I can't find it. Is btrfs raid 1 + chattr +C as dangerous as it seems to me?

                  Comment


                  • #19
                    Originally posted by TheCycoONE View Post
                    On the topic of btrfs, as it's timely:

                    I have a 4 disk RAID1 setup. On Monday I had some corruption on one of my disks (not enough to bring it out of the raid set), and for the most part scrub took care of it, but having naively followed the advice of the ArchLinux wiki, my postgresql data directory (/var/lib/postgres/data) was chattr +C, and as a consequence of how btrfs implements No_Cow there is no checksums available, so the database got corrupted and postgres refuses to read from it (toast errors).

                    Now presumably I know that the unreliable copy is /dev/sdd3 - that's where all of the detected corruption was, there should be a way to recover my chattr +C files off the working disk; but I can't find it. Is btrfs raid 1 + chattr +C as dangerous as it seems to me?
                    Yes. You're supposed to solve tyke checksums in your application if you turn off checksums with nodatacow. Without checksums there is no way for btrfs to detect data errors,and on redundant profiles it can't heal the files.

                    You could remove the bad disk completely and run on degraded mode while you copy your database file,assuming there are no further errors caused by the corruption.

                    I suggest you join #btrfs on libera.chat to get help from the devs.

                    A side note. Had you been running mdadm raid1 with ext4/xfs you also would have not been able to correct the situation. Possibly even not seen any error until your disk caused serious problems.

                    Comment


                    • #20
                      Originally posted by mdedetrich View Post

                      RAID5/6 is fine for ZFS which fixes the RAID 5/6 write hole. Also the main issue about RAID 5/6 (i.e. rebuilds potentially causing a cascading failure with the RAID system) is almost a non issue for modern enterprise NAS drives so its pretty much FUD at this point.
                      Says who?
                      Are all the new raid solutions to prevent simultaneous failing of SSDs pointless?
                      .. and how about all the papers that evaluate the risk of a second drive failure as a function of HDD size?

                      Comment

                      Working...
                      X