Announcement

Collapse
No announcement yet.

Mdadm .VS hardware RAID

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • #16
    I run the OS side of most of my Sun servers in RAID1 (or read-optimized RAID10 if I need the speed), using mdadm under Linux or using the native mirroring tools under Solaris. If I have Veritas on Solaris I generally go that route.

    I like being able to go between any of my servers with the drives (as long as it takes spud brackets anyway) and being able to recover the array.
    Apps generally run on SAN hardware, so speed isn't an issue there (having 16-32GB of fast cache helps there )

    Comment


    • #17
      why are you using raid1 and not 5 or 6? Just asking.

      Comment


      • #18
        Originally posted by energyman View Post
        why are you using raid1 and not 5 or 6? Just asking.
        Mirroring is more robust, particularly with software RAID: parity-based RAID has some horrible failure cases; for example, if one 2TB drive fails due to old age, the odds of a second 2TB drive failing while you try to recreate the RAID array are not insignificant.. then your entire RAID is gone.

        Comment


        • #19
          not with raid6 where you can have 2 devices fail and still recover. And mirroring - what do you do when one disk reports the contents of block X as FOO but the other one as BAR. Which one has the correct block?

          Comment


          • #20
            The quick answer is software RAID is faster unless your server is under intense CPU use all the time (and thus are getting the benefit of the onboard coprocessor), but I would never recommend using mdadm in anything relied on for production.

            I have stats from a 3ware 8500 and dual HighPoint RocketRAID 1820s that say that RAID 5 and RAID 6 are slignificantly (20%+) faster in software (mdadm) than either Linux 2.6.18 driver. I don't think anyone would say that these cards are fakeRAID. Granted this is a little dated, but still.

            However, in both cases I got lots of fake bad disks (i.e. the software RAID would detect an intermittent blip and disable the drive) which made using it this way a nightmare to maintain in a production environment. At first I thought it was bad SATA cables, or drive enclosures, or a bad batch of disks, etc. However, my arrays were degraded almost 5% of the time since one of these bad disk events would occur almost weekly under load. Eventually I decided the speed wasn't as important as the reliability of the array and switched back to hardware. Same disks/ cables/ enclosures/ controllers never had an issue in hardware RAID.

            Comment


            • #21
              Originally posted by energyman View Post
              not with raid6 where you can have 2 devices fail and still recover.
              Sure, but you need a minimum of five disks before that's more efficient than mirroring.

              And mirroring - what do you do when one disk reports the contents of block X as FOO but the other one as BAR. Which one has the correct block?
              Generally speaking, if a disk fails then either the sector will be readable or it won't: with mirroring you can keep the readable sector and ignore the unreadable one. You could get the disks out of sync if they were writing during a power-down such that one disk was written to and the other wasn't, but that's a problem with any form of RAID which doesn't have battery backup.

              Admittedly much of this could be worked around by smarter RAID implementations: i.e. don't fail the entire disk if one block is unreadable, but continue attempting to read the rest of the data as you rebuild the parity onto a new disk. Doesn't ZFS do something like that?

              Comment


              • #22
                Originally posted by energyman View Post
                why are you using raid1 and not 5 or 6? Just asking.
                most Sun boxes don't come with a gaggle of disks. I have the SAN for any big arrays.

                my v20zs, for instance, have two disk drives. my v40zs can hold six, but I really don't need six local disks. The SAN can outperform anything locally that I could install. The v40zs where I have more than two disks, I usually just use the extra for liveupdate or RAID10.

                Comment


                • #23
                  Originally posted by lbcoder View Post
                  Software raid can NEVER be *faster* (it is at best AS fast for stupid raid types, like striping) than hardware raid unless those who implemented the hardware raid were a bunch of rabid chimps on acid. Any kind of parity-raid (raid 5, 6, etc.) will ALWAYS be slower in software raid than in hardware raid, especially when the host CPU is under load.
                  I think I may have to disagree with you on this one. I know for a fact that my CPU is a hell of a lot faster than pretty much every areca board I've used.

                  Comment


                  • #24
                    Originally posted by movieman View Post
                    Sure, but you need a minimum of five disks before that's more efficient than mirroring.



                    Generally speaking, if a disk fails then either the sector will be readable or it won't: with mirroring you can keep the readable sector and ignore the unreadable one. You could get the disks out of sync if they were writing during a power-down such that one disk was written to and the other wasn't, but that's a problem with any form of RAID which doesn't have battery backup
                    that is not about 'dying disk'. A unclean shutdown, a glitch and suddenly you have two different blocks. With raid5/6 you can recover. With Raid1 you only can hope that the kernel makes the right guess.

                    Comment


                    • #25
                      Originally posted by energyman View Post
                      With raid5/6 you can recover. With Raid1 you only can hope that the kernel makes the right guess.
                      I'm not sure I understand the difference: if you corrupt random blocks on a RAID5, how do you tell which are the correct ones? Obviously if you can tell that one disk didn't complete all its writes you can recreate the RAID from the other disks by completely rewriting that one (assuming the other disks don't have bad blocks in files you last updated months ago which cause the whole array to die while rebuilding), but the same applies to a RAID1. Maybe you could do some magic by looking at the output from recovering a block from different combinations of disks to try to figure out which is corrupt.

                      Personally the more I read about RAID failures, the more I'm coming to the conclusion that I need to install a Solaris server at home and switch to ZFS... it seems to be the only RAID system that doesn't risk losing all your data when there's a disk error.

                      Comment


                      • #26
                        Originally posted by movieman View Post
                        I'm not sure I understand the difference: if you corrupt random blocks on a RAID5, how do you tell which are the correct ones?
                        You compute the correct data using the relevant ECC algorithm, which works as long as you have enough non-corrupt pieces for that block.

                        Comment


                        • #27
                          Originally posted by Ex-Cyber View Post
                          You compute the correct data using the relevant ECC algorithm, which works as long as you have enough non-corrupt pieces for that block.
                          That doesn't really help me to understand what RAID5 is supposed to be doing. From what I do understand, RAID5 writes multiple data blocks and then a parity block; we're talking about a situation where all those blocks are read from the disk with no errors, but one of them has corrupt data. How can it recover from that? You can tell that one of the blocks is wrong because the parity is wrong, but I don't see how you can tell _which_ block is wrong.

                          ZFS, for example, stores a checksum which allows it to determine which of the blocks is corrupt (if it gets the correct checksum by merging data from disks 1, 2 and 3 then the data on disk 4 must be bad), but RAID5 doesn't seem to store any extra data which would allow it to do something similar.
                          Last edited by movieman; 11-06-2009, 01:33 PM.

                          Comment


                          • #28
                            Originally posted by movieman View Post
                            That doesn't really help me to understand what RAID5 is supposed to be doing. From what I do understand, RAID5 writes multiple data blocks and then a parity block; we're talking about a situation where all those blocks are read from the disk with no errors, but one of them has corrupt data. How can it recover from that? You can tell that one of the blocks is wrong because the parity is wrong, but I don't see how you can tell _which_ block is wrong.
                            I might be getting my schemes mixed up; I thought RAID 5 used something more complicated than parity (e.g. Reed-Solomon).

                            Comment


                            • #29
                              Originally posted by Ex-Cyber View Post
                              I might be getting my schemes mixed up; I thought RAID 5 used something more complicated than parity (e.g. Reed-Solomon).
                              I may be confused too, but all I could find on RAID5 implied that it typically just XORs the data together and stores that as parity, so you can verify the block by XORing all the parts and checking the result is zero.

                              Obviously if you go to two parity disks then you can figure out which block is correct if only one of them is wrong.

                              Comment


                              • #30
                                Originally posted by lbcoder View Post
                                The advantage of software raid is portability (i.e., you can plug your disks into ANY controller and access your array rather than being stuck buying a new card if the card burns out), and cost (i.e., you don't need to buy a raid card for software raid).
                                ..and flexibility. At least with the hardware RAIDs I've played with (though it's been a long time since my Adaptec-worshipping days ;-), you could only work with whole disks. /sda+/sdb+/sdc combined to only one of RAID0|RAID1|RAID5 and that was that. Software RAID lets you have /sda1+/sdb1+/sdc1 be a RAID1 root, with /sda2+/sdb2+/sbc2 being a RAID0 /tmp and /sda3+/sdb3+/sdc3 as RAID5 /home, etc.

                                Comment

                                Working...
                                X