Announcement

Collapse
No announcement yet.

Linus Torvalds Doesn't Recommend Using ZFS On Linux

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Originally posted by k1e0x View Post
    k? I'm sure he is. It's odd you talk about the "XFS guy" .. where as ZFS is a community of paid developers, is only one person working on improving XFS? That is no good.
    https://lwn.net/Articles/796847/
    There is the XFS project lead who you see in that video laying out a plan then multi different personal including Oracle staff bring it into reality.

    Originally posted by k1e0x View Post
    I did watch this on the XFS design and plans.
    Teaching an old dog new tricks. https://www.youtube.com/watch?v=wG8FUvSGROw
    So you have watched it.

    Originally posted by k1e0x View Post
    Talking about loopback devices cp and tar... and showing scripts that implement this.. it's ugly.
    That was only the prototype. iomap complete no loopback devices in play

    Originally posted by k1e0x View Post
    He also has a small mistake in that ZFS does not cache the file "bash" from 1000 clones.
    That was not refering ZFS this is your expertise getting in the way its referring the issue btrfs and it interaction with the VFS layer above it. It also something that could happen with namespaces as well with ZFS.

    Originally posted by k1e0x View Post
    The ARC only works on blocks, 1000 clones of bash if unmodified share the same blocks. The ARC also isn't a LRU cache like most other filesystems. It's why benchmarking ZFS isn't really fair if you defeat it's cache, because the ARC is integral to it's use and it actually works, unlike LRU.
    iomap work is to bring ARC like block management to all Linux kernel supported filesystems in time. Also provide a route so this declone at block layer is not undo by something at the VFS layer. Something you forget Linux kernel memory management above ZFS when running on Linux is also a LRU. Some of the work of iomap is to fix the Linux kernel memory system.

    I hope ARC is ready to handle 2Meg blocks and other horrible caused by the up coming Linux kernel large pages.
    Last edited by oiaohm; 01-24-2020, 12:37 PM.

    Comment


    • Originally posted by k1e0x View Post

      I can assure you ZoL *IS* used in enterprise. I've used ZFS in enterprise since ~2006 on Solaris, FreeBSD and Linux. It is pretty much the fix for ransomware.

      The CDDL nor the GPL have restrictions on usage, only on distribution. You can and always have been able to do whatever you want on your own systems.

      I posted this before. 100 PB - ZoL Implementation. Storage of that size actually isn't that unique for ZFS (typically more Solaris and FreeBSD but ZoL is getting more popular.) Most enterprises don't share information on their storage platforms. Since that one is a quite impressive research system they do have details posted about it.
      https://medium.com/codedotgov/oss-sp...x-6596fca6e5f6

      "unlike ext4/EXFS, when ZOL breaks (and I've seen it break badly in production environment, including very recently) you get to keep all the pieces." -- seriously? You're going to make the claim ext4 is *better* at data integrity than ZFS? No.. It's not. Not in any universe. Like Linus you don't seem familiar with ZFS.. that is ok.. but unlike Linus you shouldn't go making claims when you don't have the basic facts.

      FYI, ZFS is a copy on write, always consistent file system with a checksum on every data block.
      EXT4 is... not. It is a journaled, extent based file system that updates blocks in place making it not always consistent. Do you like fsck? Or data blocks with silent corrupt files? EXT4 is for when you want a small but present possibility of your pictures looking like this and never knowing till you open it.
      Either my writing skills have massively deteriorated or your reading skills (intentionally or unintentionally) are lacking.

      1. ZFS on Oracle (Be that Solaris or Unbreakable Linux) != ZOL.
      One is *supported* by a major enterprise-grade OS supplier the other, is, well, not.
      Comparing the two is humorous, at best.
      Most of the ZFS on Linux users don't bother to pay the huge sums of money required to get proper support.
      And if it breaks (see 3) they will get to keep all the pieces.

      2. ext4 *is* support by enterprise-grade OS supplier (Be that RHEL, SUSE or Ubuntu).
      Good luck trying to get support for your RHEL data cluster if you use anything ZFS.
      Same goes for any other enterprise grade Linux distribution (beyond Oracle, that is, see 1).

      3. Copy on write my protect you from single-bit-curroption. It *does* not protect you from bugs.
      Having just witnessed a 0.8PB cluster (w/ ZFS) go up the flames, I'm not that impressed.

      Again, if you use ZFS on a support platform, good for you.
      If you are stupid enough to use ZOL on an unsupported platform thinking that copy-on-write might save your ass, think again.

      - Gilboa
      DEV: Intel S2600C0, 2xE5-2658V2, 32GB, 6x2TB, GTX1080, F32, Dell UP3216Q 4K.
      SRV: Intel S2400GP2, 2xE5-2448L, 96GB, 6x2TB, GTX550, F32, Dell U2711.
      WIN: Gigabyte B85M-HD3, E3-1245V3, 32GB, 5x1TB, GTX980, Win10Pro.
      BAK: Asus H110M-K, i5-6500, 16GB, 3x1TB + 128GB-SSD, F32.
      LAP: ASUS Strix GL502V, i7-6700HQ, 32GB, 1TB+256GB, 1070M, F31.

      Comment


      • Originally posted by gilboa View Post

        Either my writing skills have massively deteriorated or your reading skills (intentionally or unintentionally) are lacking.

        1. ZFS on Oracle (Be that Solaris or Unbreakable Linux) != ZOL.
        One is *supported* by a major enterprise-grade OS supplier the other, is, well, not.
        Comparing the two is humorous, at best.
        Most of the ZFS on Linux users don't bother to pay the huge sums of money required to get proper support.
        And if it breaks (see 3) they will get to keep all the pieces.

        2. ext4 *is* support by enterprise-grade OS supplier (Be that RHEL, SUSE or Ubuntu).
        Good luck trying to get support for your RHEL data cluster if you use anything ZFS.
        Same goes for any other enterprise grade Linux distribution (beyond Oracle, that is, see 1).

        3. Copy on write my protect you from single-bit-curroption. It *does* not protect you from bugs.
        Having just witnessed a 0.8PB cluster (w/ ZFS) go up the flames, I'm not that impressed.

        Again, if you use ZFS on a support platform, good for you.
        If you are stupid enough to use ZOL on an unsupported platform thinking that copy-on-write might save your ass, think again.

        - Gilboa
        1. I have never seen Oracle ZFS storage appliance used in the enterprise personally, I know they exist I've just never seen them. Only Sun Microsystem's before Solaris version 10. (Pool version 28~) Generally it's used on FreeBSD storage clusters that are secondary storage to NetApp, EMC or DDN. (around 32-64 spindles) You can get paid commercial ZFS (and ZoL) support from both FreeBSD 3rd parties and Canonical. Ubuntu 19.10 has ZFS on root in the installer and so will 20.04-LTS

        2. All I got to say about ext4 is hope you like fsck.

        3. You don't know what you're talking about. COW alone has nothing to do with bit-rot or uncorrectable errors. You're thinking of block checksums, and yes, they are good. COW provides other features such as snapshots, cloning and boot environments. Boot environments are pretty cool.. maybe Linux should get on that... oh wait.. ZFS is the only Linux file system that does it and we can't have *that*.

        Check this out.. FreeBSD 12 has a new command for boot environments..
        https://www.freebsd.org/cgi/man.cgi?query=bectl

        bectl create zroot@snap
        bectl jail zroot@snap

        You just cloned your running OS into a writable, bootable, virtual environment and spawned a shell in it and it was instant.
        Last edited by k1e0x; 01-28-2020, 10:52 PM.

        Comment


        • Originally posted by k1e0x View Post
          3. You don't know what you're talking about. COW alone has nothing to do with bit-rot or uncorrectable errors. You're thinking of block checksums, and yes, they are good. COW provides other features such as snapshots, cloning and boot environments. Boot environments are pretty cool.. maybe Linux should get on that... oh wait.. ZFS is the only Linux file system that does it and we can't have *that*.

          Check this out.. FreeBSD 12 has a new command for boot environments..
          https://www.freebsd.org/cgi/man.cgi?query=bectl

          bectl create zroot@snap
          bectl jail zroot@snap

          You just cloned your running OS into a writable, bootable, virtual environment and spawned a shell in it and it was instant.
          You really need to stop that claim now. ostree supports XFS reflink. Ostree used in combination with XFS with relink gives this exact same feature of cloning boot environments quickly with disc space effectiveness and created checksums to locate damage. Technically any file system that supports relink could be used.

          So there are three file systems under Linux that can do boot environments. ZFS, Btrfs/LVM and Ostree/XFS. Ostree/XFS you see in Fedora project silverblue.

          Really ZFS does not have as many advantages as one would think other than its per block protection. XFS being a part CoW file system with reflink means with software like ostree on top it can pull large number of the ZFS tricks but with higher overall performance.

          Comment


          • Originally posted by oiaohm View Post

            You really need to stop that claim now. ostree supports XFS reflink. Ostree used in combination with XFS with relink gives this exact same feature of cloning boot environments quickly with disc space effectiveness and created checksums to locate damage. Technically any file system that supports relink could be used.

            So there are three file systems under Linux that can do boot environments. ZFS, Btrfs/LVM and Ostree/XFS. Ostree/XFS you see in Fedora project silverblue.

            Really ZFS does not have as many advantages as one would think other than its per block protection. XFS being a part CoW file system with reflink means with software like ostree on top it can pull large number of the ZFS tricks but with higher overall performance.
            OpenSUSE has pretty much the cutting edge of btrfs integration. From their manual.
            3.3.3 Limitations
            A complete system rollback, restoring the complete system to the identical state as it was in when a snapshot was taken, is not possible.
            There is a whole list of things that it can't restore (including the kernel) This isn't true on FreeBSD it can restore everything to the way it was before.

            I did a search for XFS writable snapshots... Google said "Did you mean ZFS writable snapshots?" I think that's telling..

            Linux does not really have this feature in anything I've seen.. I don't see it in any distros and even there is a way to do it it'd def not as clean or well implemented. It probably also does a copy so it's slow as hell.

            On FreeBSD it takes one command to add a boot environment you can select from the boot loader. No packages, no config changes. no waiting for anything to copy. It's writable, closable and transportable to another system and works even on an encrypted disk. The jail integration is just the cherry on top.. that's *two* commands.

            Use.. maybe your system upgrade went sideways and you need to get the system back up. Restore the old environment and boot production then send the failed update system to a dev system and boot the environment in a jail. After you fix and test it you can send it back.
            Last edited by k1e0x; 01-29-2020, 02:01 AM.

            Comment


            • Originally posted by k1e0x View Post
              OpenSUSE has pretty much the cutting edge of btrfs integration. From their manual.[/URL]
              Do note I write Btrfs/LVM not pure Btrfs. LVM can do the perfect roll back snapshots under the file system. That quote from manual was pure Btrfs solution.

              Originally posted by k1e0x View Post
              I did a search for XFS writable snapshots... Google said "Did you mean ZFS writable snapshots?" I think that's telling..

              On FreeBSD it takes one command to add a boot environment you can select from the boot loader. No packages, no config changes. It's writable, closable and transportable to another system.
              Basically its telling that you are tunnelled visioned.

              https://www.projectatomic.io/docs/os-updates/ Yes ostree from project atomic is also registering after 1 command in the boot loader.

              That because the XFS/ostree one is not called snapshots. Its atomic upgrades https://ostree.readthedocs.io/en/lat...omic-upgrades/. "cloning boot environments" does not have to equal snapshots.

              Please note these atomic upgrades can function without relink just not as disc effective.

              https://ostree.readthedocs.io/en/lat...ting-existing/

              What is different here is you are creating the snapshots above the file system using the VFS layer todo the bending. Really do you want your boot environment fully writable?

              ostree is slower without the relink system but will work anyhow.

              Ostree/project atomic is clone able and transportable to other systems that are not using the same file system type.


              There are three places you can do snapshoting.

              1) Block layer. LVM
              2) VFS layer (mount namespaces under Linux)
              3) File system.

              Snapshotting to be disc space effective just need a cow of some from. LVM contains a CoW. XFS contains a CoW by relink. Of course ZFS and Btrfs contain a CoW.

              k1e0x I guess the concept of a VFS layer snapshotting never crossed your mind.
              Last edited by oiaohm; 01-29-2020, 02:09 AM.

              Comment


              • Originally posted by oiaohm View Post

                Do note I write Btrfs/LVM not pure Btrfs. LVM can do the perfect roll back snapshots under the file system. That quote from manual was pure Btrfs solution.



                Basically its telling that you are tunnelled visioned.

                https://www.projectatomic.io/docs/os-updates/ Yes ostree from project atomic is also registering after 1 command in the boot loader.

                That because the XFS/ostree one is not called snapshots. Its atomic upgrades https://ostree.readthedocs.io/en/lat...omic-upgrades/. "cloning boot environments" does not have to equal snapshots.

                Please note these atomic upgrades can function without relink just not as disc effective.

                https://ostree.readthedocs.io/en/lat...ting-existing/

                What is different here is you are creating the snapshots above the file system using the VFS layer todo the bending. Really do you want your boot environment fully writable?

                ostree is slower with the relink system.

                Ostree/project atomic is clone able and transportable to other systems that are not using the same file system type.


                There are three places you can do snapshoting.

                1) Block layer. LVM
                2) VFS layer (mount namespaces under Linux)
                3) File system.

                Snapshotting to be disc space effective just need a cow of some from. LVM contains a CoW. XFS contains a CoW by relink. Of course ZFS and Btrfs contain a CoW.

                k1e0x I guess the concept of a VFS layer snapshotting never crossed your mind.
                Wow that sounds like a lot of limitations... well don't worry.. Linux will catch up someday. The technology to do this is only a decade old.. and still not in RHEL.. shame..

                Those github setup guides look fun.. but you know I think I'd just like to do:
                Step 1. Install FreeBSD
                Step 2. You already have atomic upgrades enabled, there is no step two.
                Last edited by k1e0x; 01-29-2020, 02:17 AM.

                Comment


                • Originally posted by k1e0x View Post
                  Those github setup guides look fun.. but you know I think I'd just like to do:
                  Step 1. Install FreeBSD
                  Step 2. You already have atomic upgrades enabled, there is no step two.
                  Swap FreeBSD for RHEL/Centos Atomic Host or Silverblue. Yes step 2 stays the same and you have atomic upgrades enabled outbox no matter the file system you choose.

                  I don't think FreeBSD has atomic upgrades if choose some other file system other than ZFS on install. Yet the ones I listed you can choose many different file systems and still have atomic upgrades.

                  I don't think ZFS will ever catch up on raw performance.
                  Last edited by oiaohm; 01-29-2020, 03:03 AM.

                  Comment


                  • Originally posted by oiaohm View Post
                    I don't think ZFS will ever catch up on raw performance.
                    Hard to do when people cripple it with the benchmarks by defeating the ARC. Want to drag race a 4x mirror with a L2ARC cached on an nvme? Nahh.. I don't trust you after the ageist post. haha

                    Comment


                    • Originally posted by k1e0x View Post
                      Hard to do when people cripple it with the benchmarks by defeating the ARC. Want to drag race a 4x mirror with a L2ARC cached on an nvme? Nahh.. I don't trust you after the ageist post. haha
                      LOL Its not like its impossible for XFS to have a 4x mirror with a block cache in front of it behind it in the next Linux kernel releases. Sorry you don't have the speed.

                      Yes block cache on nvme is also possible. So that does not give you a speed advantage. Its sad to watch these ZFS fans be out of date on benchmarks because they don't want to admit their ass is kicked in raw performance. Feature advantage is not as big as they want to make out either.

                      Comment

                      Working...
                      X