Announcement

Collapse
No announcement yet.

OpenZFS File-System Merges Support For Using Zstd Compression

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • #11
    Originally posted by uid313 View Post
    Nice to see (Open)ZFS evolve with new features and algorithms.

    I wonder how the future file systems will look in ten or twenty years from now. In a world without SATA, SCSI, SAS and mechanical HDD.
    SSDs are still block devices, so I don't see why the filesystems should change.
    At least on the consumer side, I'm sure we will still have ntfs on whatever form Windows PCs will have in the future

    Comment


    • #12
      Originally posted by k1e0x View Post
      That sounds like Linux. To me it just seems like a waste of time to take a filesystem from the 80's and add this. If that is the case.. we can just do something new. The right tool for the right job.
      why adding hardware support features to rock solid filesystems with a great track record when we can reinvent the wheel and add all the features they support to a new filesystem.

      And then you complain about btrfs when it is literally doing the same thing.

      Comment


      • #13
        Originally posted by starshipeleven View Post
        SSDs are still block devices, so I don't see why the filesystems should change.
        At least on the consumer side, I'm sure we will still have ntfs on whatever form Windows PCs will have in the future
        That is the fun part refs, exfat, fat32 and NTFS all are not designed for zoned storage. Microsoft does not have a single file system that can support zoned without major overhaul and with how long it taken Microsoft to get refs ready I don't see that problem improving any time soon. I can see Microsoft having to release a dm-zoned equal due to no other choice.

        The reality is all consumer SSD are really zoned devices under the controller. For allowing better utilisation doing like SMR(zoned )harddrives have with a host aware mode would be good for consumer SSD.

        So zoned based devices are going to come more common.

        Comment


        • #14
          Originally posted by oiaohm View Post
          That is the fun part refs, exfat, fat32 and NTFS all are not designed for zoned storage. Microsoft does not have a single file system that can support zoned without major overhaul and with how long it taken Microsoft to get refs ready
          yeah refs was basically booted out of the door and kept as-is when they tried to implement more complex stuff that btrfs and ZFS do and found out it was too complex.


          The reality is all consumer SSD are really zoned devices under the controller. For allowing better utilisation doing like SMR(zoned )harddrives have with a host aware mode would be good for consumer SSD.
          So zoned based devices are going to come more common.
          Yes but The Windows Way(tm) is "solve in hardware what could have been solved far better in software", so just as RAID must still be done with hardware cards in 2020 (while Linux software raid has been viable and reliable for decades, and a ZFS RAID5 on a 4GB system laughs at the performance of a RAID5 made with a RAID controller with 1GB of cache), SSDs in consumer space will still have to create an abstraction of their contents for an OS that is unaware of their true architecture.

          AppleOS isn't particularly better in this regard either. They 100% still need a storage controller that emulates a block device in all their products.
          Last edited by starshipeleven; 25 August 2020, 12:20 PM.

          Comment


          • #15
            Originally posted by starshipeleven View Post
            Yes but The Windows Way(tm) is "solve in hardware what could have been solved far better in software", so just as RAID must still be done with hardware cards in 2020
            This is not really the case. Intel Rapid Storage Technology and few other intel based raids are in fact software raids under windows.


            Originally posted by starshipeleven View Post
            They 100% still need a storage controller that emulates a block device in all their products.
            A dm-zoned could be released for windows at some point same way rapid storage technology is done. So no you don't need for Windows a storage controller that emulates a block device into the future You will just need something that emulated a block device.

            AppleOS is a different game they have control of their hardware.

            It would be funny if we see some vendors releasing cheap windows laptops that is windows inside a hypervisor that is Linux that is mapping the zoned drives as normal block devices. This is in theory possible with Linuxboot instead of normal EUFI. Why would a vendor do this SMR and ZAC SSD drives can be cheaper per unit if the drive is not emulating block device.

            I do see even in the consumer space block devices in hardware will go away. But I do also expect emulation of block devices to hang around for quite some time.

            Comment


            • #16
              Originally posted by oiaohm View Post
              This is not really the case. Intel Rapid Storage Technology and few other intel based raids are in fact software raids under windows.
              Also Marvell cards and other low-end RAID cards are technically "software raid", yes I know. It's still not true software raid as it is a vendor-specific thing that works only with their hardware. (example, RAID made with Intel RAID isn't inter-operable with Marvell RAID) At least on Windows, Intel RAID is inter-operable with mdadm and the software raid of FreeBSD afaik.

              A dm-zoned could be released for windows at some point same way rapid storage technology is done.
              Yes you can release all kinds of drivers you want for Windows, but it will still be done by a hardware vendor like Intel so it will be hardware-specific. Aka Intel DM-Zoned Technology won't work on devices connected to a Sata card or a SAS card or even on AMD chipsets.

              AppleOS is a different game they have control of their hardware.
              And yet even if they can, they are going the same route of "solve it in hardware"

              SMR and ZAC SSD drives can be cheaper per unit if the drive is not emulating block device.
              but it will not allow the hardware manufacturer to play with numbers and caches to make their SSD go faster in benchmarks or stupid serial read/write speed tests, as now the smarts will be the same for everyone, in the single driver everyone uses.

              Comment


              • #17
                Originally posted by starshipeleven View Post
                Also Marvell cards and other low-end RAID cards are technically "software raid", yes I know. It's still not true software raid as it is a vendor-specific thing that works only with their hardware. (example, RAID made with Intel RAID isn't inter-operable with Marvell RAID) At least on Windows, Intel RAID is inter-operable with mdadm and the software raid of FreeBSD afaik.
                True to a point.

                Originally posted by starshipeleven View Post
                Yes you can release all kinds of drivers you want for Windows, but it will still be done by a hardware vendor like Intel so it will be hardware-specific. Aka Intel DM-Zoned Technology won't work on devices connected to a Sata card or a SAS card or even on AMD chipsets.
                https://www.softraid.com/features/so...-softraidlite/ That if Intel decides to be that way. Softraid does show you can build generic raid drivers for windows that are able to sit on top of the sata card drivers. Now as long as those let though ZAC/ZNS commands . Of course this could be like Western digital releasing a driver to drive zoned drives correctly by adding a raid device to windows that is really just software manage zoned devices under NTFS.

                Originally posted by starshipeleven View Post
                And yet even if they can, they are going the same route of "solve it in hardware"
                Maybe not we are not sure what apple is up to. You have power management on SSD drives fail when you don't use apple custom ones.

                Originally posted by starshipeleven View Post
                but it will not allow the hardware manufacturer to play with numbers and caches to make their SSD go faster in benchmarks or stupid serial read/write speed tests, as now the smarts will be the same for everyone, in the single driver everyone uses.
                This is you missed something.
                https://www.tweaktown.com/news/70100...eds/index.html

                Dramless SSD as what is being done for the xbox series X you don't have the caches. Dram is expensive on SSD over a million units. There is going to be more and more work to drive SSD without caches because it a cost saving. Yes ZNS provides means todo it better from software.

                There is basically a mighty dollar motivator to get rid of dram caches off SSD and reduce dram on hard drives with the only way to achieve this is to in fact reduce what the controller is processing. Yes its the winmodem all over again right down to expect fragmention.

                Comment


                • #18
                  Originally posted by oiaohm View Post
                  Of course this could be like Western digital releasing a driver to drive zoned drives correctly by adding a raid device to windows that is really just software manage zoned devices under NTFS.
                  I really hope that if this happens they will be benevolent enough to add it in the default install image of Windows because it would be a massive PITA to have to add the driver on install.

                  Maybe not we are not sure what apple is up to. You have power management on SSD drives fail when you don't use apple custom ones.
                  It's a bit hard to not use apple custom ones when the SSD is soldered to the motherboard. Apple stopped allowing people to change the SSD in 2016 I think.


                  This is you missed something.
                  https://www.tweaktown.com/news/70100...eds/index.html

                  Dramless SSD as what is being done for the xbox series X you don't have the caches. Dram is expensive on SSD over a million units. There is going to be more and more work to drive SSD without caches because it a cost saving. Yes ZNS provides means todo it better from software.
                  You think they are doing any kind of software-side thing? Because they aren't, the SSD is still emulating a block device and still keeping its own internal structure tables somewhere (in this case, on flash). Many consumer NVME drives are DRAMless and get away with it because they have an internal caching system that uses some flash as SLC and the flash is very fast to begin with.

                  The Sata-based dramless SSDs actually suck balls, but that's a different market and they sometimes don't even have a fast SLC-like cache at all.

                  Comment


                  • #19
                    Originally posted by starshipeleven View Post
                    I really hope that if this happens they will be benevolent enough to add it in the default install image of Windows because it would be a massive PITA to have to add the driver on install.
                    I agree giving to Microsoft as a free driver would be a good idea.

                    Originally posted by starshipeleven View Post
                    It's a bit hard to not use apple custom ones when the SSD is soldered to the motherboard. Apple stopped allowing people to change the SSD in 2016 I think.
                    That is for laptops. Desktop apple that are expensive as hell you still have changeable SSD.

                    Originally posted by starshipeleven View Post
                    You think they are doing any kind of software-side thing? Because they aren't, the SSD is still emulating a block device and still keeping its own internal structure tables somewhere (in this case, on flash). Many consumer NVME drives are DRAMless and get away with it because they have an internal caching system that uses some flash as SLC and the flash is very fast to begin with.
                    https://www.elinfor.com/knowledge/dr...ng-hmb-p-10922

                    HMB host memory buffer. Its not possible to make a NVME that performs well without using dram somewhere. Host Memory Buffer can be used with NVME as a different option to going ZNS inst4ead the OS still has to include support for this so the NVME drive can use your host OS memory. Really how long until a memory exploit really. Of course this still requires a very complex controller. SLC in flash really never gives the performance lot of the ssd performance problem is dealing with block de-fragmentation.

                    Originally posted by starshipeleven View Post
                    IThe Sata-based dramless SSDs actually suck balls, but that's a different market and they sometimes don't even have a fast SLC-like cache at all.
                    Sata based dramless SSD don't have the option of host memory buffer because sata is not fast enough. For sata base dramless SSD the only hope to fix performance is move block management out the controller and into the host completely and this is ZNS as in Zoned.

                    Emulating a block device on a flash device with performance requires dram somewhere also you require cpu core able to process it as well. Now remember cpu core and dram are both costs in dollars and power. Something else horrible here to consider storage vendors also want to claim to be the most power effective device ZNS where you move the block management into the cpu and ram of the host lowers the SSD devices power usage by slide of hand. There is pressure to completely get rid of get rid of dram and block device emulation off all from of SSD this for vendors of computers:
                    1 to lower device cost
                    2 to be able to claim lower power usage

                    Of course users want performance and that what HMB and ZNS are to fix. Block device emulation that works with performance cannot be done without dram somewhere. Yes the SSD drive makers have well and truly tried using flash instead and other options like that in current generations of dramless SSD and it has not worked..

                    Yes dramless SSD will stay around and come more dominate when the performance issue is fixed. Like who will pay more if it don't run faster. HMB vs ZNS on NVME is going to get interesting what one is the faster tech both are going to take a percentage of your host memory and cpu time and require driver at this stage ZNS looks faster.

                    Comment


                    • #20
                      Originally posted by oiaohm View Post
                      ... That is for laptops. Desktop apple that are expensive as hell you still have changeable SSD. ...
                      At least the Mac mini has soldered M.2 PCIe SSD (and CPU).

                      Comment

                      Working...
                      X