OpenZFS File-System Merges Support For Using Zstd Compression

[starshipeleven]

That is for laptops. Desktop apple that are expensive as hell you still have changeable SSD.

Ah yes the iMacs and the cheese grater Pro, I forgot. Are those having problem with non-Apple SSDs though? I never heard of that on iMacs.

https://www.elinfor.com/knowledge/dr...ng-hmb-p-10922

HMB host memory buffer.

Neat, they are using DMA to use the host RAM as SSD controller cache. This is possible only because NVME is PCIe and as such it has DMA access.
Still a trick at hardware level, still a controller creating a virtual abstraction, the OS isn't involved in this.

Sata based dramless SSD don't have the option of host memory buffer because sata is not fast enough. For sata base dramless SSD the only hope to fix performance is move block management out the controller and into the host completely and this is ZNS as in Zoned.

Sata has no DMA so there is no way in hell that they can use host RAM as their cache, like NVME drives do.
That said, Sata dram-less SSDs are targeting the cheapest of the cheap consumer segments, and isn't going to exist anymore in the long run.
I really doubt it makes sense to invest about making a windows driver to control them from the OS.

Emulating a block device on a flash device with performance requires dram somewhere also you require cpu core able to process it as well.

And that's already the road they are invested in. Dram is a dumb component they just buy and stick to the side of their prized controller with their secret sauce firmware. They can do shenanigans like DMA-ing into the host RAM instead of using a Dram chip onboard, but I don't see them dropping controllers and go for OS drivers in consumer sector for a long while still, if ever.

In businness sector on the other hand, ZNS SSDs is the buzzword of the moment https://www.westerndigital.com/compa.../zns-explained

So it will probably be done, also because in businness sector you really need the performance gains of that.

[oiaohm]

Ah yes the iMacs and the cheese grater Pro, I forgot. Are those having problem with non-Apple SSDs though?

Yes the problem is non-Apple SSD are running into power management functions of SSD not working. That of course makes the SSD run a lot hotter than it should.

Sata has no DMA so there is no way in hell that they can use host RAM as their cache, like NVME drives do.
That said, Sata dram-less SSDs are targeting the cheapest of the cheap consumer segments, and isn't going to exist anymore in the long run.
I really doubt it makes sense to invest about making a windows driver to control them from the OS.

Except the problem of no DMA don't end at Sata drives. Think removable flash drives.

And that's already the road they are invested in. Dram is a dumb component they just buy and stick to the side of their prized controller with their secret sauce firmware. They can do shenanigans like DMA-ing into the host RAM instead of using a Dram chip onboard, but I don't see them dropping controllers and go for OS drivers in consumer sector for a long while still, if ever.

You are making a lot of the same arguments people made before win-modems come the normal why dial-up modems would not lose their controllers.

Problem here DMA is really a shenanigans there is a long list of security exploits that lead back to DMA offloading to ram. There is also locations like over usb 3.0 and SATA where DMA is not an option either.

https://www.anandtech.com/show/12819/the-toshiba-rc100-ssd-review/2

Please note the HMB devices first turned up 2 years ago. Using controller to host ram even for a NVME instead of on board Dram chip still cost performance. Round tripping from the NVME controller to the motherboard memory controller and back is not free. This is what starts work on ZNS. Doing zoned in the cpu and ram of the host you avoid a lot of round trips. Sorry that prised controller with dram that they can cheat different benchmarks with costs quite a bit.

So it will probably be done, also because in businness sector you really need the performance gains of that.

You need to think volume. If business sector is using ZNS controller chip does it really make sense to keep on making a non ZNS controller chip for the cheaper devices. Its the same way AMD uses same blocks of silicon from desktop all the way though to epic giving them a competitive advantage. SSD ZNS controller gets to be smaller and less complex than a SSD block device emulating controller so cheaper. Yes getting rid of DRAM is the and using HMB host memory buffer was precursor to SSD ZNS work. At some point some storage vendor will release a consumer driver for ZNS if Microsoft does not just to simplify their production and cut over all costs. Storage market is really cut throat on profit margins..

The reality is SSD vendors are investing in how to make their controllers cheaper using less silicon as this lowers cost be this for business or consumer market. Like a 1 cent saving over a volume of 100 million units is still 1 million dollars.

Yes the storage vendors are doing a lot of work to make business customers happy with ZNS as long term it save storage vendor production cost.

[starshipeleven]

Except the problem of no DMA don't end at Sata drives. Think removable flash drives.

This is so far from being a priority that it's not even funny. USB flash drives can't be particularly fast because they need to be very small, USB hard drives have been converted to DM SMR (aka the bad kind of SMR) years ago since for that kind of market speed was not seen as major factor anyway.

You are making a lot of the same arguments people made before win-modems come the normal why dial-up modems would not lose their controllers.

Did they really? Modem hardware (dialup, DSL) has a dedicated controller and a real-time firmware handling the real meat of the job (frequency modulation and hardware-level part of the protocol) and is offloading very higher-level things to the OS, when the communication is well into the network layers and we start talking of packets.

But all the "modems" that still exist and are used to this day (dialup is dead in most of the first world countries) are actually modem-routers of some form, and there is an additional CPU where a crappy Linux firmware is running.

So at the end of the day to actually reach the Internet now you have your PC's CPU, the network card's controller, the modem-router's network controller, the modem-router's CPU and then the actual modem's controller. Are we sure it is less? Because it seems it is more.

Meanwhile 3/4/5G modems (outside of Android devices) are also running their own Linux OS to do these higher-level networking handling things, and again your PC has its CPU, then we encounter the modem's CPU, and then the modem's controller (many times it's just another core of the same CPU that is dedicated to run a VXWorks-based firmware).

We can also observe the same movement in Wifi hardware. The controller and firmware running in the actual wifi hardware are bigger and bigger and more powerful and handling more complex and abstract things.

Really, I think that your example with network devices is misplaced. It's a completely different thing.

Problem here DMA is really a shenanigans there is a long list of security exploits that lead back to DMA offloading to ram. There is also locations like over usb 3.0 and SATA where DMA is not an option either.

And as I said above, speed for "bulk storage" devices isn't a real priority since ages ago.

Round tripping from the NVME controller to the motherboard memory controller and back is not free. This is what starts work on ZNS. Doing zoned in the cpu and ram of the host you avoid a lot of round trips. Sorry that prised controller with dram that they can cheat different benchmarks with costs quite a bit.

Yeah, but developing a driver and keeping everything you make supported and not breaking anything with a driver that runs on Windows and Linux isn't cheap either. There is no standard interface between the SSD and the driver, because the only way to make the product of the day better is to have all kinds of wonky interfaces and APIs that change with device revision or model, and use those from their driver that abstracts all that crap into something the OS can work with.
With a device firmware and a mostly stable interface between that and the outside world you are ensuring that whatever device you make you can forget and stop supporting 5 months later when it is time to hype and sell the new version.

You need to think volume. If business sector is using ZNS controller chip does it really make sense to keep on making a non ZNS controller chip for the cheaper devices.

Maybe yes, maybe not. It's not a logical consequence. Plenty of businness-grade hardware and protocols have no equivalent for consumer, or have a very crappy and cut down version for consumers.

The classic example is SAS vs Sata. They are similar, but SAS is so much better on all points of view.

[oiaohm]

Maybe yes, maybe not. It's not a logical consequence. Plenty of businness-grade hardware and protocols have no equivalent for consumer, or have a very crappy and cut down version for consumers.

The classic example is SAS vs Sata. They are similar, but SAS is so much better on all points of view.

This here is the problem.

If the consumer version is having dram added its no longer a equal to business-grade or a cut down from business grade. Its upgraded compared to what business grade is using.

Consumer grade is normally built cheaper than server grade. When it not pressure will come down. Sata is a cutdown versions of SAS. You just think though the problem of starting with a SSD ZNS controller and cutting it down to make a consumer drive. Remember adding block device emulation requires upgrading the drive controller so adding cost and is basically in the wrong direction.

Linux well on way to having zoned device support built in. WD will most likely work out all the finer points of dm-zoned on Linux

There is no standard interface between the SSD and the driver

ZNS is written into the official standard. Of course at some point someone will want to use a ZNS from server in a desktop.

https://www.flashmemorysummit.com/Proceedings2019/08-06-Tuesday/20190806_ARCH-102-1_Bjorling.pdf

We are not talking small differences. ZNS is offering 5 times the performance with 5 times the life span to start off with vs a traditional SSD controller. Even your best SSD block emulation controller does not come close(yes that 5 times the performance of the best current day SSD controller with Dram on board). Next you are getting these performance gains with a DRAM-less controller even better with a less complex cpu in the controller. Finally lower over-provisioning this means you are able to put on the devices label a larger size that is really usable to the consumer.

The simple reality is SSD ZNS drives in every important metric kick current days SSD drives to the kurb. Only one problem is software support. This level of difference we can expect change. Really the block device emulation is horrible slow. Linux workstations when ZNS SSD come affordable you can fairly much bet Linux people will use them.

[Sin2x]

Reiser4 supported zstd in 2017.

[WorBlux]

https://zonedstorage.io/linux/fs/

As insane as it sounds XFS is being worked on behave correctly on zone based storage. The reality is all consume SSD are chunks of non-volatile storage that due to controller magic appear to be block device. Linux kernel has dm-zoned that is basically a cpu run equal to what has been in your SSD drive controller. This is really like the winmodem thing again.

...

Between dm-zoned and NVDIMM virtual block device stuff block device is going to be around well after we no longer have block devices in hardware. Of course most likely not the best performing choice using a file system designed for only block device.

I think you're mixing together ZNS and open channel SSD's somewhat. ZNS still requires an FTL on the drive controller, though the RAM requirments to hold the L2P table is a lot less, Additionally For supported filesystems or application you won't need to keep an FTL on the host . For unsported applications, Dm-zoned will keep a host-side FTL and remap access that can't deal with ZNS/zac/zbc directily., It reduces controller complexity and load, but isn't going to replace if fully. the controller any time soon. ZNS leverages the work already done for SMR drives, and adds zone append which is pretty cool and and speed up certain filesystem operations.

[oiaohm]

ZNS still requires an FTL on the drive controller,

https://www.usenix.org/sites/default/files/conference/protected-files/nsdi19_slides_bjorling.pdf

That wrong. With ZNS the FTL(Flash Translation Layer) on the drive controller is optional. ZNS operations do not need FTL as ZNS can directly align with the physical mappings of the SSD.

I think you're mixing together ZNS and open channel SSD's somewhat.

This is where you are badly wrong ZNS and open channel SSD are part of the same thing. Yes open channel SSD is a subset of all ZNS ssd drives. Open channel SSD starts with HDD’s SMR specification (ZAC/ZBC) the alterations to those specifications to suite SSD come ZNS.

SSD ZNS with FTL in the controller is the same as Host aware SMR HDD and a SSD ZNS without FTL (open channel) as Host Managed SMR HDD. So new open channel SSD drives are ZNS drives.

ZNS is the protocol to control open channel and other zoned SSD devices.

[WorBlux]

[url]ZNS can directly align with the physical mappings of the SSD.

That sounds like a bad idea unless the application has some understanding of wear leveling, the flash geometry, number of channels,media error checking, and the over-provision scheme. ZNS certainly makes these functions a lot easier, but most in mainstream filesystem applications, you'll have a (greatly simplified L2P mapping like 100MB/TB rather than 1GB/TB)) and some other FTL functions on the controller.

This is where you are badly wrong ZNS and open channel SSD are part of the same thing. Yes open channel SSD is a subset of all ZNS ssd drives.

You've got it backwards, ZNS is a subset of the latest open channel work. The Open Channel work started well before ZNS was fully formed, and then they noticed similarities with SMR work and decided it would be nice to leverage a single interface. And my perspective ZNS + basic FTL on the controller is going to have the widest usability, though lacking the full determinism of entirely open channel SSD's would provide in specific applications.

Open channel SSD starts with HDD’s SMR specification (ZAC/ZBC) the alterations to those specifications to suite SSD come ZNS.

SSD ZNS with FTL in the controller is the same as Host aware SMR HDD and a SSD ZNS without FTL (open channel) as Host Managed SMR HDD. So new open channel SSD drives are ZNS drives.

ZNS is the protocol to control open channel and other zoned SSD devices.

I disagree. First is that nobody is actually making a host-aware SMR. It's a terribly awkward idea in an awkward space. It won't enforce sequential or even throw a warning back if you get it wrong. I think a hybrid SMR is a better solution in the "is or isn't the host compatible?" game.

Just ZNS (with a basic FTL on disk) is enough to put it just barely on the Host-Managed SMR side of, because zone-append allowed the host to manage some of the sequentialization with the zone-append command. But you didn't necessarily want to spend precious CPU time serializing those writes in the first place.

But the full open channel with ZNS isn't any closer or further away. There's the same responsibility between the host to make sure writes can be serial within within a zone. You get more control of what actually goes where, and when (You know what zone is mapped where on the drive) but that doesn't really change how the zone-aware filesystems/drive mappers/applications interact with the NVME or the host-side FTL. And as I mentioned earlier, I only see it being used in some fairly specialty applications.

[oiaohm]

That sounds like a bad idea unless the application has some understanding of wear leveling, the flash geometry, number of channels,media error checking, and the over-provision scheme. ZNS certainly makes these functions a lot easier, but most in mainstream filesystem applications, you'll have a (greatly simplified L2P mapping like 100MB/TB rather than 1GB/TB)) and some other FTL functions on the controller.

https://www.flashmemorysummit.com/Proceedings2019/08-06-Tuesday/20190806_ARCH-102-1_Malikowski.pdf

Page 4. Intel plan for ZNS. No FTL on ZNS drive controller at all the FTL is now pushed into the host OS.

You've got it backwards, ZNS is a subset of the latest open channel work. The Open Channel work started well before ZNS was fully formed, and then they noticed similarities with SMR work and decided it would be nice to leverage a single interface. And my perspective ZNS + basic FTL on the controller is going to have the widest usability, though lacking the full determinism of entirely open channel SSD's would provide in specific applications.

Read above PDF page 11. There is a reason why operationally Open Channel is a subset of ZNS because Open Channel end up with a wrapper on top that turns Open Channel into plain ZNS in the host. So operationally be the drive ZNS or Open Channel you treat it ZNS.

Operationally Open Channel is basically ZNS because it gets proxy 1 to 1 to ZNS by OCSSD/ZNS adaptor in the future stack going forwards.

[WorBlux]

Operationally Open Channel is basically ZNS because

Looks like the only ZNS drive you can get right now supports Open Channel 2.0 as well.

https://www.radianmemory.com/u-2-ssd-rms-350/

The thing to note on that flash summit presentation is that it's about the SPDK.

The Storage Performance Development Kit (SPDK) provides a set of tools and libraries for writing high performance, scalable, user-mode storage applications. It achieves high performance through the use of a number of key techniques:

• Moving all of the necessary drivers into userspace, which avoids syscalls and enables zero-copy access from the application.
• Polling hardware for completions instead of relying on interrupts, which lowers both total latency and latency variance.
• Avoiding all locks in the I/O path, instead relying on message passing.

The bedrock of SPDK is a user space, polled-mode, asynchronous, lockless NVMe driver. This provides zero-copy, highly parallel access directly to an SSD from a user space application. The driver is written as a C library with a single public header. See NVMe Driver for more details.

And I've never personally been in a spot where I thought "hey I wish I would pass this whole disk with a weird interface directly to an applications" P.11 seems to suggest any OCSSD can be cast to a ZNS semantic, but not necessarily the inverse. And the FTL (SPDK) discussed here doesn't directly manage low-level housekeeping functions, but rather provides a flat LBA view of top of a zone backing to an application.

Going back to the radian drive, it provides "idealized flash" in both modes, and engages in cooperative Garbage collection and wear leveling via relocation polling. There's certainly some about of translation and indirection going on with the drive rather than mere access to raw flash. Additionally their proprietary operating mode allows GC and wear leveling to be moved entirely device side. And I expect that should ZNS get on desktop drives

It also looks like something like bands can be exposed through the namespace mechanisms.

Another exhibit in relation to p.4 that you mention: figure 1. of http://cidrdb.org/cidr2020/papers/p17-picoli-cidr20.pdf that shows the ZNS FTL as either on host or on device.

So in conclusion... It seems you're correct that a ZNS drive can have features mapped 1:1 (more or less) to OCSSD However I don't think that's s not necessarily so.

And to get a a cheaper high-capacity desktop drive you'd need some sort of FTL on windows at least of the level of dm-zoned, but that would likely not support name-space->channel mapping or cooperative GC/wear-leveling, to avoid canibilizing sales in the enterprise space. And I don't really care that the such a drive isn't 99+% deterministic, I just don't want my 8-10TB drive needing 8 GB of on board flash. And using Linux and filesystems that aren't 25 years behind the state of the art, you'd bypass the need for the complicated Host FTL anyways.

Having an FTL on the host doesn't mean there's not another layer of abstraction below that on the drive, just that there's less of one. It seems that my mistake is overestimated how direct an OCSSD had to be.

Anyways this whole discussion if probably far beyond the level of detail that most people are going to care about. I'll just agree that at first approximation that ZNS=Open Channel, with the caveat that any ZNS drive actually in the consumer space is likely not going to have the full feature set enterprise ZNS drives will have.