Western Digital Has Been Developing A New Linux File-System: Zonefs

[stormcrow]

I am just curious about something...

I don't mean to support them, but, what would the government do with that data, if it is benign?

Do they have an evil plan?

At this point this is almost becoming ridiculous... posts like the one you're responding to. This guy shouldn't even be using a computer if he's posting conspiracy drivel like that. Don't feed the trolls.

The ability to audit every level of a computer from the circuit level to the UI by a single person disappeared in the 1980s. Some level of trust is required to utilize any computer system at all these days. This particular guy is going on about an open source file system designed by the same company that probably built the hard drives in his computer... assuming he's not using a Z-80 hand built computer via 300 baud modem.

To me it reads like deliberate trolling, however.

Now, all that said, I've been hoping to see someone come up with a filesystem that better fits shingled drive media. Looking forward to what becomes of it. Hopefully the BSDs can take concepts and build their own such file systems.

[oiaohm]

What's the benefit of zoned storage over normal storage?

Do you have a SSD drive? If so you are using like zoned storage right now hidden behind a controller to pretend not to be like zoned storage.

https://www.extremetech.com/extreme/210492-extremetech-explains-how-do-ssds-work

You file-system is like doing 4kb sectors. The reality in a SSD you have blocks normally 256KB or 4MB. These blocks have to be erased as a whole but can be added to. Basically the core tech of a SSD behaves like a Shingled magnetic recording (SMR) harddrive. Except there is one very big difference we could be looking at a block size of few 100Meg or larger with SMR harddrives.

SSD block storage and SMR harddrives are about one thing increased storage density. Normal non zoned harddrive will not have the data tracks overlapped so will not be anywhere near the same data density. Same with blocks in SSD reducing down connections required.

Big difference is the size of SMR harddrive blocks this is not going to be simple or practical really to cover with the controller. SSD can stall in performance because the harddrive is busy reordering stuff was well. HDD stalling in performance while they order blocks is not going to be very nice.

Zoned is basically lets push it up a level to the file system and hope the file system has better idea of what were and can do the zone/block reworks more sanely.

[torsionbar28]

At this point this is almost becoming ridiculous... posts like the one you're responding to. This guy shouldn't even be using a computer if he's posting conspiracy drivel like that. Don't feed the trolls.

Some folks are so drawn to conspiracy theory nonsense, they'll concoct new ones on the fly to fit any discussion topic. Maybe they've watched too many X-Files episodes, who knows. It's comical, at best.

Now, all that said, I've been hoping to see someone come up with a filesystem that better fits shingled drive media. Looking forward to what becomes of it. Hopefully the BSDs can take concepts and build their own such file systems.

This SMR technology works fine for archives, backups, and other mostly static data. But I was really hoping they'd achieve more with it. In the capacities available today, SMR is not gaining much. When it was first introduced, I think SMR had something like a 25% capacity advantage. But that gap has been closing. It seems like it's only a ~1 year lag between when a new SMR capacity point ships, and when a standard perpendicular recording drive of the same capacity is released. In fact, I think today the highest capacity point for both SMR and perpendicular recording is 16 TB, so the capacity advantage of SMR has been reduced to zero.

[oiaohm]

This SMR technology works fine for archives, backups, and other mostly static data. But I was really hoping they'd achieve more with it. In the capacities available today, SMR is not gaining much. When it was first introduced, I think SMR had something like a 25% capacity advantage. But that gap has been closing. It seems like it's only a ~1 year lag between when a new SMR capacity point ships, and when a standard perpendicular recording drive of the same capacity is released. In fact, I think today the highest capacity point for both SMR and perpendicular recording is 16 TB, so the capacity advantage of SMR has been reduced to zero.

Sorry to say no.

https://www.westerndigital.com/products/data-center-drives/ultrastar-dc-hc600-series-hdd

Current SMR is 20TB.

https://www.westerndigital.com/products/data-center-drives/ultrastar-dc-hc500-series-hdd

Current perpendicular recording is 18TB.
Roughly a 10% improvement when buying drives. But if you get the density per area of platter the SMR is still 25% greater than perpendicular recording. Yes the WD 20TB SMR has less platter area than the WB 18TB perpendicular recording. So if the discs had identical platter area you would be expecting the SMR to be 22TB not 20TB.

The gap between the technologies really has not closed at all. Thinking SMR requires software stack changes to be effective companies like WD are not pushing the limited of what SMR can do because they really don't have the consumers yet and this is about getting the software stack changes done.

[starshipeleven]

With big tech giants being so close to various governments there is a 0% chance I would use a filesystem developed by them.

Guess you are back to stone age then

[torsionbar28]

Sorry to say no.
https://www.westerndigital.com/produ...600-series-hdd
Current SMR is 20TB.
https://www.westerndigital.com/produ...500-series-hdd
Current perpendicular recording is 18TB.

Sorry to say no. I'm talking about shipping products you can buy today. The 18 and 20 TB sizes are future products, not shipping until later in 2020.

The gap between the technologies really has not closed at all.

False. The fact is that 16 TB is the max drive size available today from any vendor. Precisely as I said in my original post. 16 TB drives are available in both SMR and PMR. Ergo, the capacity gap is zero.

Thinking SMR requires software stack changes to be effective companies like WD are not pushing the limited of what SMR can do because they really don't have the consumers yet and this is about getting the software stack changes done.

False. There are no software stack changes required. LBA48 takes us to 128 PiB per drive. And all modern filesystems and volume managers scale to petabyte range. Cloud and big data vendors are buying capacity as fast as it comes to market. Same for disk-based enterprise backup solutions. We just bought a 400 TB disk based backup solution at work last month. The customer base is more than ready and they are clamoring for more. This is not desktop peecee hardware.

[starshipeleven]

False. There are no software stack changes required.

Incorrect.
You don't need software stack changes if you are in "desktop peecee hardware" land and buy "disk-managed" SMR (aka the disk's own controller deals with more abstraction to make it look like a normal hard drive, similar to the SSD's own controllers).

If you are in the businness hardware, or "big boy toy" (to stay in line with your cringy naming) then yeah you find a lot more "host-managed" SMR, that of course have better performance, but need software stack changes as now they are expecting the system to treat them for what they truly are.
Linux software RAID does have specific "host-managed" SMR support code https://www.phoronix.com/scan.php?pa...=Linux-4.13-DM
RAID cards do need to be SMR-aware to use these drives properly "Microsemi has added support for the command set extensions used by host-aware and host-managed Shingled Magnetic Recording (SMR) hard drives." https://www.anandtech.com/show/11984...and-raid-cards

LBA48 takes us to 128 PiB per drive. And all modern filesystems and volume managers scale to petabyte range.

It's not about capacity, it's about different drive architecture that you have to paper over somehow because they are NOT the same as normal hard drives, the same issue SSDs face.

This "filesystem" is a "block layer filesystem" that is supposed to be used UNDER a normal filesystem, it's more similar to mdadm's raid volumes than a filesystem.

We just bought a 400 TB disk based backup solution at work last month.

Probably not "disk-managed" SMR. Unless it's cold storage (i.e. you expect any sort of write speed out of it) it's not likely to be SMR at all.

[oiaohm]

Sorry to say no. I'm talking about shipping products you can buy today. The 18 and 20 TB sizes are future products, not shipping until later in 2020.

That wrong. The two WD drives I quoted you can buy now as a enterprise customer. General market will be mid 2020. So they are shipping products to limited customers apparently you are not one of those limited customers. WD is currently the lead brand getting these products out door.

Seagate drives 18TB CMR/PMR will be the start of 2020 to enterprise with 20TB SMR end of 2020 to enterprise and possible consumers.

False. The fact is that 16 TB is the max drive size available today from any vendor. Precisely as I said in my original post. 16 TB drives are available in both SMR and PMR. Ergo, the capacity gap is zero.

There is so such thing as a 16TB SMR drive by any vendor. WD and Seagate made 14TB SMR drives never made a 16 TB. Toshiba has not done a SMR product for end consumer or enterprise yet. So you were talking about comparing two products with one product that has never existed.

https://www.anandtech.com/show/15064/seagate-18-tb-hdd-due-in-first-half-2020-20-tb-drive-to-ship-in-late-2020

Its on the chart here that there is new MAMR and HAMR coming we are basically at the limit PMR can do. 22TB is about all you can fit using SMR using PMR style read/write heads. 18TB is about the limit CMR/PMR. SMR can be used with MAMR and HAMR to give a 25% uplift in storage as well.

False. There are no software stack changes required. LBA48 takes us to 128 PiB per drive. And all modern filesystems and volume managers scale to petabyte range. Cloud and big data vendors are buying capacity as fast as it comes to market. Same for disk-based enterprise backup solutions. We just bought a 400 TB disk based backup solution at work last month. The customer base is more than ready and they are clamoring for more. This is not desktop peecee hardware.

Really you need to wake up on this very quickly. As starshipeleven pointed out your controllers and software need to understand SMR drives. If you insert SMR drives into a storage device made for CMR drives only you will end up with a mega screw up where yes the SMR accept LBA48 write instructions until you attempt to overwrite a already written area that you have not done the directions to clear. Lets just say having drive refusing to write because it not getting the extra instructions SMR drives need does not turn out well. Having a drive reported as failed when it perfectly fine is something that does happen with SMR drives when they are inserted into incompatible and can be cause of data loss if it like the N+1 drive.

Basically there are software and hardware changes required to be in place to use SMR drives and you cannot use SMR drives in production safely without them. Most drives enterprise are currently using are CMR/PMR not SMR/PMR.

[torsionbar28]

If you are in the businness hardware, or "big boy toy" (to stay in line with your cringy naming) then yeah you find a lot more "host-managed" SMR, that of course have better performance, but need software stack changes as now they are expecting the system to treat them for what they truly are.

Which is the "cringy" naming? Enterprise disk-based backup? That's not my naming, that's common terminology for the industry and market segment. What would you call it? And no, it doesn't use SMR. We're talking about backup solutions here. *Archiving* solutions (aka WORM data) use SMR, but not backup where the media is regularly updated and re-written.

Probably not "disk-managed" SMR. Unless it's cold storage (i.e. you expect any sort of write speed out of it) it's not likely to be SMR at all.

Ding ding ding. It isn't SMR at all because backup solutions don't use SMR. Archive solutions do. Different product segment entirely. Backup is used to recover from data loss scenarios. Archive is for historical record, often due to regulatory compliance. I'm talking about the former, no SMR to be found here.

[torsionbar28]

That wrong. The two WD drives I quoted you can buy now as a enterprise customer. General market will be mid 2020. So they are shipping products to limited customers apparently you are not one of those limited customers. WD is currently the lead brand getting these products out door.

Sorry, you don't understand how product launches work. Find me a part number for an 18 or 20 TB drive. You cannot, because they don't exist. The industry terminology "shipping to limited customers" does not mean what you think it means.

Seagate drives 18TB CMR/PMR will be the start of 2020 to enterprise with 20TB SMR end of 2020 to enterprise and possible consumers.

I.e. they don't exist *today*. If you read my original post, you'll see that magic word "today".

There is so such thing as a 16TB SMR drive by any vendor. WD and Seagate made 14TB SMR drives never made a 16 TB. Toshiba has not done a SMR product for end consumer or enterprise yet. So you were talking about comparing two products with one product that has never existed.

Not sure how this helps your argument. So PMR drives are *larger* than SMR drives today? If so, the capacity gap isn't zero, it's a negative number, LMAO.

Really you need to wake up on this very quickly.

That's funny. I've been employed in the enterprise data storage market since 1993. Were you even born yet? Serious question.

As starshipeleven pointed out your controllers and software need to understand SMR drives. If you insert SMR drives into a storage device made for CMR drives only you will end up with a mega screw up where yes the SMR accept LBA48 write instructions until you attempt to overwrite a already written area that you have not done the directions to clear. Lets just say having drive refusing to write because it not getting the extra instructions SMR drives need does not turn out well. Having a drive reported as failed when it perfectly fine is something that does happen with SMR drives when they are inserted into incompatible and can be cause of data loss if it like the N+1 drive.

Basically there are software and hardware changes required to be in place to use SMR drives and you cannot use SMR drives in production safely without them.

No, that's not how it works. I've tested SMR drives in RAID configurations in the lab at work. There is no failure or error condition. There is a performance penalty, yes, but that's not much different from SSD's and TRIM. The fact is, there are no software or hardware changes *required*. None. There are changes that are certainly desirable to optimize performance for this new SMR recording method, for sure, but there is no hard requirement. Yes I'm sure you could get some RAID implementations to break by rapidly overwriting SMR sectors, but lets be real, no sane person is attempting this use case, so it's a purely academic point.

Most drives enterprise are currently using are CMR/PMR not SMR/PMR.

Quite true. And if you re-read my posts, you'll find I was speaking very specifically about enterprise products. Quite frankly I don't give a crap about SMR drives because the price/performance simply isn't there for the vast majority of use cases. It's a niche technology with little to no benefit over PMR - which is precisely what each of my previous posts has alluded to.