Apple Designs New File-System To Succeed HFS+

[name99]

Reinventing the wheel, again..

Did you say that when IBM created JFS? When SGI created XFS? When Sun created ZFS? When Samsung created F2FS?

Did it ever occur to you that the file system is the foundation of everything an OS does, and the ability to UNILATERALLY change the file system is thus extraordinarily important to an OS vendor? Consider how many changes Apple has introduced over the last 15 years that have more or less possible because they controlled the entire file stack:
Adding journaling. Spotlight and general indexing. Various metadata, including security relevant metadata and ACLs. Time Machine. File versioning. Compressed files. Fusion. etc etc.

Insisting that Apple use an "open" file system, and that that file system will absolutely support their needs for the next 25+ years, is to claim that no changes of similar magnitude can be expected, changes which will require at least some monkeying around with the file system.

A file system is not just the disk layout (the part that is [mostly] immutable, though even that can have new features added as long as older OS code is blind to them, and JHFS+ has done that), it is also the implementation code. And that implementation code may occasionally need to change substantially (for features like Fusion). There's very little to Apple's advantage in tying themselves to either an existing disk layout OR to an existing source base that they cannot rapidly unilaterally change.

Beyond all this, there's an element of mass stupidity to these supposedly technical complaints.
What has been the biggest change in storage over the past ten years? Easy --- the use of SSDs. But SSDs can be used acceptably as fast hard drives, with hard-drive optimized file systems, because the gap in speed between RAM and flash is still so large.
Now what's going to be the biggest change in storage over the NEXT ten years? Also easy --- the use of persistent RAM (technologies like Optane). But persistent RAM REQUIRES
(a) new usage models that enforce storage ordering via CPU cache-control instructions (as opposed to how storage ordering was done with SSDs and HDs)
(b) new models for both how the file system is laid out on "disk", and how it is accessed --- basically you want something that looks like, and with an API like, an in-memory database.

Don't you think Apple is perfectly well aware of these issues? Hell, the newest version of the ARMv8 ISA has instructions added for precisely this purpose, and Apple is no doubt implementing them on its next CPU or two right now. So WTF would Apple switch to ZFS or ext4 or btrfs or some other file system optimized for storage circa 2000, or even F2FS or something optimized for storage circa 2010, when they have the chance to introduce something optimized for storage circa 2020?

[name99]

My theory is that apple wants better support for distributing/syncing data in their FS. They added in iOS 10 the ability to view files across many devices. Maybe they want this support in the FS level.

distributing/syncing data across network connected devices isn't something supported in btrfs or ZFS. If I remember correctly.

Exactly. This is just one example of the sort of thing they probably want to add.
Right now they synchronize files through API (using things like NSFileCoordinator) because the UNIX file locking primitives or so utterly retarded and useless. I expect that, along with everything else they're doing (as described in my comment above) they're adding some sort of file/range locking facility at the OS level that actually WORKS, even if you do have to augment the standard UNIX system calls to access it.

[liam]

Well, in the page you linked they talked of the modifications they made to ZFS to make it portable. "They" is "the people of the open-zfs project", not the original devs nor Oracle.

You can also see how they said their ZFS on OSX is mostly a copy-paste of the linux port with a bunch of minor wrappers and things, so what works on linux can be adapted relatively easily.

Then again it's a wiki so it's probably 87% wrong and lacks 271% of information.

OK, perhaps that wasn't the best source.

If their readme is telling the truth, it's a complete rewrite https://github.com/maharmstone/btrfs So I don't think it will help much.

To me, it suggests either: that dev is a superstar or btrfs is very easy to port to the interfaces windows provides.

well, I think there is and will really remain quite a bit of grey area between single-disk consumer device <---> bigass SAN.

Apart from usual server use where a SAN is beyond overkill, for example home NAS market has simply exploded in the last years, and btrfs there is VERY nice. Netgear uses it already in their NAS lines. Most commercial NAS devices offer similar easy setups as the usual cloud providers.

I wasn't aware that the home NAS market had seen a substantial increase in sales.
It's also very surprising to hear that netgear uses btrfs. From their whitepaper it appears that they just started offering btrfs, so I hope they actually contribute to its development.

Let me please remind you that designing something that isn't doing something particularly new (there are implementations of all features it does already, in various different filesystems, also opensource) is easier than just dashing into the unknown.

These devs had the luxury of doing "lesson-learned" design, seeing what other projects did and what went wrong. Not saying they copied, saying they learned from other's mistakes.

Also as I said, making a filesystem that does not tackle RAID nor checksumming is very easymode.

I completely agree that implementing system_x is much easier if it wholly consists of solved problems. Of course, we are talking about a fs, in this case. They are notoriously difficult to get right. F2FS isn't breaking any new ground and it is still in very active development.

[starshipeleven]

To me, it suggests either: that dev is a superstar or btrfs is very easy to port to the interfaces windows provides.

To be fair, there is always the third possibility: "the dev has bitten more than it can chew".

Anyway, for the sake of providing more data points, there is a btrfs driver for UEFI bootloaders (afaik limited to read-only and can see only first volume of btrfs) and it works on my PC since that's bundled with rEFInd and I did try it on a btrfs opensuse install, and also for u-boot embedded bootloader (don't know the limitations, never used it myself, probably similar limitations)

I wasn't aware that the home NAS market had seen a substantial increase in sales.

It went more or less hand-in-hand with the growth of low-storage mobile devices like tablets and such. High-speed Internet is not common enough for the pure cloud approach to work.

But the big boost was when the integrated ARM cpu stopped sucking big way, and they started turning their NAS things into miniservers that could install "apps" to expand functionality. Synology (best brand for non-technical users imho) official app store for example (there are also unofficial stores of unofficial apps): https://www.synology.com/en-us/dsm/app_packages/all_app

Others technically lower-end ones like Zyxel saw their NAS firmwares systematically hacked to add applications by third parties.
Showing an uncommon level of intelligence for an embedded device manufacturer, they swiftly adopted the same method themselves integrating the package installer in the firmware and hosting repos of "official" packages (also letting easy modification of the repo list so adding third party repos is now easy).
It's so crucial now that when the lead dev of the third party repo told them on their forums that a specific thing (a technicality, a compilation option in 64k instead of 4k or something in the kernel) in their newer devices firmware (a 4-disk NAS box) would make package mainteneance much harder for them, Zyxel capitulated within months and removed that.

It's also very surprising to hear that netgear uses btrfs. From their whitepaper it appears that they just started offering btrfs

Afaik it's since their "OS6", released in 2013 that they use btrfs as default. Thay claim they use code from Oracle Linux, so their btrfs is safe and production-ready. I don't believe any of that after fun experiences with their older nas products that modified their firmware's mdadm so as to render impossible recovery of the data without another of their NAS, but hey.

so I hope they actually contribute to its development.

There is one netgear dev in btrfs mailing list, and something is being pushed by him every now and then. Not a major contributor, but they do have a token presence at least.

http://www.mail-archive.com/search?a=1&l=linux-btrfs%40vger.kernel.org&haswords=%40netgear.com&x=0&y=0&from=&subject=&datewithin=1d&date=&notwords=&o=relevance

[liam]

To be fair, there is always the third possibility: "the dev has bitten more than it can chew".

The fact that they got it working, even if only partially, makes me think preclude that possibility.
Honestly, I don't expect that port to ever support all of btrfs' features, and, I don't think, the dev expected that either.

Anyway, for the sake of providing more data points, there is a btrfs driver for UEFI bootloaders (afaik limited to read-only and can see only first volume of btrfs) and it works on my PC since that's bundled with rEFInd and I did try it on a btrfs opensuse install, and also for u-boot embedded bootloader (don't know the limitations, never used it myself, probably similar limitations)

Apparently refind uses the btrfs implementation from grub2, but ported to work on the refind framework.
I couldn't find any reference in the uboot tree for any btrfs support (though it does support ZFS).

It went more or less hand-in-hand with the growth of low-storage mobile devices like tablets and such. High-speed Internet is not common enough for the pure cloud approach to work.

But the big boost was when the integrated ARM cpu stopped sucking big way, and they started turning their NAS things into miniservers that could install "apps" to expand functionality. Synology (best brand for non-technical users imho) official app store for example (there are also unofficial stores of unofficial apps): https://www.synology.com/en-us/dsm/app_packages/all_app

Others technically lower-end ones like Zyxel saw their NAS firmwares systematically hacked to add applications by third parties.
Showing an uncommon level of intelligence for an embedded device manufacturer, they swiftly adopted the same method themselves integrating the package installer in the firmware and hosting repos of "official" packages (also letting easy modification of the repo list so adding third party repos is now easy).
It's so crucial now that when the lead dev of the third party repo told them on their forums that a specific thing (a technicality, a compilation option in 64k instead of 4k or something in the kernel) in their newer devices firmware (a 4-disk NAS box) would make package mainteneance much harder for them, Zyxel capitulated within months and removed that.

Afaik it's since their "OS6", released in 2013 that they use btrfs as default. Thay claim they use code from Oracle Linux, so their btrfs is safe and production-ready. I don't believe any of that after fun experiences with their older nas products that modified their firmware's mdadm so as to render impossible recovery of the data without another of their NAS, but hey.

That makes a lot of sense. Thanks for the background.
I can't believe that netgear was selling btrfs as a product three years ago
Also, great story about how they, basically, made mdadm act like hardware raid (where, if your card goes down, you lose your array unless you can get an identical replacement card). I'm sure neil brown loved that
This is a bit off topic but you seem to know a bit about these things, so I hope you don't mind if I ask you a question: I'm looking for a SMALL 4 bay nas case to store, occasional, backups from my main file/media server. It needs to be small b/c I'm going to store it in an out of the way place when not being used.
I've been looking for a long time to find something like those synology/drobo/qnap/etc. cases but the ones I've found seem to have issues (either with cooling or reliability).
Do you have any info that could be useful in this search?

Best/Liam

[liam]

Argh!
The green scream of unapproved!!!!!!!!!!!!!!!!!!!!!!!!!!!!!1

[liam]

To be fair, there is always the third possibility: "the dev has bitten more than it can chew".

The fact that they got it working, even if only partially, makes me think preclude that possibility.
Honestly, I don't expect that port to ever support all of btrfs' features, and, I don't think, the dev expected that either.

Anyway, for the sake of providing more data points, there is a btrfs driver for UEFI bootloaders (afaik limited to read-only and can see only first volume of btrfs) and it works on my PC since that's bundled with rEFInd and I did try it on a btrfs opensuse install, and also for u-boot embedded bootloader (don't know the limitations, never used it myself, probably similar limitations)

Apparently refind uses the btrfs implementation from grub2, but ported to work on the refind framework.
I couldn't find any reference inthe uboot tree for any btrfs support (though it does support ZFS).

It went more or less hand-in-hand with the growth of low-storage mobile devices like tablets and such. High-speed Internet is not common enough for the pure cloud approach to work.

But the big boost was when the integrated ARM cpu stopped sucking big way, and they started turning their NAS things into miniservers that could install "apps" to expand functionality. Synology (best brand for non-technical users imho) official app store for example (there are also unofficial stores of unofficial apps): https://www.synology.com/en-us/dsm/app_packages/all_app

Others technically lower-end ones like Zyxel saw their NAS firmwares systematically hacked to add applications by third parties.
Showing an uncommon level of intelligence for an embedded device manufacturer, they swiftly adopted the same method themselves integrating the package installer in the firmware and hosting repos of "official" packages (also letting easy modification of the repo list so adding third party repos is now easy).
It's so crucial now that when the lead dev of the third party repo told them on their forums that a specific thing (a technicality, a compilation option in 64k instead of 4k or something in the kernel) in their newer devices firmware (a 4-disk NAS box) would make package mainteneance much harder for them, Zyxel capitulated within months and removed that.

Afaik it's since their "OS6", released in 2013 that they use btrfs as default. Thay claim they use code from Oracle Linux, so their btrfs is safe and production-ready. I don't believe any of that after fun experiences with their older nas products that modified their firmware's mdadm so as to render impossible recovery of the data without another of their NAS, but hey.

That makes a lot of sense. Thanks for the background.
I can't believe that netgear was selling btrfs as a product three years ago
Also, great story about how they, basically, made mdadm act like hardware raid (where, if your card goes down, you lose your array unless you can get an identical replacement card). I'm sure neil brown loved that
This is a bit off topic but you seem to know a bit about these things, so I hope you don't mind if I ask you a question: I'm looking for a SMALL 4 bay nas case to store, occasional, backups from my main file/media server. It needs to be small b/c I'm going to store it in an out of the way place when not being used.
I've been looking for a long time to find something like those synology/drobo/qnap/etc. cases but the ones I've found seem to have issues (either with cooling or reliability).
Do you have any info that could be useful in this search?

Best/Liam

[starshipeleven]

The fact that they got it working, even if only partially, makes me think preclude that possibility.

Well, for me it needs to be usable or GTFO. Not necessarily feature-complete but must be stable and usable. Anything less is a failure.

Honestly, I don't expect that port to ever support all of btrfs' features, and, I don't think, the dev expected that either.

Likely. Windows filesystem drivers run in userspace (akin to FUSE on linux) unless they get the green light from MS and there is a lengthy process in that with money, companies, pages intentionally left blank and other fun corporate stuff. Probably the project is not more than a driver to read stuff off btrfs.

I couldn't find any reference inthe uboot tree for any btrfs support (though it does support ZFS).

Interesting. I was sure i read of patches for btrfs in its mailing list... but there is nothing. Nevermind.

(though it does support ZFS).

Good enough to support my general point.

I'm looking for a SMALL 4 bay nas case to store, occasional, backups from my main file/media server. It needs to be small b/c I'm going to store it in an out of the way place when not being used.
I've been looking for a long time to find something like those synology/drobo/qnap/etc. cases but the ones I've found seem to have issues (either with cooling or reliability).
Do you have any info that could be useful in this search?

Uhm, for "case" you mean full NAS, right?

Drobo is out because it uses a proprietary filesystem, it is basically a hardware raid.

These are the brands I know something of:
Synology and Qnap are the rockstars, many many features, tons of apps, easy to use, docs in many languages, but also more expensive. You don't seem to need them.
Zyxel is aimed more at a geek-ish public, but has many apps.
WD MyCloud devices are decent but lack any noteworthy feature beyond standard NAS featureset, and unless you go for single-core crap they are horrendosuly overpriced.

Some models from each brand are lemons, some aren't. Need to evaluate each model from reviews, there is no "good brand" and "bad brand" as far as hardware design goes.

I'd say to go with Zyxel's NAS540 based on good experiences with it (bigass 120mm fan), although it's probably not as small as you'd like (because bigass 120mm fan). It's around 230$.

Another possibility is a HP microserver Gen7, these things sell for less than 200$ and are a small server with ecc ram and so on (up to 16GB supported, not officially anyway), there is also a very active community with a wiki still up, that hacked their bios to remove retarded HP restrictions and unlock another Sata port (used for a ODD bay which I'm not sure what the hell is doing in a headless system) and allow the usage of the esata port to extend the RAID too (total 6 sata ports).
It's very liked by low-end FreeNAS users, and in general rapes any other 4-bay (and many 6-bay ones too) NAS under the sun as it has a decent AMD x86 processor instead of a meh ARM thing.
There are also newer microservers from HP but they are more expensive.


Quite frankly for your usecase (occasional backups and lives most of its life in storage and not in use, you basically want a bigass usb 3.0 hard drive, not a NAS) I'd say you should also look at an external enclosure with sata multiplier instead.

While most 4-bay NAS devices nowadays can nearly saturate Gigabit with reads and have decent write speeds over simple samba, they still suck balls in backup speed over rsync and similar network backup. Speeds between 20 and 40 MB/s are common.

With an external enclosure with a multiplier (NOT those that make a hardware raid) you just make a soft raid from your server and then use its vastly more powerful CPU to move data around.

Like this thing for example, http://www.pc-pitstop.com/sata_port_...s/scsat05b.asp
It's dumb as a brick so it's 5 external drives for 160$ .

Although with external enclosures you usually need to use a pcie sata controller card that supports the feature, so you need to get one of such cards for 30-40$, and the price goes to the same as a NAS.

[liam]

Well, for me it needs to be usable or GTFO. Not necessarily feature-complete but must be stable and usable. Anything less is a failure.

Depends on the use case, I would think.
If the intent is to run btrfs volumes IN windows then, yeah, you want a fully functional solution.
I thought the intent, however, was to just provide basic access to btrfs volumes from windows analogous to the fuse-ntfs use case, for instance.

Good enough to support my general point.

Well, it depends on whether or not they used that compatibility layer

Uhm, for "case" you mean full NAS, right?

Dammit, I've really had an issue with clearly communicating myself, recently.
I mean a "case", as in: "I want to install the OS myself". I don't think I'm going to go with traditional RAID (well, I'll probably mirror it while I'm using just two disks, but once I add more disks I think I'll move to snapRAID since that doesn't have the perils associated with RAID rebuilds.

The use case for this is that it is my actual backup to my always online raid-like storage array.
The backup doesn't need to mirror everything, so it can afford to be smaller.

Drobo is out because it uses a proprietary filesystem, it is basically a hardware raid.

These are the brands I know something of:
Synology and Qnap are the rockstars, many many features, tons of apps, easy to use, docs in many languages, but also more expensive. You don't seem to need them.
Zyxel is aimed more at a geek-ish public, but has many apps.
WD MyCloud devices are decent but lack any noteworthy feature beyond standard NAS featureset, and unless you go for single-core crap they are horrendosuly overpriced.

Some models from each brand are lemons, some aren't. Need to evaluate each model from reviews, there is no "good brand" and "bad brand" as far as hardware design goes.

I'd say to go with Zyxel's NAS540 based on good experiences with it (bigass 120mm fan), although it's probably not as small as you'd like (because bigass 120mm fan). It's around 230$.

That zyxel is actually perfect, as far as size goes. The best cases I've found have been these two:

http://www.casetronic.com/corporates/141-c254.html

http://www.in-win.com.tw/Server/zh/goods.php?cid=145

I'm also considering some of the smaller fractal design and lian li offerings, but I'd really prefer having easy access to the hdd bays.

Another possibility is a HP microserver Gen7, these things sell for less than 200$ and are a small server with ecc ram and so on (up to 16GB supported, not officially anyway), there is also a very active community with a wiki still up, that hacked their bios to remove retarded HP restrictions and unlock another Sata port (used for a ODD bay which I'm not sure what the hell is doing in a headless system) and allow the usage of the esata port to extend the RAID too (total 6 sata ports).
It's very liked by low-end FreeNAS users, and in general rapes any other 4-bay (and many 6-bay ones too) NAS under the sun as it has a decent AMD x86 processor instead of a meh ARM thing.
There are also newer microservers from HP but they are more expensive.

Wow, that's is pretty damn great info.
I'd already looked at these (well, the gen8), and I'm still considering them, but I hadn't heard about the bios restrictions being "lifted" (those restrictions had been a big reason why I didn't just snatch one of these and call it a day).
I've only managed to find one of these (a "new" one) for less than ~$450 on ebay, and that was from someone with less sales than I'm usually comfortable with (http://www.ebay.com/itm/HP-ProLiant-...cAAOSwGIRXbgz4). It's also unclear if the Turion would be a bottleneck for the hash.

Quite frankly for your usecase (occasional backups and lives most of its life in storage and not in use, you basically want a bigass usb 3.0 hard drive, not a NAS) I'd say you should also look at an external enclosure with sata multiplier instead.

While most 4-bay NAS devices nowadays can nearly saturate Gigabit with reads and have decent write speeds over simple samba, they still suck balls in backup speed over rsync and similar network backup. Speeds between 20 and 40 MB/s are common.

With an external enclosure with a multiplier (NOT those that make a hardware raid) you just make a soft raid from your server and then use its vastly more powerful CPU to move data around.

Like this thing for example, http://www.pc-pitstop.com/sata_port_...s/scsat05b.asp
It's dumb as a brick so it's 5 external drives for 160$ .

Although with external enclosures you usually need to use a pcie sata controller card that supports the feature, so you need to get one of such cards for 30-40$, and the price goes to the same as a NAS.

I'm aware of these (thanks to someone else on these forums), and I'm not against them (since I've not bought any parts for this backup). My concern is that I would be completely relying on another system, which has seen A LOT more use, for these ultimate backups.
It's probably not a reasonable concern, however

[starshipeleven]

That zyxel is actually perfect, as far as size goes. The best cases I've found have been these two:

http://www.casetronic.com/corporates/141-c254.html

http://www.in-win.com.tw/Server/zh/goods.php?cid=145

I'm also considering some of the smaller fractal design and lian li offerings, but I'd really prefer having easy access to the hdd bays.

You have already found the one I'd recommend. In-win.

I've only managed to find one of these (a "new" one) for less than ~$450 on ebay, and that was from someone with less sales than I'm usually comfortable with (http://www.ebay.com/itm/HP-ProLiant-...cAAOSwGIRXbgz4).

Here a FreeNAS thread about its speed

https://forums.freenas.org/index.php?threads/freenas-on-hp-microserver-amd-turion-2-neo54.16854/

Here the wiki with bios mods and other stuff http://n40l.wikia.com/wiki/HP_MicroServer_N40L_Wiki

As for not finding flocks on ebay, eh, used market is volatile (it seems to be mostly available in australia atm). As for the one you found, as long as you use Paypal it should be fine.

It's also unclear if the Turion would be a bottleneck for the hash.

Kinda. That's the same for most gen8 miniservers too as the Intel processors in the affordable ones do not have AES acceleration either.

I'm aware of these (thanks to someone else on these forums), and I'm not against them (since I've not bought any parts for this backup). My concern is that I would be completely relying on another system, which has seen A LOT more use, for these ultimate backups.
It's probably not a reasonable concern, however

Well, any other PC with proper sata expander support (i.e. with a pcie sata card that has that feature since it's not usually built-in in Intel chipsets) and drivers for the filesystem in the hard drives will be able to read and operate that just fine.
As long as you are using linux desktop PCs, it's nearly as portable as a usb hard drive, lol.