Announcement

Collapse
No announcement yet.

Apple Designs New File-System To Succeed HFS+

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • #81
    Originally posted by ctwise View Post
    If you're looking for a little more detail on APFS - http://dtrace.org/blogs/ahl/2016/06/19/apfs-part1/
    Thanks.

    a quick tl;dr for others (the stuff in "" is quoted directly):

    -they claim it's from scratch, and it started in 2014, given the following featureset I'd say it's ok.
    -encryption
    -btrfs-like snapshots
    -btrfs-like volumes with their own encryption key and policies and stuff, this is probably useful for app sandboxing
    -hardlinks-like deduplication (copy one file and you are just copying metadata, not physical data, you are basically making links to the same data blocks), a feature they themselves admit to not have a specific use for. Making hardlinks instead of copies works in any decent filesystem since a while ago tho. The fact that this works only if done by file manager but not by commandline cp tells me the logic for this is not implemented at filesystem level, akin to most NTFS features.
    -designed with flash logical structure in mind (akin to f2fs), of course not suited for raw flash.
    -TRIM support (no duh)
    -latency QoS, some kind of logic that prioritizes user applications over background applications
    -no RAID support whatsoever in the filesystem itself, you must rely on hardware raid or whatever you were using until now (easymode development turned ON, understandable though, they don't have any RAID in their main products anyway)
    -"novel copy-on-write metadata scheme" (unknown), "APFS does not employ the ZFS mechanism of copying all metadata above changed user data which allows for a single, atomic update of the file system structure" (don't know enough to say if this is good or not)
    -there is a filesystem checker for apfs
    -no checksum whatsoever, (easymode development over 9000) "The APFS engineers I talked to cited strong ECC protection within Apple storage devices." (ECC checks happen all day and all the night in any decent disk controller since the dawn of time, dafuq is "strong"? Delegating again to hardware something that can't be dealt with by software?), "Apple engineers I spoke with claimed that bit rot was not a problem for users of their devices," (Breaking news: it's not for the average Windows or even linux user either)


    Heh, a modern filesystem for consumer devices, cool for its niche, but the lack of checksumming and RAID makes it completely unworthy of going beyond single-disk PCs and mobile segment.

    btrfs is still our lord and saviour. ALL HAIL BTRFS!!!!
    Last edited by starshipeleven; 20 June 2016, 11:31 AM.

    Comment


    • #82
      Originally posted by starshipeleven View Post
      The posix-compliant-ness affects the userspace interface of course.
      Being posix-compliant for a filesystem means supporting a bunch of features needed by posix programs to work on it.
      The only one that I remember atm is renaming or deleting an open file. On ext4 I can do it all the time and nothing breaks, on NTFS (on windows) nope.
      Look up on google, there are more requirements than this.
      Yup, no arguments here.

      As for the "what interfaces the kernel provides", consider that btrfs is also pretty self-contained (does not rely on mdadm, lvm, and company), it asks only the most basic kernel interfaces.
      This makes porting it in the same ballpark than porting ZFS on linux. Annoying but NOT ANYWHERE NEAR making something similar from scratch.
      I'm not certain about this. You might be right, but I'm genuinely not sure.
      IMHO, the massive layer violating nature of btrfs means that it has to tailor itself to linux-isms, rather than the well defined (even if not unchanging) system internal interfaces.
      ZFS, aiui, was a very special case. It was written with a compatibility layer (http://open-zfs.org/wiki/Platform_co...0on.C2.A0Linux) for, at least, a large amount of its code. I THINK this is why zfs, on linux, doesn't have a reclaimable page cache.
      A better example would be to look at the recent, partial, btrfs port to the window's environment. I haven't looked at that, at all, but how they managed that would be very demonstrative of the difficulties.

      On FreeNAS forums the standard is "don't go lower than 8GB as it can be hazardous" (this on a Unix system), better if more. And this ram isn't cache. It's used by the filesystem. Also in the official site http://www.freenas.org/hardware-requirements/

      Also, unless you have at least a 2-disk array you aren't using most of ZFS features (checksums can detect corruption but there isn't any redundancy to fix it), this is true also for btrfs, although with btrfs there is the "make a raid1 inside the same partition" option, which is a bit masochistic but would work.
      I don't know what is happening in freenas-land b/c according to http://www.freenas.org/hardware-requirements/ the 32 bit version of freenas that used UFS had a 4GB requirement. For a system that is supposed to be a well tuned dedicated file server, those requirements seem insane.
      As for making best use of zfs, I won't argue any of that. Btrfs, however, can save multiple copies of a data block on single device (using -d dup when making the fs).

      Current CoW filesystems are not suited for anything that isn't a server, ok with that.

      For "embedded" we need to make a difference, mobile devices are increasingly getting in collision with PC userbase (also for Apple, remember Cook saying their iPad Pro shoud replace PCs?), so in a near-ish future it might.

      Although the trend I'm seeing is more Google-ish, "providing devices with low storage and offering large cloud storage services", which isn't even bad (if you have a decent Internet connection), as handling RAIDs, making backups, and doing other kinds of shit is beyond the average user's skill level anyway.
      Data in the cloud is much less hard to lose from the common accidents that I see people lose data from (malware, hardware failure, idiots installing Linux wrong).
      Yeah, I was deliberating whether to specify "embedded mobile".
      The ipad pro, and the various surface products, are good examples of convergent devices (the surface, though, is REALLY just a laptop with a detachable keyboard). I have to imagine that apfs (or whatever it's called) had something like the ipad pro in mind. Smaller devices, like phones, especially iphones, have relatively simple requirements, but once you start using heavy local apps with multitasking and large working sets, you start to need a more expansive set of features.
      I also agree with your assessment of our cloud-based future. That's largely why I think that btrfs is simply here too late. The future,imho, are these clustering filesystems (ceph, iirc, has even expressed scalability concerns with the vfs/io interface) which store/distribute/checksum our data in the cloud, and scale across datacenters.


      EDIT:
      I just read your post above. I'm surprised they only started this in 2014, but zfs, with its far larger feature set, took only a year or two longer.
      I suppose this shows that if you have a DEDICATED (only working on this on thing) group (some number between two and much more, but small enough that everyone knows what everyone else is doing) of developers, you can make something as subtle as a new, non-trivial, fs in pretty short order.
      I still hope something comes of bcachefs
      Last edited by liam; 20 June 2016, 07:01 PM.

      Comment


      • #83
        Originally posted by liam View Post
        I also agree with your assessment of our cloud-based future. That's largely why I think that btrfs is simply here too late. The future,imho, are these clustering filesystems (ceph, iirc, has even expressed scalability concerns with the vfs/io interface) which store/distribute/checksum our data in the cloud, and scale across datacenters.
        I haven't done much reading on it, but btrfs appears to be the recommended filesystem for Ceph clusters. Don't they go hand-in-hand?

        Comment


        • #84
          Originally posted by fuzz View Post

          I haven't done much reading on it, but btrfs appears to be the recommended filesystem for Ceph clusters. Don't they go hand-in-hand?
          Depends on the deployment.
          From what I recall CERN deploys ceph with xfs.
          The ceph devs rely, currently, on btrfs for certain features, but still suggest xfs.
          The cephfs, which presents a fs interface to the ceph store, still isn't production ready.

          Comment


          • #85
            Originally posted by liam View Post
            I'm not certain about this. You might be right, but I'm genuinely not sure.
            IMHO, the massive layer violating nature of btrfs means that it has to tailor itself to linux-isms, rather than the well defined (even if not unchanging) system internal interfaces.
            ZFS, aiui, was a very special case. It was written with a compatibility layer (http://open-zfs.org/wiki/Platform_co...0on.C2.A0Linux) for, at least, a large amount of its code. I THINK this is why zfs, on linux, doesn't have a reclaimable page cache.
            Well, in the page you linked they talked of the modifications they made to ZFS to make it portable. "They" is "the people of the open-zfs project", not the original devs nor Oracle.

            You can also see how they said their ZFS on OSX is mostly a copy-paste of the linux port with a bunch of minor wrappers and things, so what works on linux can be adapted relatively easily.

            Then again it's a wiki so it's probably 87% wrong and lacks 271% of information.

            A better example would be to look at the recent, partial, btrfs port to the window's environment. I haven't looked at that, at all, but how they managed that would be very demonstrative of the difficulties.
            If their readme is telling the truth, it's a complete rewrite https://github.com/maharmstone/btrfs So I don't think it will help much.

            I also agree with your assessment of our cloud-based future. That's largely why I think that btrfs is simply here too late. The future,imho, are these clustering filesystems (ceph, iirc, has even expressed scalability concerns with the vfs/io interface) which store/distribute/checksum our data in the cloud, and scale across datacenters.
            well, I think there is and will really remain quite a bit of grey area between single-disk consumer device <---> bigass SAN.

            Apart from usual server use where a SAN is beyond overkill, for example home NAS market has simply exploded in the last years, and btrfs there is VERY nice. Netgear uses it already in their NAS lines. Most commercial NAS devices offer similar easy setups as the usual cloud providers.

            I just read your post above. I'm surprised they only started this in 2014, but zfs, with its far larger feature set, took only a year or two longer.
            Let me please remind you that designing something that isn't doing something particularly new (there are implementations of all features it does already, in various different filesystems, also opensource) is easier than just dashing into the unknown.

            These devs had the luxury of doing "lesson-learned" design, seeing what other projects did and what went wrong. Not saying they copied, saying they learned from other's mistakes.

            Also as I said, making a filesystem that does not tackle RAID nor checksumming is very easymode.


            Comment


            • #86
              good news

              Comment


              • #87
                Originally posted by liam View Post
                I'd say that apple, more often than not, deliver well designed products that are pretty forward looking.
                i'd say that apple, more often than not, just copies features from android

                Comment


                • #88
                  Originally posted by starshipeleven View Post
                  btrfs has the free space issue (due to it being a CoW filesystem, they still have not even placed some arbitrary "reserved space limits" like for example there are on ext2-3-4 for other reasons) and the fact that it isn't working beyond RAID1 (both big issues). Plus assorted hiccups.

                  f2fs is mostly feature-complete and relatively stable afaik.
                  brainfuck is also feature-complete, it does not make it usable. f2fs has no raid at all, so obviously raid is more usable in btrfs than in f2fs. Plus assorted hiccups.

                  Comment


                  • #89
                    Originally posted by alien View Post
                    btrfs is only usable if you have plenty of space to keep enough of it empty.
                    still more usable than f2fs. for example i use in on 3 drives, f2fs's raid1 support is unusable

                    Comment


                    • #90
                      Originally posted by pal666 View Post
                      brainfuck is also feature-complete, it does not make it usable.
                      brainfuck is designed for research or sado/masochism purposes, if you are not using it for research or to deal pain to people you are using it wrong.
                      Is a hammer usable as a screwdriver? No. You are using it wrong.

                      f2fs has no raid at all, so obviously raid is more usable in btrfs than in f2fs. Plus assorted hiccups.
                      is there something stopping me from using mdadm and LVM (or even hardware raid, for that matter) on f2fs?

                      Since f2fs does not have checksumming in any shape or form, there is no logic reason it needs to implement RAID or volume management at the filesystem level, and it can live happy with letting the usual suspects deal with that.

                      btrfs (and also ZFS for that matter) must implement RAID and volume management at the filesystem level because it is checksumming things, and letting mdadm and LVM work on it would screw up checksumming.

                      Comment

                      Working...
                      X