Announcement

Collapse
No announcement yet.

Google Is Exploring Potentially Using Btrfs In Android

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • #71
    Originally posted by aht0 View Post
    Android would probably refuse to work BEFORE you hit that low-end RAM limit for ZFS.
    This isn't a discussion about Android.

    Meanwhile, btrfs works fine in my nas that has 512 MiB of RAM total (running LEDE or Debian).

    It's misconception like "ZFS was designed hurriedly and corners were cut all the time".
    Although it's certainly case with the Btrfs.
    You mean that ZFS was designed from the start to allow online grow/shrink and change of array type (raid0 to raid6 and back) and also to allow to go beyond RAID concept allowing the user to decide parity, striping and whatnot, and do so on the fly?
    Because I always thought it was btrfs that was designed to allow that, and that ZFS added something basic like array shrinking quite a bit of time after first release in production.

    That's the main reason it is taking more time to develop. It's not cutting corners.

    Comment


    • #72
      Originally posted by starshipeleven View Post
      This isn't a discussion about Android.
      check the title again. "Google Is Exploring Potentially Using Btrfs In Android"

      Originally posted by starshipeleven View Post
      This isn't a discussion about Android.
      Meanwhile, btrfs works fine in my nas that has 512 MiB of RAM total (running LEDE or Debian).
      Until first file-system-destroying-corruption-case. Stories about btrfs stability are generally just so much fairytales by it's apologists. In reality, install it on some machine, give it to average user and inside of a month, he or she is back with machine you'd have to fix again. Unless you do the installation only employing strictly "safe" features and do a training course for the user of "what to do when.."

      Btw, ZFS would run on machine with a 512Mb of RAM. Pseudo-argument.
      Add more ram if you need to. It's dirt cheap. Even unbuffered-ECC, I have 20 GB (1x8,2x4,2x1) worth of those unbuffered-ECC sticks in my drawers, no real point trying to sell on ebay or on local aftermarket. How much disk space your home NAS does employ? 1TB-6TB? 8GB RAM would suffice and that would cost what? 30 euros?

      Originally posted by starshipeleven View Post
      You mean that ZFS was designed from the start to allow online grow/shrink and change of array type (raid0 to raid6 and back) and also to allow to go beyond RAID concept allowing the user to decide parity, striping and whatnot, and do so on the fly?
      Because I always thought it was btrfs that was designed to allow that, and that ZFS added something basic like array shrinking quite a bit of time after first release in production.
      You can grow the ZFS, it's not a problem. Exact mechanics of the process is not quite like you are used with. Shrinking? Problem tends to be the opposite usually - running out of space. So you dug up pseudo-issue. Again.

      If you need to change the type of raid on the fly, maybe you should be less rash and have given more thought about what you need before setting up the raid. There is quite a difference between raid 0 and 6. Pseudo-argument and symptom of feature creep at once.
      Originally posted by starshipeleven View Post
      That's the main reason it is taking more time to develop. It's not cutting corners.
      Sounds like an apologism. My impression is the exact opposite. Design phase on "drawing board" was woefully short for BTRFS. Designers took likely a peek at ZFS and ignored things they could not understand (for example WHY is ZFS 128bit).

      Then came period of feverish coding and adding feature after feature AFAP. FS proved pretty unreliable, then development slowed down, now it seems to have picked up again and at least it seems bug fixing is more important than cramming in new features.

      It's been developed now what? 10 years? And is still in the phase, where - if you wanted to use it, you would have to make sure you do not apply use case a, use case b, use case c because it would potentially endanger your data. While, where ZFS does not have gazillion extra fancy features of btrfs - it's old, proven itself and works.
      Last edited by aht0; 06-18-2017, 08:35 AM.

      Comment


      • #73
        Originally posted by aht0 View Post
        check the title again. "Google Is Exploring Potentially Using Btrfs In Android"
        Yeah, it is about btrfs in Android.
        BTRFS, running on Android.

        The fact that Android wastes ram for its own use is irrelevant because that ram won't be available to either btrfs or ZFS.

        Until first file-system-destroying-corruption-case. Stories about btrfs stability are generally just so much fairytales by it's apologists. In reality, install it on some machine, give it to average user and inside of a month, he or she is back with machine you'd have to fix again.
        Yeah we get it, you need to shit on btrfs to show how good is ZFS, and can't do a fair-headed comparison.

        Unless you do the installation only employing strictly "safe" features and do a training course for the user of "what to do when.."
        I'm pretty sure mobile won't need RAID5/6 or aggressive snapshotting (100+ snapshots), just saying. It's actually the lack of encryption the main thing they asked.

        Btw, ZFS would run on machine with a 512Mb of RAM. Pseudo-argument.
        Not in a raid configuration though. I was talking of RAID1, the most a mobile device might actuallty need (imho it's unlikely they need RAID1, they are more likely after snapshots and subvolumes.

        Add more ram if you need to. It's dirt cheap.
        On a Zyxel nsa325v2? It's an embedded device.
        I made a comparison with it because it is NOT comparable with big iron servers ZFS was designed for.

        Modern PCs are more or less what old big iron servers were, so no duh it runs fine there, you don't say?

        How much disk space your home NAS does employ? 1TB-6TB? 8GB RAM would suffice and that would cost what? 30 euros?
        30 euros plus the price of a PC-like device I can plug it into, and I think it's beyond overkill for a 2TB RAID1 with 2 drives.

        But I'm not discussing my own home nas here, it was just to say that btrfs RAID1 can run fine and with acceptable performance (kinda half of mdadm raid with ext4 on the same hardware) on total crap hardware like my home NAS I hacked to run something more advanced than stock firmware.

        You can grow the ZFS, it's not a problem. Exact mechanics of the process is not quite like you are used with. Shrinking? Problem tends to be the opposite usually - running out of space. So you dug up pseudo-issue. Again.
        Yeah yeah, let's handwave away stuff ZFS can't do and say it's useless because True Men never do that anyway.

        If you need to change the type of raid on the fly, maybe you should be less rash and have given more thought about what you need before setting up the raid. There is quite a difference between raid 0 and 6. Pseudo-argument and symptom of feature creep at once.
        Still handwaving.
        Of course production environments might want to move between RAID10 to RAID5 or RAID6 or change redundancy levels to find the one that gives them the best performance with their specific workload, while for other more home-y users it's neat to be able to switch back and forth between RAID0-1-JBOD because y'know maybe people does not have multiple large-sized drives around to move data so he can nuke the array and rebuild it.

        Sounds like an apologism. My impression is the exact opposite. Design phase on "drawing board" was woefully short for BTRFS. Designers took likely a peek at ZFS and ignored things they could not understand (for example WHY is ZFS 128bit).
        That's because it was designed for big iron servers in a time when clusters weren't yet a thing, so they wanted to be dead sure that they could keep it running in big iron servers doing the same thing in the future. Considering how history turned out, that's useless overprovisioning.

        Nowadays, it makes far more sense to work with a cluster well before you get anywhere near 64-bit limit on a local fs like btrfs/ext/xfs, so you'll want some cluster-grade fs for that, like Ceph or whatever.

        Then came period of feverish coding and adding feature after feature AFAP. FS proved pretty unreliable, then development slowed down, now it seems to have picked up again and at least it seems bug fixing is more important than cramming in new features.
        That's because they needed to have at least a basic mockup of all that was supposed to be in the final fs to make sure other features didn't cut away corners and block implementation of them.
        I'm sure that ZFS also had a similar stage behind closed doors in Oracle, back then when it wasn't released yet.
        Being opensource has this effect, people see the software well before it would be ready for a "first release" in commercial products, so they think it must be horrifically bad or that it is taking a very long time.
        Most closed stuff takes a long time too and goes through phases where it is a mess, bit you don't see that because it happens behind closed doors.

        Hell, the same happens for those that pay for a kickstarter for a game and don't know that development times for a decent non-indie PC game are 4-6 years. And then go and whine.

        It's been developed now what? 10 years? And is still in the phase, where - if you wanted to use it, you would have to make sure you do not apply use case a, use case b, use case c because it would potentially endanger your data. While, where ZFS does not have gazillion extra fancy features of btrfs - it's old, proven itself and works.
        And is still squarely aimed at devices that are similar to what were big iron servers when it was designed, with little way of changing that. This still makes it unworthy of consideration from Google for Android, it seems. The butthurt is strong, I feel it.

        Comment


        • #74
          Originally posted by starshipeleven View Post
          Can I point out that Oracle is also contributing to Linux kernel too so they do have a standing to bite if they feel like it?
          Sure, but Oracle can already sue you for shipping proprietary drivers then. No additional risk for using ZFS.

          Comment


          • #75
            Originally posted by starshipeleven View Post
            Yeah, it is about btrfs in Android.
            BTRFS, running on Android.
            Decide for once. Post before, you did claim "This isn't discussion about Android".

            You can't have it both ways at once, depending how it happens to suit you.

            Originally posted by starshipeleven View Post
            The fact that Android wastes ram for its own use is irrelevant because that ram won't be available to either btrfs or ZFS.
            Technically, Android can't sometimes even use the whole RAM the phone has these days. My Galaxy S7 has 4GB of RAM. Because it's running 32bit Android it can't fully utilize all of it. Lack of RAM is least of problems in modern Android devices

            Originally posted by starshipeleven View Post
            Yeah we get it, you need to shit on btrfs to show how good is ZFS, and can't do a fair-headed comparison.
            Stating that it is unusable unless you avoid using "unstable" features of Btrfs was "shitting on it"? Or stating that it's design phase was rather short?
            What I see as a main issue: apologists of btrfs are trying to show it in better than it technically is.

            Result: possibly screwing over users who are trusting your claims. "Sort of works and it's OKAY" is general attitude backfiring badly on btrfs, linux and free software as a whole. And it's a serious problem among Linux crowd as a whole.

            Originally posted by starshipeleven View Post
            I'm pretty sure mobile won't need RAID5/6 or aggressive snapshotting (100+ snapshots), just saying. It's actually the lack of encryption the main thing they asked.
            Not in a raid configuration though. I was talking of RAID1, the most a mobile device might actuallty need (imho it's unlikely they need RAID1, they are more likely after snapshots and subvolumes.
            Originally posted by starshipeleven View Post
            On a Zyxel nsa325v2? It's an embedded device.
            I made a comparison with it because it is NOT comparable with big iron servers ZFS was designed for.
            You did not speak WORD about it being embedded device. Could have been some machine running 512Mb of RAM dating back to 2002 for all I could deduce from your previous posts.

            Originally posted by starshipeleven View Post
            Modern PCs are more or less what old big iron servers were, so no duh it runs fine there, you don't say?
            "old big iron servers" is your own misconception.

            ZFS came out 2004. Not in 1994. Back in 2004 configurations involving 1GB+ RAM were pretty standard for PCs. It was that time where 64bit CPUs started making their appearance and people started running into limitations of 32bit OS. Sun did not work only with servers but had also plenty of workstations on it's production line.

            Originally posted by starshipeleven View Post
            30 euros plus the price of a PC-like device I can plug it into, and I think it's beyond overkill for a 2TB RAID1 with 2 drives.
            Suit yourself. Though your embedded NAS probably cost multiple times more than some low-power PC you could hammer together. For 2TB couple gigs would suffice and that would be like 5 pounds (bought 2x1GB sticks from UK year a go for this price).
            You probably spend more in a grocery store per day.

            Originally posted by starshipeleven View Post
            But I'm not discussing my own home nas here, it was just to say that btrfs RAID1 can run fine and with acceptable performance (kinda half of mdadm raid with ext4 on the same hardware) on total crap hardware like my home NAS I hacked to run something more advanced than stock firmware.
            Looking at it specs, dunno, isn't it rather seriously CPU bottlenecked?

            Originally posted by starshipeleven View Post
            Yeah yeah, let's handwave away stuff ZFS can't do and say it's useless because True Men never do that anyway.
            You honestly picked piss poor examples but suit yourself again.

            Originally posted by starshipeleven View Post
            Of course production environments might want to move between RAID10 to RAID5 or RAID6 or change redundancy levels to find the one that gives them the best performance with their specific workload, while for other more home-y users it's neat to be able to switch back and forth between RAID0-1-JBOD because y'know maybe people does not have multiple large-sized drives around to move data so he can nuke the array and rebuild it.
            What's stopping me from nuking it without bothering with a rebuild, configuring the "new" raid and copying data back to NAS from backup? It makes out much the same, lol. Feature creep.
            If they do not have backup, well, they are going to regret it soone or later, raid is not a backup system.

            Originally posted by starshipeleven View Post
            That's because it was designed for big iron servers in a time when clusters weren't yet a thing, so they wanted to be dead sure that they could keep it running in big iron servers doing the same thing in the future. Considering how history turned out, that's useless overprovisioning.
            Misconception.

            Originally posted by starshipeleven View Post
            I'm sure that ZFS also had a similar stage behind closed doors in Oracle, back then when it wasn't released yet.
            Being opensource has this effect, people see the software well before it would be ready for a "first release" in commercial products, so they think it must be horrifically bad or that it is taking a very long time.

            Most closed stuff takes a long time too and goes through phases where it is a mess, bit you don't see that because it happens behind closed doors.
            Sigh..
            ZFS: 2001-2004. From idea to ready initial product. It was closed source until OpenSolaris 2005. More features were added later, gradually.
            Btrfs: Autumn 2007 - late 2008. From presented idea to finalized on-disk format. Tell me it wasn't "hurry-hurry".

            If devs had continued by adding features gradually, tested/finished them properly and before cramming in more new ones - I would not tell single bad word about it. As it was, you got what was to be expected. File system that had destructive behaviour and did it suddenly and when you least expected..

            Just for shit and giggles, I am going to install btrfs on home NAS using RAID setup (I am using PC for it) and see how long it lasts. Probably by using OpenSUSE or Manjaro for it.
            Last edited by aht0; 06-19-2017, 02:12 PM.

            Comment


            • #76
              Originally posted by chithanh View Post
              Sure, but Oracle can already sue you for shipping proprietary drivers then. No additional risk for using ZFS.
              They sue when they feel like they could make the most money.

              For example they went omnomnomnom on Google when Android was a huge success and they could get some pretty large amount of cash from the litigation, not just after they started using Java in Android and it was still worth pretty much shit (like say 2011).

              Comment


              • #77
                Oracle also lost that court case AFAIK (about Java).

                Google could technically screw Oracle over (ZFS-wise) if Google could change the Android's license from GPL to something that is CDDL-compatible. Does not have to be BSD license though Google has used this one in Android as well (Bionic)

                I do not actually advocate "need ZFS in my phone". Perfectly happy with ext4. Just theoretical.
                Last edited by aht0; 06-19-2017, 12:16 PM.

                Comment


                • #78
                  [QUOTE=aht0;n958199]Decide for once. Post before, you did claim "This isn't discussion about Android".[QUOTE]If the discussion is about btrfs on android it is not about Android.
                  I think I said this discussion is about btrfs on Android, which is also what is stated in title, don't try to pretend I'm inconsistent.

                  If someone says he is talking about rabbits in forests, then the main subject is rabbits, not forests. Please improve your reading.


                  Technically, Android can't sometimes even use the whole RAM the phone has these days. My Galaxy S7 has 4GB of RAM. Because it's running 32bit Android it can't fully utilize all of it. Lack of RAM is least of problems in modern Android devices
                  ^
                  |
                  Takes as an example of Android a flagship device that costs 600 euros, assumes that most other devices running Android are in a similar situation, requires external assistence to figure out that it's nonsense.


                  Stating that it is unusable unless you avoid using "unstable" features of Btrfs was "shitting on it"? Or stating that it's design phase was rather short?
                  You were exaggerating it, and you know it, c'mon.

                  Result: possibly screwing over users who are trusting your claims. "Sort of works and it's OKAY" is general attitude backfiring badly on btrfs, linux and free software as a whole. And it's a serious problem among Linux crowd as a whole.
                  I never claimed that currently unstable or experimental features were stable, but RAID1/0 and RAID10 are fine.

                  RAID5/6 uses mostly different code as it deals with striping, not plain mirroring, so bugs in that code won't affect other modes.

                  You did not speak WORD about it being embedded device.
                  I'm a funny guy in a funny world. And we are talking of embedded devices in this thread about btrfs on Android.

                  "old big iron servers" is your own misconception.

                  ZFS came out 2004. Not in 1994.
                  Yeah, but the design is from 2001 and being closed source it might even be other than that.

                  Suit yourself. Though your embedded NAS probably cost multiple times more than some low-power PC you could hammer together.
                  As I said, I'm using it as an example of low-end crap still running btrfs decently, not as an example of most cost-effective hardware.

                  I didn't buy that for btrfs, I got it for like 50 euros (actually more because I bought it used with used drives inside whose price I subtracted here) and I did some tests with btrfs on it after I ported it to LEDE.


                  Looking at it specs, dunno, isn't it rather seriously CPU bottlenecked?
                  Correct, also ext4 + Samba maxes the CPU and decreases performance. ext4 + nfs runs better and nearly reaches 100 MB/s of output (less with writes).

                  With btrfs and NFS it's running at up to 40 of read speed, which considering even my 2 year-old 150 euro mediatek phone is like x4 as powerful just by core version (this is ARM v5, my phone is ARM v7), should bode well for btrfs on actual devices that also have far faster flash storage.

                  Kirkwood devices like it have crc32c acceleration (the checksum used by btrfs), but I don't know if it is using that or not.

                  What's stopping me from nuking it without bothering with a rebuild, configuring the "new" raid and copying data back to NAS from backup?
                  Downtime?
                  Don't know about you but reloading a multi-TB server backup can take up to days depending on how crappy is the backup medium (yes tapes I'm talking of you) and even a few hours of downtime can rise by quite a bit if you factor other operational tasks like putting a server offline without fucking up the service it is providing and so on.

                  Misconception. Sun also wanted to use it on it's workstations which ran much lighter hardware.
                  I'm sure that running it on workstations is kinda unrelated to it being a 128bit filesystem.

                  ZFS: 2001-2004. From idea to ready initial product. It was closed source until OpenSolaris 2005. More features were added later, gradually.
                  Btrfs: Autumn 2007 - late 2008. From presented idea to finalized on-disk format. Tell me it wasn't "hurry-hurry". If devs had added features, tested/finished them properly and added new ones gradually - I would not tell single bad word about it.
                  Why are you taking disk format as a date? You know what disk format is? It's just "from block 1 to 10 goes X, fromn block 20 to 21 goes Y" and so on.

                  You can do that pretty quickly, the hard part is making the software that moves data, not defining the pattern of data written on disk.

                  Comment


                  • #79
                    Originally posted by aht0 View Post
                    Oracle also lost that court case AFAIK (about Java).
                    Details, true evil will not be stopped by so little unimportant failures.

                    Google could technically screw Oracle over (ZFS-wise) if Google could change the Android's license from GPL to something that is CDDL-compatible. Does not have to be BSD license though Google has used this one in Android as well (Bionic)
                    Friendly reminder, Android is using Linux kernel (GPL) with Google's stuff in userspace that is Apache/BSD/permissive license anyway.

                    The most likely case here is Google switching to Fuchsia (has permissive license) and somehow deciding to port OpenZFS to Fuchsia. But since afaik the only ZFS implementations that have encryption are the closed ones from Quintessential Evil.... ehm I mean Oracle, it's far easier to go and add this feature to btrfs or see if they can get ext4's encryption to work or whatever.

                    Comment


                    • #80
                      Originally posted by starshipeleven View Post
                      Details, true evil will not be stopped by so little unimportant failures.

                      Friendly reminder, Android is using Linux kernel (GPL) with Google's stuff in userspace that is Apache/BSD/permissive license anyway.

                      The most likely case here is Google switching to Fuchsia (has permissive license) and somehow deciding to port OpenZFS to Fuchsia. But since afaik the only ZFS implementations that have encryption are the closed ones from Quintessential Evil.... ehm I mean Oracle, it's far easier to go and add this feature to btrfs or see if they can get ext4's encryption to work or whatever.
                      Friendly reminder, while Android kernel is GPL, C library between it and userland is BSD licensed Bionic (portions of which btw originate from Open- and FreeBSD). So they could technically change Android's license to whatever the fuck they wanted, as long as they own all the rights for the Android's own sources.

                      Wrong. FreeBSD has working ZFS encryption through GELI. And encryption means encryption, not leaving portions of file system (like kernel) uncovered.
                      While FreeBSD's ZFS encryption is not directly usable on Linux it does show you do not specifically need Oracle for achieving that goal.

                      About Evilness. "Evil" assumes definition of morality and behaviour not adhering to it. Profit-driven corporate entity cannot be neither "evil" or "good". It does not have moral. Neither have the corporate entities you consider "good". It all comes down to chosen business strategy which reflects back in people's opinion about them. As long as people who think them "bad" cannot directly influence their profits - they could not care less.
                      Last edited by aht0; 06-20-2017, 08:10 AM.

                      Comment

                      Working...
                      X