Announcement

Collapse
No announcement yet.

How Google's Android Maintains A Stable Linux Kernel ABI

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • #41
    So why can't Android userland be updated while the kernel remains the same manufacturer-supplied kernel?

    Comment


    • #42
      Originally posted by eigenlambda View Post
      So why can't Android userland be updated while the kernel remains the same manufacturer-supplied kernel?
      Occasionally Google updates parts of the userland to require certain kernel features to be present. When that happens then the userland needs kernel updates.

      Part of what Google is working on now is getting it all setup so a manufacturer only has to target a certain kernel so they could then do all of their customizations with kernel modules which means that only the modules would have to be updated and that can be included in a userland update -- think locked bootloader with the ability to take partial kernel updates...not necessarily a good thing for the root and rom community. Combine that with FS-VERITY and DM-VERITY and it'll soon be a lot harder for XDA people to find exploits for devices since the manufacturers will be able to checksum and verify every freakin stock file a device ships with.

      Comment


      • #43
        Originally posted by coder View Post
        What I'm saying is a false dichotomy is the idea that a stable ABI means never changing it. IMO, a sane middle-ground would be some ABI version numbering scheme and general plan around how often and at what points incompatible changes would be introduced. You could separately version the ABI of different subsystems, or just tie the incompatible changes to the major version of the kernel (i.e. have it actually mean something).
        This idea does not hold up in the real world examples Solaris , windows....

        Originally posted by coder View Post
        Yes, any time you constrain kernel developers, there's always a downside - more hoops for them to jump through, to avoid introducing breaking changes at the wrong times, and maybe even a few cases where features have to get deferred for a few releases.
        I wish it was only features deferred a few release. Everything to do a stable ABI has got trapped. Windows 32 bit full PAE support allowing large memory in all cases has basically got deferred to never functional so the platform support dies first. So a feature deferred for a few releases to maintain compatible risk turning into never.

        There are examples in Solaris and other operating systems were deferred to maintain kernel ABI for drivers turns into the feature never landing as default user because the feature is kept turned off because it might break drivers users wishes to run.

        Originally posted by coder View Post
        All I'm saying is there are also upsides, and not only for closed-source, out-of-tree drivers.
        Except that is really the only major advantage.

        Originally posted by coder View Post
        Except I never said that. What I'm describing would be a compromise between the two extremes. It should be obvious that major versions mean there are points across which old drivers can no longer work.
        This is arguement of person who has not looked at the history of stable kernel ABI. The reality is if you have a stable kernel ABI and you want to break old drivers you will have massive resistance. Yes those fighting for a stable kernel ABI now is only a small amount of resistance. If you can avoid you don't want to increase resistance to change.

        Originally posted by coder View Post
        The very idea of there being some kind of policy around when incompatible changes should be introduced is specifically intended to counter the classic slippery-slope argument. Basically, you're saying that if kernel developers accept any kind of constraints, then they risk ceding all control. I think that's just absurd.
        You think the ceding all control is absurd outcome but this is what the history of every operating system to attempt a stable kernel ABI. Either you end up ceding all control or you drop the idea of stable kernel ABI and tell the driver makers to suck it up. Solaris and freebsd both attempted stable kernel ABI for drivers as well. So you are suggesting something that has been tried with know effects.

        Originally posted by coder View Post
        IMO, something so exotic is probably a lot harder to gain traction. But, good luck.
        Problem here for Linux kernel developers doing LTS kernels Semantic Patch is not exotic is how most of the patches in fact move from development branch back to the LTS kernels. There is a problem that there is fragmentation is how the Semantic patches are made with no policy for them to be both directions. So you find Semantic Patch suites to take code from Linus branch back to LTS but out of tree branches up to Linus branch you would be down right lucky to find the semantic patches todo that. There is also no policy that when you add a breaking change you have to create a Semantic patch either so it end up landed on those working on LTS branches to create them.

        So there is a policy problem.

        https://linuxplumbersconf.org/event/...ummit_2019.pdf
        If you read this above this is not the only policy problem.

        There is a bigger policy problems. You cannot start a stable kernel ABI without a functional test suite system to confirm it and bug system to repair the issues. You cannot really have a well tested forwards porting and back-porting Semantic patch system without a functional test suite system to confirm and bug system to repair the issues it either.

        Guess what the Linux kernel does not have a functional test suite system or bug system to repair the currently detected bugs without adding ABI or Semantic patch bugs on top.

        There might be less out of tree drivers as well if the development processes of Linux were in fact properly functional.

        Coder you called Semantic Patch exotic it is outside kernel space. Its not exotic for linux kernel space developers. You have presumes about the effects of a stable kernel ABI problem is history has it playing out the same way every single time its been done.

        The road to hell is paved with good intentions. Stable kernel ABI is one of those things that always starts out with good intentions but down the road a bit every time it been done turns to hell.

        So we do need to find another way of achieving the Stable Kernel ABI result without the major downside everything you are suggesting has failed to work in the Windows,. Solaris and freebsd attempts. The one thing we can be fairly sure of it will not be a Stable Kernel ABI that will be the correct answer. Could be like a stable kernel bytecode for drivers or semantic patch system for source...

        The correct solution is going to have some way to transform the driver code binary form.

        There are still many examples on windows where spectre and meltdown exploits still work in third party drivers this is a side effect of stable kernel ABI. So stable kernel ABI lowers security. Those with third party drivers to the Linux kernel hate the API changes. A stable API is from a security point of view a better choice than stable ABI in kernel space due to the security problems the ABI brings.

        If you cannot have s stable API something like Semantic Patch kit that is maintained is still quite a good solution so you can migrate from one API version to the next without manual coding.

        Comment


        • #44
          Originally posted by Aeder View Post

          Given Google's track record I fully expect them to abandon it and Fuchsia the millisecond it's not an instant success installed across all smartphone brands and driving billions of sales.
          But the point was that they *are* developing their own kernel. The point was not "how long will they be committed".

          Comment


          • #45
            Originally posted by Vistaus View Post
            But the point was that they *are* developing their own kernel. The point was not "how long will they be committed".
            https://www.businessinsider.com.au/g...-policy-2015-4

            That is not straight forwards. Google has this 20 percent time idea that keeps their developers happy. So at google a stack of staff and work on a project without any major plans for it going anywhere.

            There have been some quite spectacular R&D OS developments that have just dead ended in time due to hardware support issues. Good example Singularity OS from Microsoft. 7 years of full time development team not a final product and not a final product was the result of Singularity OS.

            You can extract stats from the fuchsia git. You will find if you do its a total of 12 individual developers all from google. There are no hardware vendors making drivers in the mix.

            12 developers working on fuchsia total there is over 10000 at google working on Linux in different areas. Fuchsia gets a lot of press but its development team is only the size of a company expendable R&D team. Singularity OS that failed was a team of over 200 so Fuchsia is not highly resourced.

            Vistaus is google developing their own kernel or are they just after a kernel where they can mess around with ideas then end up taking up stream into Linux? Or worse is fuchsia just developed by google to be a barging chip? Time will tell. Until we see products shipped with Fuchsia we will be guessing. That right up until Google ships products with Fuchsia its just a mirage of a future OS that may not have any real world usage or any long term development. Fuchsia at this stage is no different to walking into a planning office and seeing a nicely planned out future city that never comes into real world existence yet people are attempting to write how life will be in that city.

            Comment


            • #46
              Originally posted by oiaohm View Post

              https://www.businessinsider.com.au/g...-policy-2015-4

              That is not straight forwards. Google has this 20 percent time idea that keeps their developers happy. So at google a stack of staff and work on a project without any major plans for it going anywhere.

              There have been some quite spectacular R&D OS developments that have just dead ended in time due to hardware support issues. Good example Singularity OS from Microsoft. 7 years of full time development team not a final product and not a final product was the result of Singularity OS.

              You can extract stats from the fuchsia git. You will find if you do its a total of 12 individual developers all from google. There are no hardware vendors making drivers in the mix.

              12 developers working on fuchsia total there is over 10000 at google working on Linux in different areas. Fuchsia gets a lot of press but its development team is only the size of a company expendable R&D team. Singularity OS that failed was a team of over 200 so Fuchsia is not highly resourced.

              Vistaus is google developing their own kernel or are they just after a kernel where they can mess around with ideas then end up taking up stream into Linux? Or worse is fuchsia just developed by google to be a barging chip? Time will tell. Until we see products shipped with Fuchsia we will be guessing. That right up until Google ships products with Fuchsia its just a mirage of a future OS that may not have any real world usage or any long term development. Fuchsia at this stage is no different to walking into a planning office and seeing a nicely planned out future city that never comes into real world existence yet people are attempting to write how life will be in that city.
              They are working with a few vendors, including Honor, on Fuchsia-powered devices, so it's not a mirage at all.
              And they hired an important former Apple staff member Apple as well, so it's not just old-school Google employees working on Fuchsia: https://www.androidpolice.com/2019/0...us-fuchsia-os/
              Also, Huawei is helping shape Fuchsia, so it's not just the 12 Google employees working on it: https://www.gizchina.com/2019/08/01/...le-fuchsia-os/
              Last edited by Vistaus; 09-17-2019, 11:51 AM.

              Comment


              • #47
                Originally posted by Vistaus View Post
                And they hired an important former Apple staff member Apple as well, so it's not just old-school Google employees working on Fuchsia:
                Microsoft did the same thing with failed Singularity OS. Please remember the 20 percent time idea applies to new employees as well.

                Originally posted by Vistaus View Post
                Also, Huawei is helping shape Fuchsia, so it's not just the 12 Google employees working on it:
                That is before the trump mess. You can see that his commits have stopped. Huawei developer in mix gave access to HiSilicon future chips. To be a future OS you need the hardware developers on side to make drivers for unreleased hardware.

                Originally posted by Vistaus View Post
                They are working with a few vendors, including Honor, on Fuchsia-powered devices, so it's not a mirage at all.
                Its a mirage the few vendors. Honor is a solely own subsidiary of Huawei. Google + 1 vendor is all you have been looking at. Yes that one vendor has many solely own subsidiaries so it looked to those who did not look closely that there were many entries when there was only one.

                I can give something historically equal that got devices out the door and stilled died.
                https://en.wikipedia.org/wiki/MeeGo

                The only people currently working on Fuchsia is google personal. The trump trade war stuff has in fact done a number on Fuchsia to the point we may not see any released products or just a token amount.

                Making a operating system without hardware vendor support is normally a dead end may take a while to admit it. I would have more faith Fuchsia future if google acquired Motorola was still able to make current generation cellular modem chips but they cannot so google needs a hardware vendor to make a phone and that is what Fuchsia lost recently. Now can Fuchsia replace the lost hardware vendor this is going to be interesting.

                Comment


                • #48
                  Originally posted by You- View Post
                  The only proper fix is for the drivers to be upstreamed.
                  Perhaps, if you're only concerned about the issue of driver support for a given kernel. However, having more stability, around these interfaces, would also make it easier to maintain and back-port patches to both drivers and the core.

                  If you accept that view, there are a lot more stakeholders to consider - many of them even acting in good faith with respect to GPL.

                  Comment


                  • #49
                    Originally posted by skeevy420 View Post
                    Occasionally Google updates parts of the userland to require certain kernel features to be present. When that happens then the userland needs kernel updates.
                    Yup. Features are added within an API level, and API levels are tied to Android OS releases, each which have certain kernel requirements (as mentioned in a previous post).

                    https://source.android.com/setup/start/build-numbers

                    Comment


                    • #50
                      Originally posted by oiaohm View Post
                      This idea does not hold up in the real world examples Solaris , windows....
                      I don't accept those examples, since the only open source one is FreeBSD and that's too niche.

                      The problem with using Windows as an example is that some hardware says "works on Windows XP" (or pick your flavor of choice). Well, WinXP was supported for like 10 years or more. That's way longer than what I'm talking about.

                      The other problem with Windows and Solaris is that the majority of your drivers are closed-source / out-of-tree. So, the coordination effort is much greater, whenever there's a desire to change something.

                      Next issue: money complicates matters. There are lot of scenarios and permutations to this, but suffice to say it never makes things easier.

                      With all that said, I think the real issue is that many who are opposed to any kind of ABI stability are (at least partially) using it as a lever against out-of-tree drivers. This will work, as long as Linux remains in a dominant position. As soon as that falters, it will be one element of Linux' undoing. I'd rather Linux evolve and remain relevant, but I won't go down with a sinking ship. I'd suggest those with a deeper stake in Linux' future take this issue more seriously.

                      The other cynical explanation is just developer-laziness. However, that would be an entirely unfounded accusation from me, since (as I said), I really have no background in this area.

                      I also feel like you have an agenda that you're not really disclosing.

                      Comment

                      Working...
                      X