How Google's Android Maintains A Stable Linux Kernel ABI

[starshipeleven]

Project Treble stuff has already landed.

Yes but it has limits. Eventually they will need the underlying kernel to support something that older kernels don't, and then you would be screwed.

With this system, you could just leave the kernel modules in the "system" partition and be able to update the kernel too if needed.

[oiaohm]

Yes but it has limits. Eventually they will need the underlying kernel to support something that older kernels don't, and then you would be screwed.

With this system, you could just leave the kernel modules in the "system" partition and be able to update the kernel too if needed.

Stable driver kernel ABI has kind of equal problem. At some point you will be doing something in the stable ABI that you don't want to do any more because it causes horrible performance or the like. Linux kernel has removed a few syscalls to userspace over time because they came basically not supportable. Microsoft driver ABI has broken in places as well.

Remember the spectre and meltdown stuff this was only fixable properly in software by rebuilding all the drivers.

Stable Kernel ABI still does not mean that they don't need to mainline as many drivers as possible. Hard reality core OS secruity and binary blobs really don't mix. Commercial interest always want binary blobs even that long term its not good for them.

[starshipeleven]

Stable driver kernel ABI has kind of equal problem.

Only on the long run, and for high performance interfaces.

For most mobile devices that's less relevant. Most of their interfaces aren't high-speed and don't need top performance, also their realistic hardware life is not going to be much more than 5 years anyway.

They are trying to make a new trade-off for a longer supported window for specific classes of devices.

[oiaohm]

Only on the long run, and for high performance interfaces.

For most mobile devices that's less relevant. Most of their interfaces aren't high-speed and don't need top performance, also their realistic hardware life is not going to be much more than 5 years anyway.

High performance and mobile kind of does match up. Not for being exactly fast. Less processing a driver consumes less battery time it consumes operating.

When I said performance I don't just mean the case of speed. It can be the case of battery life. If the ABI for a driver is not fine grained enough you can be holding hardware active that could be suspended.

Kernel drivers having a stable ABI comes back and bites you. Windows CE and Windows Mobile run into a lot of battery performance problems caused by the driver stable ABI.

Yes google is trying to make longer support window for devices.

[coder]

Kernel drivers having a stable ABI comes back and bites you. Windows CE and Windows Mobile run into a lot of battery performance problems caused by the driver stable ABI.

IMO, it's a false dichotomy to debate whether or not to have a stable driver API. Obviously, there will need to be some incompatible API changes, over time. That's why God invented major versions, after all.

So, the real issue seems to be that there's no policy around introducing incompatible changes. That doesn't seem like such a big ask, but it's understandable why some GPL purists would rather not have to think about such matters.

However, I think it's not only vendors with proprietary drivers that would benefit. Anyone trying to port fixes forward or backwards could potentially benefit, and not just in kernel modules, but also the the core.

I should add that I haven't previously followed this debate, and my opinions are based on no actual experience with kernel or driver development. So, my statements should be treated as the view of a relative outsider (though one who's gotten a few scars trying to build or use kernel modules against mismatched versions, over the years).

[oiaohm]

IMO, it's a false dichotomy to debate whether or not to have a stable driver API. Obviously, there will need to be some incompatible API changes, over time. That's why God invented major versions, after all.

Its not a false dichotomy. Its the downside of stable kernel ABI. Yes you might be able to invent a major version. Microsoft windows 32 bit PAE support. This is the real reason why Windows 32 bit locked itself at 4g of memory max for desktop users. They had a stable ABI for drivers the drivers were made for 32 bit of memory and could not cope when you gave them PAE with 64G of memory on a 32 bit machine.

So, the real issue seems to be that there's no policy around introducing incompatible changes. That doesn't seem like such a big ask, but it's understandable why some GPL purists would rather not have to think about such matters.

I am not past the idea of a policy about introducing incompadible change this does not in fact require a stable kernel ABI and would most likely be way better long term without stable kernel ABI.

However, I think it's not only vendors with proprietary drivers that would benefit. Anyone trying to port fixes forward or backwards could potentially benefit, and not just in kernel modules, but also the the core.

http://coccinelle.lip6.fr/

Let say we had a policy that every incompatible change had to make a SmPL (Semantic Patch Language) SingularitySingularity to transform from old code to new code and with mirror for new code to old code.

There would be no need for a stable kernel ABI if that policy existed for open source parts of course that not going to help proprietary drivers one bit. Also it would avoid the set in stone pile of drives problems windows CE, Windows NT, Solaris,... basically everyone who has ever implemented a stable kernel ABI has run into.

The idea that major versions will dig you out of closed source drivers using the old ABI and refusing the upgrade is wrong.

There is a lot of not looking at the history of failures of the stable kernel ABI idea. You make a stable ABI in kernel space for drivers expect to be walled in by users with devices where the vendor is no more or the vendor will not make a new driver demanding you don't break the ABI so seeing you stuck. This is the direct price of the stable ABI in kernel space idea every single time its been done. It really about time we attempt to find another way of achieving the objective and avoiding this problem.

Microsoft with Singularity that came a failure was that all drivers would be written in bytecode that Microsoft could semantic patch by updating the bytecode to native converter to get away from stable kernel driver ABI problem. The stable kernel ABI idea is suggesting something Microsoft has found themselves stuck with and cannot get out of even throwing a few billion dollars at the problem.

Yes you need drivers to be provided in a form they can be modified source code is one option. We need policies so that updating code or backporting code is straight forwards.

Please note if you cannot really Semantic Patch changing major versions creates a huge stack of work and those making drivers will skip out on doing it. So a functional Semantic patch policies between incompatible changes is need with or without stable kernel ABI and we don't have that policy.

[jacob]

Blame nvidia.

Ps. if FreeBSD and Solaris have stable ABI's, then it seems it doesn't work for them. It's not some magical solution to success.

Having a stable kernel ABI *would* make life easier for users and third party driver developers alike, but it would come at a cost that the stakeholders don't seem to be prepared to pay. FreeBSD and Solaris are less popular than Linux, but I believe the primary reasons for that have little to do with a stable kernel ABI (which FreeBSD doesn't really have either, by the way).

[anarki2]

Sooo, literally the one and only successful product that utilizes the Linux kernel broadly disputes and defeats stable_api_nonsense.txt. It's about time Linux developers seriously reconsider ... everything they're doing.

Naturally, a lot of clowns here still defend the approach. It indeed worked greatly in the past 25 years, right? No.

[coder]

Its not a false dichotomy.

What I'm saying is a false dichotomy is the idea that a stable ABI means never changing it. IMO, a sane middle-ground would be some ABI version numbering scheme and general plan around how often and at what points incompatible changes would be introduced. You could separately version the ABI of different subsystems, or just tie the incompatible changes to the major version of the kernel (i.e. have it actually mean something).

Its the downside of stable kernel ABI.

Yes, any time you constrain kernel developers, there's always a downside - more hoops for them to jump through, to avoid introducing breaking changes at the wrong times, and maybe even a few cases where features have to get deferred for a few releases. All I'm saying is there are also upsides, and not only for closed-source, out-of-tree drivers.

The idea that major versions will dig you out of closed source drivers using the old ABI and refusing the upgrade is wrong.

Except I never said that. What I'm describing would be a compromise between the two extremes. It should be obvious that major versions mean there are points across which old drivers can no longer work.

You make a stable ABI in kernel space for drivers expect to be walled in by users with devices where the vendor is no more or the vendor will not make a new driver demanding you don't break the ABI so seeing you stuck.

The very idea of there being some kind of policy around when incompatible changes should be introduced is specifically intended to counter the classic slippery-slope argument. Basically, you're saying that if kernel developers accept any kind of constraints, then they risk ceding all control. I think that's just absurd.

Please note if you cannot really Semantic Patch changing major versions creates a huge stack of work and those making drivers will skip out on doing it. So a functional Semantic patch policies between incompatible changes is need with or without stable kernel ABI and we don't have that policy.

IMO, something so exotic is probably a lot harder to gain traction. But, good luck.

[You-]

The only proper fix is for the drivers to be upstreamed.

AFAIK this is happening and everyone benefits from it. The stragglers must be dragged along.

The alternative is integrated components falling out of support a couple of years after release - which the maufacturers dont care about as its past thewir supported release cycle.