NVIDIA Transitioning To Official, Open-Source Linux GPU Kernel Driver

[dfx.]

Wow, is there a trap I missed somewhere?

The trap is that:
1) It's just a shim between signing-jailed binary blob firmware running on card's built-in CPU and binary blob userspace.
2) Userspace API with that shim is unstable.
3) It's not feature-complete with its firmware API either.

It's just a fancier shim and empty-gesture PR move. Not much different from bundled shim or Broadcom's "open-source drivers".
But fanatics love to get excited around their corporate slop all the same…

[oiaohm]

Again, your message is so full of inaccuracies and things you wanna think are true but they aren't not, I don't want to continue this conversation any longer.

https://kernel-recipes.org/en/2019/talks/cves-are-dead-long-live-the-cve/

You need to watch this video because you are keep on with inaccuracies.

• What was the last time you visited https://github.com/lm-sensors/lm-sensors ? Suprise, the project is essentially dead, bugs linger for years, there's zero active development.
• What was the last time you checked commits in https://git.kernel.org/pub/scm/linux.../drivers/hwmon ? Likewise, some commits here and there, overall quite a bleak picture.
• What is the source of "Hardware sensors we are seeing more released and with specifications these days and included in the mainline kernel"? Your imagination?
• The existing hw sensor drivers continue to show sensors without names and without any proper formulas.
• AMD and Intel continue to release fucking nothing in terms of properly supporting a ton of MSR registers responsible for various hardware states.
• What was the last time you ran HWiNFO64 and compared to what lm-sensors provide?

The f*k it matters that old deprecated versions of HWiNFO64 don't work in Windows 11 when the app is compatible with all Windows versions starting from 95: https://www.hwinfo.com/about-software/ which is basically unheard of in Linux? Show me a single serious GUI application which works in Fedora 36 and RedHat 5.0 from 1996 out of the box without using hacks, tricks, chroots, VM, etc.? No, nothing?

This is stack of false arguments based on a stack of incorrect information. lm-sensors and hwmon should be compared to what Windows provides out the box not to HWINFO64. Yes what windows provides is not good either we need Intel and AMD and others todo better everywhere.

https://git.kernel.org/pub/scm/linux.../drivers/hwmon checking commits is really easy and hwmon is not dead and if you had you would not have not questions my statement on stuff being added from specifications at a higher rate than it use to be. I agree not developing as fast as I would like does not make my statement false.

You overlook that the version of HWINFO64 that works on Windows 10 release does not work after the 2 year kernel update then release after that update does not work after future updates this comes a repeating pattern. Yes HWINFO64 supports lot of versions of Windows but HWINFO64 deprecating lots of versions because it has a Windows Kernel driver and Microsoft is changing the driver ABI lot often than people claim. So like or not the program HWINFO64 is in fact example of Windows unstable kernel ABI and it compatible with multi versions of Windows due to the developers effort not Microsoft providing a stable kernel ABI.. Also find alternative to HWINFO64 on Windows once you attempt that you will find out that its odd on Windows as well the alternatives don't make a kernel space driver because it hard work and don't provide the same detail.

HWINFO64 is commonly updating it driver to support newer versions of windows as the Windows driver ABI changes and breaks the driver. This arguement that I cannot use tricks to allow old applications to work would be basically equal to say run old windows applications on Windows 11 with SXS removed right. SXS is mostly a userspace feature without Windows does not support applications as much..

Linux will continue to be nothing on the desktop until someone realizes that ISVs and IHVs absolutely require stable API/ABI. There are no ifs and whats, period. That's the fact of life, that's how the software industry has operated for the past three decades. No amount of preaching about the "advantages" of the open source development model will change that.

No this requires you to admit the facts. Stable kernel driver ABI really don't exist on Windows, Linux, Mac OS.... Only appears to work due to IHV, ISV effort. So that different parties require a stable driver ABI don't add up. Birdie there are somethings not matter how much you like cannot properly be done.

The old microkernel idea where the drivers are a different ring/memory space to core kernel are the only cases where you find functional stable kernel driver ABI. Yes this is exactly like Linux kernel cuse fuse, UIO drivers even this is not perfect. Why is this because in reality when you load module/driver into kernel space directly it can interface with what ever by direct memory actions this leads to a problem.

Linux kernel include modversion because at one point the Linux kernel developers did attempt to have a stable kernel driver ABI and I mean serous-ally attempt.. There has been dispute between Linus Torvalds and Linux Distributions over modversion. Linus Torvalds is correct that modversions does not pick up all the defects a kernel driver could have. modversion is designed to version the imported symbols into module to hopefully pickup when those have changed to prevent crashing system.

Is there any different solutions yes there is Solaris one is interesting their compiler linker supports relinking so you have a kernel module for one version of the Solaris kernel you need it to work in a different version you take the module file and link a wrapper object file to that kernel module and now hopefully it should work. Yes this also gave Solaris the interesting option of taking kernel module driver and converting it into a usermode driver in cases you have driver accessing changed structures.. Reality here this that Solaris had done is basically the kernel module/driver equal of Microsoft SXS.

Lastly, open source et al, why do unsupported device drivers get dropped from the Linux kernel? How does that coincide with "Release specs, we'll write drivers and it'll be all good?" That's always been a bloody lie. 99% of contributions to the Linux kernel are made by large corporations which only care about their own use cases, software and devices.

The reality here drivers are commonly removed from the Linux kernel because there is no one with the hardware to test that the updates to the driver works any more and that is after normally 2 years of that being the case. So the driver is technically bit rotted in that case. Yes this does show up because hardware that has software emulation on vm or boch or something else remain being supported because those drivers still can be tested.

Releasing the specs helps, Releasing emulator for the hardware also helps.

birdie the reality nothing you have show proves a stable ABI for driver exist https://kernel-recipes.org/en/2019/t...-live-the-cve/ this presentation shows you that does not exist from RHEL because it details how RHEL brakes it stable ABI promise legally(not morally) and also explains what happens with Android and other items. This issues causing HWINFO64 and Nvidia and all other parties using drivers on Windows to update their drivers show stable driver ABI does not exist on Windows.

At some point you have to accept reality. Stable driver ABI in anything other than microkernel designed OS cannot exist because security faults and other things will require you to change the structures inside kernel space so breaking that Stable driver ABI. Problem here is that inside kernel space drivers can use direct pointers to structures so your ABI is very large very complex and really simple to break even a different in compiler can do. A intel developer at one point did experiment with running linux kernel drivers in individual virtual machines. You have Solaris that did relinking. Solaris with relinking has 30+ year old binary solaris drivers still working on current day Solaris. Remember Solaris is not using stable driver ABI they are using wrappers to convert old ABI to new ABI and where required to move driver out of kernel space so that kernel structures presented to driver can be faked.

Yes SXS under windows using manfests and wrapper DLL is fairly much doing the same thing as how Solaris got it to work. Not have stable ABI at all but have a system to provide wrappers so something can use old deprecated ABI yet work anyhow.

birdie what you are asking for basically does not work with the test of time. Its about time you pull your head out the sand and have look around at reality. Stable ABI mostly does not exist. Where it appear to exist and function well someone is normally working very hard with wrappers or rebuilding stuff.

The hard point birdie if you want driver to keep on remaining work no matter what you most likely not what a stable driver ABI instead want a Solaris driver relinking system as this allows old drivers to be updated and put into compatibility modes.

Same with wanting stable ABI for applications in user space you mostly don't want that instead you want something like Microsoft SXS that is a stack of wrappers so newer applications can use newer ABI and gain performance advantages.

Stable ABI requests in most cases like it or not are asking for the wrong thing.

[birdie]

Again, your message is so full of inaccuracies and things you wanna think are true but they aren't not, I don't want to continue this conversation any longer.

• What was the last time you visited https://github.com/lm-sensors/lm-sensors ? Suprise, the project is essentially dead, bugs linger for years, there's zero active development.
• What was the last time you checked commits in https://git.kernel.org/pub/scm/linux.../drivers/hwmon ? Likewise, some commits here and there, overall quite a bleak picture.
• What is the source of "Hardware sensors we are seeing more released and with specifications these days and included in the mainline kernel"? Your imagination?
• The existing hw sensor drivers continue to show sensors without names and without any proper formulas.
• AMD and Intel continue to release fucking nothing in terms of properly supporting a ton of MSR registers responsible for various hardware states.
• What was the last time you ran HWiNFO64 and compared to what lm-sensors provide?

The f*k it matters that old deprecated versions of HWiNFO64 don't work in Windows 11 when the app is compatible with all Windows versions starting from 95: https://www.hwinfo.com/about-software/ which is basically unheard of in Linux? Show me a single serious GUI application which works in Fedora 36 and RedHat 5.0 from 1996 out of the box without using hacks, tricks, chroots, VM, etc.? No, nothing?

Linux will continue to be nothing on the desktop until someone realizes that ISVs and IHVs absolutely require stable API/ABI. There are no ifs and whats, period. That's the fact of life, that's how the software industry has operated for the past three decades. No amount of preaching about the "advantages" of the open source development model will change that.

Lastly, open source et al, why do unsupported device drivers get dropped from the Linux kernel? How does that coincide with "Release specs, we'll write drivers and it'll be all good?" That's always been a bloody lie. 99% of contributions to the Linux kernel are made by large corporations which only care about their own use cases, software and devices.

Open Source, as a hobby which can produce wonderful software products, is largely a myth. Again, there's Linux which perfectly manifests that (driven largely by corporations, supports only what it needs to support) and we have a ton of obscure minor OS'es like Menuet which are dead and in which you can barely do anything.

Good luck and have fun living in the open source bubble where hwmon and lm-sensors are in a good shape. And I haven't even addressed the rest of your comment.

[oiaohm]

There's not a single proof stable API prevents and makes it impossible to reach "performance or stability" in the Linux kernel. Also, no one is talking about sticking to API for infinity - 5 to 10 years is enough as again proven by RHEL and Android.

There is one very big nightmare https://kernel-recipes.org/en/2019/t...-live-the-cve/ You need to watch this video carefully. In this a kernel developer in fact documents how RHEL does not keep it stable kernel ABI driver promise. Its horrible RHEL kernel developers takes out fake CVE to allow them to break ABI inside the rules of their stable driver ABI promise to fix performance issues. This happens at least 1 every 2 years more commonly 1 every 9 months. So much for your stable ABI promise.

So RHEL does not prove what you claimed at all. The reality here is RHEL and Windows are basically same of somewhere between 9 months to 2 years of driver ABI stability. That would basically be the same as using a LTS kernel

I've not seen any benchmarks which prove that stable Windows driver API results in degraded performance. NTFS is indeed slower than Linux filesystems but that has nothing to do with stable kernel APIs. NTFS is on a whole different level when it comes to ACL, reliability and other features most Linux filesystems can only dream of.

Of course not because Windows ABI driver stablity is only 1 to 2 years these days to prevent it stack up into a performance problem.

Windows kernel API is fucking stable, period. You can use some ... Windows 7 drivers in ... Windows 11. Windows 11 currently supports all Windows 10 drivers sans GPU (not sure about that, need to check it out) and we are talking about tens of thousands of devices. That's 7 years of 100% API/ABI compatibility for most devices.

Really the horrible fact is you can force load 7 year old modules from 7 year old Linux kernel on mainline Linux kernel and some of them work to be correct a huge percent work. This is all you have really done with Windows.

Stable fast reliable OSes with stable kernel API do exist and actually there are the absolute majority of the computing industry.

Did you really check no because you are saying RHEL is one right. Reality Watch the video RHEL is not one. Android is not one either that in fact provides a proper stable kernel ABI. Windows is not one either. The reality is 1 to 2 years of stable driver ABI is all get at best. Yet people like you will go around claiming 5 to 10 years that is not the reality. Stuff happens to work by luck out to 5 to 10 years not by good design.

I mean solid fucking proofs are in your face and you doubt them and find ways to dismiss them. How does that even work?

No you are out of your depth. What you think is solid proofs that stable driver ABI works are not in fact. Once you go and properly look it turns out stable ABI are really short promises.

Why in the +30 years of Linux development there's been zero proofs the Linux kernel works as is? Why do absolute most distros and people actively eschew the vanilla kernel - riddle me fucking this. What? We need to wait 20+ more years to make it reality? Like really?

For the same reason stable driver ABI is fiction. Distributions have this idea "We want to make our kernel perform better and if we add some our own code it has to be better right. " Yes watch the presentation I linked to happens to go though how what distributions do end up causing more security faults and makes a slower running kernel in most cases. This active eschew of the vanilla kernel seriously makes bugger all sense in most cases. Fiction is fiction. The driver stable ABI fiction and the distributions not waiting to use the mainline kernel majority of this is going after performance.

Lastly there are a number of things which will never be open sourced/open specced/etc. e.g. hardware sensors. Linux totally sucked in terms of sensors support 20 years ago, nowadays it sucks even more. If we don't have stable kernel API we will never get something like HWiNFO64 for Linux. Isn't it amazing? I meant fucking horrible but who cares, STABLE API NONSENSE ALL THE WAY.

The reality you don't have a proper stable kernel driver ABI under windows this is why old versions of HWINFO64 don't work with the latest updates of Windows. Yes the HWINFO64 driver has to be updated with the Windows driver ABI changes or it going not function correctly.

Hardware sensors we are seeing more released and with specifications these days and included in the mainline kernel. Yes HWINFO64 developer has done a lot of reverse engineering but he is not the only party doing this and HWINFO64 does not always report correct values either. Never is a very long time. Stable kernel API in most cases is not the solution here either.

[birdie]

• Lastly there are a number of things which will never be open sourced/open specced/etc. e.g. hardware sensors. Linux totally sucked in terms of sensors support 20 years ago, nowadays it sucks even more. If we don't have stable kernel API we will never get something like HWiNFO64 for Linux. Isn't it amazing? I meant fucking horrible but who cares, STABLE API NONSENSE ALL THE WAY.

[birdie]

long quote

So many things wrong/false in this post it's cringe-worthy.

• There's not a single proof stable API prevents and makes it impossible to reach "performance or stability" in the Linux kernel. Also, no one is talking about sticking to API for infinity - 5 to 10 years is enough as again proven by RHEL and Android.
• I've not seen any benchmarks which prove that stable Windows driver API results in degraded performance. NTFS is indeed slower than Linux filesystems but that has nothing to do with stable kernel APIs. NTFS is on a whole different level when it comes to ACL, reliability and other features most Linux filesystems can only dream of.
• USB is already mighty fast in Windows. I've yet to see a single user complain about USB performance in Windows.
• Windows kernel API is fucking stable, period. You can use some ... Windows 7 drivers in ... Windows 11. Windows 11 currently supports all Windows 10 drivers sans GPU (not sure about that, need to check it out) and we are talking about tens of thousands of devices. That's 7 years of 100% API/ABI compatibility for most devices.

Walls of text, oiaohm, don't make your posts any more valid for the lack of solid proofs/quotes. All I see is a ton of "IMO" and that's it.

Stable fast reliable OSes with stable kernel API do exist and actually there are the absolute majority of the computing industry. If that's not a proof enough in the face of never ending regressions in the Linux kernel and the inability of loading the drivers versions you need with the kernel release you need is the best proof the Linux kernel development model is flawed to the core. And again, let's not forget about the two most reliable Linux implementations, RHEL and Android.

I mean solid fucking proofs are in your face and you doubt them and find ways to dismiss them. How does that even work?

Why in the +30 years of Linux development there's been zero proofs the Linux kernel works as is? Why do absolute most distros and people actively eschew the vanilla kernel - riddle me fucking this. What? We need to wait 20+ more years to make it reality? Like really?

[stiiixy]

Really interesting conversation! Thanks for bringing it back on track.

[oiaohm]

Maintenance is cheaper, yes, but bring up is more expensive. And there's no cheaper than no maintenance, which is what they in practice do, stable or unstable ABI. Just with unstable ABI the breakage is guaranteed to happen sooner rather than later, while with stable ones you have a chance it works.

You still have incorrect presume. You miss that Nvidia closed source drivers and any closed source driver like it works for multi versions of the Linux kernel at a time. Interesting enough this generally a long time frame than they in fact work on Windows even that Windows releases kernels less often. Reason why the Nvidia break on Linux is some API change of something the Nvidia driver uses. Yes and the windows API change shows up as a minor ABI change.

Linux kernel is interesting default Linux kernel build refuses to load drivers from the wrong kernel version but its not that it cannot.

https://linux.die.net/man/8/modprobe

-f --forceTry to strip any versioning information from the module which might otherwise stop it from loading: this is the same as using both --force-vermagic and --force-modversion. Naturally, these checks are there for your protection, so using this option is dangerous.

This applies to any modules inserted: both the module (or alias) on the command line and any modules it on which it depends.
--force-vermagicEvery module contains a small string containing important information, such as the kernel and compiler versions. If a module fails to load and the kernel complains that the "version magic" doesn't match, you can use this option to remove it. Naturally, this check is there for your protection, so this using option is dangerous.

This applies to any modules inserted: both the module (or alias) on the command line and any modules on which it depends.
--force-modversionWhen modules are compiled with CONFIG_MODVERSIONS set, a section detailing the versions of every interfaced used by (or supplied by) the module is created. If a module fails to load and the kernel complains that the module disagrees about a version of some interface, you can use "--force-modversion" to remove the version information altogether. Naturally, this check is there for your protection, so using this option is dangerous.

This applies any modules inserted: both the module (or alias) on the command line and any modules on which it depends.

Please note all your Debian kernels are built with CONFIG_MODVERSIONS enabled same with most of your major Linux distribution kernels. The reality is you can in fact load drivers build from the mainline kernel source and have it on a LTS kernel. Yes you will need to use --force-vermagic to ignore the kernel version check and if the ABI of the kernel is still suspect to be close enough MODVERSIONS data will not prevent the driver from loading and running. Yes ideal is that the mainline kernel has been built with the same compiler as the LTS kernel to keep the hazard causing stuff to a min.

The reality here is Microsoft stable ABI promise turns out not to be much different to using a Linux kernel with the --force-vermagic option to load modules.

The reality here is even with Linux kernel so call unstable ABI there is still a chance a old driver will work on a new kernel or a new driver will work on a old kernel. The odds of it working right are about the same as Windows that claims a stable ABI. Linux kernel since the developers admit its not a stable option they make using that option jump though extra hoops. Windows where they claim stable ABI basically lets you straight up roll the dice and pray.

Yes a number of third party roms of Android device have in fact used newer kernel than vendor provided and have forced loading binary drivers from older kernel versions.

The horrible real world the Stable ABI promise for drivers means very little. All it really means is that we will automatically attempt to load drivers that appear close enough and when it does not work that will happen at some point you get to keep the broken pieces even that you were not told that was the case.

Linux kernel admitting that the kernel ABI is unstable and not sugar coating it and make it hard to use the less safe options some how people see this as a bad thing. Yes the Linux people don't talk other options that much. Why in most cases when you think you want to use them you really don't. Hard reality is not all CVE patches in fact made it back to LTS kernels. So the older kernels are less secure. Yes Microsoft windows the older kernels are less secure as well.

Remember, the kind of vendor that does closed source releases, sans some more responsible exceptions, only do bring up and move on. Nvidia, for a company that provides only closed source drivers, is in the responsible side of those as they do care enough to pay the maintenance cost, but they're the exception, not the rule. So the totality of their costs for most is that bring up that is cheaper by not attempting going mainline.

The reality here is any vendor who does closed source releases that not like Nvidia long term that hardware is going give you major trouble this does not matter if you are on Windows or Linux. Yes major trouble where you have to choose between updating your OS or using the hardware. Ideal world vendors that do bring up and move on we should vote with wallet and not buy from them.

We agree on the importance. But the reality is the manufacturers' policies tend to be Not My Problem (TM). Kernel policies failed to change that. I'm only contending the claim that unstable interfaces lead to more open source code, not that open sourcing code is in practice better in the long term, something I agree with.

This is not in fact true that Linux Kernel Polices are failing to change this. The mainline polices have been slowly changing different manufacturers Qualcomm that I gave as example started exactly with "Not My Problem" logic but now they mainline. This has been a slow pulling teeth process the reality its a slow pulling teeth process that slowly is progressing to more and more manufacturers mainlining their drivers.

Google with Android has helped with quite a few SoC vendors by their policy for Android kernel recommending mainlined for all drivers.

Lets say Linux kernel released a promise of Stable ABI of the same quality as Microsoft tomorrow. Linux kernel could do this just made the kernel ignore the versioning stuff straight out the box let the end users live with the instability this would fairly much make it really simple to never mainline drivers and attempt to push ABI compatibility support back into the kernel core at the cost of performance or stability . Yes some areas where Microsoft benchmarks badly comes from this and some areas like USB support where there are old functions that cause big problems if used come also from force ABI support.

Stable ABI promise like it or not is not a good thing. Windows having a Stable driver ABI promise and Linux not make less difference than most people think. Biggest difference is that Linux world makes it harder to cross kernel load drivers as in load a driver build for one kernel on a different kernel but does not make it impossible todo that. Yes the Linux kernel will not autoload a module that from the incorrect kernel by default.

Early Linux kernel history had a Stable driver ABI promise that was in fact true this did major adverse effects on kernel development and performance. Microsoft stable driver ABI is a fake promise that they don't keep this lead to people also not understanding how bad a correctly implement stable driver ABI promise is. Yes it due to how much of the kernel the stable driver ABI prevents you from changing.

So the difference is not what people think it is.

Do you want a OS:
1) That will attempt to load kernel module/driver without tell if it safe or not and when it fails you keep the broken pieces. This is windows.
2) OS that will attempt to make you use highly CI/QA/QC drivers by making that the simplest path and harder to load poorly CI/QA/QC drivers. This is Linux.

You can fairly much ignore the stable driver ABI arguement because the reality is that not a solution.

[sinepgib]

There is a claim here that is false that Microsoft in fact guarantee API/ABI driver stability they write that they do but real world that like Nvidia issue linked before proves that ABI stability does not fact happen no matter how much Microsoft attempts to. There are tones of examples where that just does not happen that there are tones of examples you do need to look at why.

I haven't experimented this even with rather fringe hardware, so I don't know.

Windows in reality you would only see what the Linux world would call longterm kernels this is in fact confirmable by getting the Windows kernel version numbers there is in fact Windows kernel version numbers. Windows as less kernel caused issues with drivers by releasing kernels less often. The horrible reality is every time Microsoft releases a new kernel to end users some drivers do in fact break even that they have promised API/ABI stability. The cause is simple Windows kernel developers don't know how driver developers are in fact using the ABI and the driver developers presume something is defined when its not. Inline development means those making core kernel changes can look at the code in fact using those functions in the drivers and see issues. The reality is even with total control like Microsoft of the kernel space they cannot keep the driver ABI stable enough not to cause problems.

Whether it is long term kernels or otherwise (they do get patched for fixes and stuff keeps working, I don't care about the feature freeze there), the fact is stuff tends to work regardless of how bad your driver is maintained, while in Linux stuff breaks. And I don't think we have the right attitude towards solving end users problems by rationalizing the situation that brings them problems to convince ourselves it's the best possible world.

Note you talked about driver developers should follow the kernel mailing list for breakages there are many drivers in the Linux kernel that have zero active developers that keep on working. There are many cases of window drivers that end up completely failing on end users resulting in either end users having to disable windows updates or stop using the hardware.

Those tend to get dropped from mainline anyway. I was thinking about the ones people maintain, and in the context of still keeping API/ABI instability and open source drivers here. If drivers are open source they may very well be out of tree and get fixed by some of the more tech savvy users, while having e.g. their own release schedule.

There is a trick to it why old drivers keep on working that are upstream that have no active developers.
https://en.wikipedia.org/wiki/Coccinelle_(software)
Semantic patches. Due to all the code being inline and upstream its possible to use Semantic patches to correct developer coded incorrect presumes. Also these can be used to location code that is not to expected usage.

And they also remain broken if nobody watches them. Semantic patches don't replace testing, for a start, it's more or less best effort in that sense.

This contains a stack of bad presumes also you have ignored what you wrote. Proprietary drivers or out of tree drivers require on going work to remain working under Windows or Linux. Linux kernel supports a lot of legacy hardware that Windows does not because the inlined drivers require less work. Semantic patches that the Linux kernels started using in 2007.

We agree in-tree drivers require less work, without doubt. But as with every decisions, it has upsides and it has downsides. I'm focusing on the downsides because the upsides have already been covered.
I'd like for you to point out exactly what are the wrong assumptions. I know proprietary drivers and out of tree drivers require ongoing work to remain working under Linux. I didn't experience that on Windows. But, in reality, what I was questioning in the quoted text was whether the policies of the Linux kernel actually led to manufacturers open sourcing their code. I'm not saying it makes it worse (definitely not), but it doesn't seem to make a dent in the market I mentioned. Instead, you get stuck with a proprietary driver bound to an ancient kernel version. It's sad but it's true. Stable interfaces could at least give you a decent chance of that blob dump to work with a newer kernel, while unstable interfaces guarantee it won't work. I'm particularly thinking about all of the SoCs used for Android devices, as well as other embedded environments.

Lets say this person is the driver developer who does not upstream their driver who is going to be their to perform the maintenance so the driver works with newer kernels the answer is no body if they don't release the source code at all.

Yes. I'm not defending that developer other than explaining their rationale. Releasing a driver as open source is easy for individuals (tho getting it upstream is not). But for companies it involves legal spending, more time allocated per project for their (often one person) dev team to deal with iterations in the kernel list, etc. And if you saw what they dump when they do release open source code you'd be sure that garbage quality code can never make it upstream without a total rewrite (you can look for sunxi kernel code for example, there's a vibrant community doing that work but what Allwinner themselves released might be the ugliest code I've read).

Releasing the source code someone else could chose to pick up the code base either maintain it or upstream it.

I think we all know the benefits of open source here
Tho it sometimes ends up being more theoretical, as it happens with many soon-to-be-dropped pieces of code that have no maintainer and nobody steps up when deprecation is announced.

The issue of being stuck on old OS versions happened in embedded space with Windows CE and other closed source operating systems used in embedded space in history. Being stuck on a decades old kernel/OS is a problem that existed before Linux was heavily used in the embedded space and so called stable kernel driver ABI did nothing to address this. Core of this problem is the developer moves on and there is no one to maintain the drivers. Inlined into the kernel tree.

Stable kernel driver ABI is never 100 percent stable while the operating system still has kernel development.

https://elinux.org/Qualcomm_SOC_Mainlining_Project

The reality is lot more complex. Lets take Qualcomm here they did the code dumps like you described and other third parties have been working on getting those code bases to quality to inline into the Linux kernel. Notice that stalls out around 2016 because then there is a change at Qualcomm

https://www.linaro.org/blog/let-s-bo...lcomm-devices/

That is because around 2014 Qualcomm started noticing that on their old boards the new kernel from mainline worked better than their custom kernels. Yes then started working to up-streaming their drivers directly.

I will not dispute the awful history of vendors just doing code drops but there are vendor after vendor like Qualcomm who have worked out that is more cost effect to upstream and gets them better quality product and needing lower cost to keep the drivers functional. Yes there is a long history of third parties picking up those code drops and up-streaming what they can.

Very well may be. I'd rather have a "throw a dice" situation than a guaranteed failure in those cases. Of course, ideally all drivers would be open source and in-tree. But it's far from the reality and, IMO, the current kernel policies aren't helping force companies to mainline their code. Not the opposite either, but they do break those old dumps.

Additional cost arguement in reality does not check out. There is additional cost to be saved if you are just going to make something and just walk away and abandon it without up-streaming your drivers or releasing the source code. Problem here if this is your objective end user is going to be stuck in the past be it a OS that claims driver ABI stability or not because there is going to be no developer to update the driver when the driver comes incompatible. The reality of the driver coming incompatible its not if but when.

There's no additional maintenance cost. But there's additional cost in lawyers and there's additional cost in lengthening the development process. Now they need to reach mainline code quality standards, which are rather high (and I'm thankful for that!), whereas before they could just duct tape things and move on.
Not those providers most often don't care whether it will become incompatible. They gave you something that sorta kinda worked and that's it. It's the end user that cares (sometimes a developer).

If you are going to keep developer on the driver maintain the driver so it works this is going to be a on going cost right. Think about it you have released the code its up-streamed one the Semantic patches are doing some of the required updating work as in the kernel developers change something they make a Sementic patch to update all drivers inlined in the Linux kernel. Third parties can decide to take up driver development.

This was addressed above. They won't keep developers. They aim at minimal cost. Which means half-assed dumps and forgetting they ever released that SoC.

Take old intel graphics cards recent new drivers made by non Intel personal because they are still using the hardware this is only possible without major reverse engineering because they had the old drivers to look at so those source code releases are important.

We agree on the importance. But the reality is the manufacturers' policies tend to be Not My Problem (TM). Kernel policies failed to change that. I'm only contending the claim that unstable interfaces lead to more open source code, not that open sourcing code is in practice better in the long term, something I agree with.

This is the problem stable kernel driver ABI promise really does not change a thing. Its raised as a solution but as soon as you properly look the stable driver ABI does not check out.

This is a problem also unstable ABI does not change. The defense for unstable ABI is mostly that it allows better fixes and optimizations. But leading to more open source drivers? I don't see it. Stable ABI may or may not keep that half-assed code kinda sorta working as expected, which for someone with the bad luck of owning such devices is better than nothing.

The reality the stable kernel driver ABI promises by all operating systems not just Windows while the operating system is still being developed/maintained does not save money because developers are required to maintain the drivers as the failures turn up. Linux and Freebsd inlined drivers in fact have a lower maintenance cost. The lower maintenance cost that has been documented over and over again has many long term kernel developers not understanding why its like pulling teeth to get vendors to upstream drivers because not up-streaming drivers is just costing them over the long term.

Yes this is a problem short term cost of up-streaming is slightly higher with the long term being way cheaper due to how much is pure autofixed with Semantic patches and having other third parties being able to submit patches so the over cost it cheaper to inline the code into the Linux and freebsd kernel trees.

The additional cost arguement is something people bring up as a case for Stable kernel ABI drivers not being aware that this arguement has been disproved over and over again.

Maintenance is cheaper, yes, but bring up is more expensive. And there's no cheaper than no maintenance, which is what they in practice do, stable or unstable ABI. Just with unstable ABI the breakage is guaranteed to happen sooner rather than later, while with stable ones you have a chance it works.
Remember, the kind of vendor that does closed source releases, sans some more responsible exceptions, only do bring up and move on. Nvidia, for a company that provides only closed source drivers, is in the responsible side of those as they do care enough to pay the maintenance cost, but they're the exception, not the rule. So the totality of their costs for most is that bring up that is cheaper by not attempting going mainline.

[oiaohm]

You're not wrong, it's often critical to keep certain drivers fully up to date, graphic drivers among the most common of those. It is often equally critical to stick with an LTS kernel to avoid bugs, and now those needs are in conflict.

Drivers should not be part of the kernel, or maybe they should, it is convenient to be able to install your OS and everything just works out of the box, but there should be an easy way to use a more recent version of specific drivers without using a newer or a custom kernel to achieve it. Similarly driver updates should not need to meet kernel merge windows to go through, long times pass between kernel releases after all, but there's no sense for a driver to wait that long for an update.

This is because you missed something. Semantic Patch yes the Coccinelle stuff. This is where the Linux core kernel developers change something about the API of the Linux kernel they make Semantic Patch to update and check everything in the mainline kernel tree. This is how the Linux kernel included drivers avoid the Microsoft Windows kernel problem of update kernel and drivers break.

So there are two ways a new open source driver in the Linux kernel is made.
1) developer maintaining the driver adds new features and so on.
2) developers maintaining the kernel core alter something and the driver is altered by Semantic Patch to match the change.

Yes its that number 2 that makes Linux kernel updating that also updates all drivers include with the kernel not to suffer from the Windows problem of update kernel have lot of random drivers fail.

Also mainline Linux kernel developer have different tools they can use to inspect how a function is used by all drivers include in the kernel. This is very useful to the core developers to know how X function is really used.

Also the Linux kernel release cycle is not that slow with a release every 9 to 10 weeks. Slow include of a driver into the Linux kernel mainline is not commonly due to the the release cycle. The reality is that 9 to 10 weeks is way shorter than most hardware release cycles. Vendors wanting to keep there next release secret is a bigger reason but its not the biggest reason. The most common reason for a driver to be slow to be in the mainline Linux kernel is failure to pass peer review because their are spot-able errors in the code base. Yes this is common automated bot checking the code and putting out a stack coding defects that have to be fixed.

Yes you have people like birdie who say they want proper CI/QA/QC and not being ware that is the key reason why drivers can be slow to get into mainline Linux is CI/QA/QC of the kernel finding problems with the driver parties want to upstream forcing the driver to go back and be repaired. So lot of cases the open source drivers that are not in Linux kernel mainline have stability problems.

Yes "no sense" on the delay into mainline Linux is not exactly true there is a reason to it. To perform CI and QA to a decent standard does take time same with fixing all the faults a decent CI and QA solution is going to find. Yes this is a higher quality bar than Microsoft driver signing and this happens to cause a delay in deleivery.

rabcor the hard reality is a non mainlined linux driver is going to be on average lower quality because of the level of Continuous Integration and Quality Assurance that is performed by the mainlining process being very high standard. I am not saying there should not be methods to backport drivers from mainline to older kernels more effectively and that their should not be methods to allow installing drivers from the developer of that drivers kernel tree.

Most people are not aware that AMD, Intel and so on developers maintains their own Linux kernel trees that they update when they like and sync to mainline. Yes the developer of the driver kernel trees there are generally lower quality than the mainline kernel tree because that automated bots like Intel 0-Day automated testing does not run against all kernel trees. https://01.org/blogs/2018/0-day-ci-test Yes there are many company bots that run over the Linux kernel mainline looking for defects.

Yes the ideal world is that all drivers get mainlined at some point with Linux so they get the most quality control possible performed. Remember tools like intel 0-day don't work on binaries you need source code.

rabcor most likely you want all drivers mainlined. Yes it costs quite bit a first due to how much drivers have to be fixed to get into mainline. Faults that developers get way with in non mainline code are not going to fly in mainline Linux kernel.

Yes your most likely want the ability to backport dependable from the mainline branch to LTS branchs of the Linux kernels as your primary option and the developers own kernel tree as your secondary. Remember this order would to get decent quality drivers. Decent quality as in properly validate against common human coding errors and even against many highly complex coded errors.

Like it or not there is very valid reasons for drivers to be mainlined.

Heck even if a older version is mainlined to the one you are using at least that would still provide the Linux kernel developers with at least some idea of how the driver developer has interpenetrated how to use the Linux kernel API for drivers. This is important information so the Linux kernel developers know functions are used and by who. One of the bigger problems Microsoft windows kernel developers have historically admit to is not knowing what functions they are exporting in the so called stable driver API/ABI are in fact used with windows.

The hard point here is for Linux core kernel developers to make the exported functions to drivers decent they need at least reference driver code from driver makers to know what they are using and how they are using it.