Linux Features Loved By Microsoft Engineers Working On WSL2

[mdedetrich]

Again no, this is not due to design but down to choice, and that on both issues.

The firs issue is that many distros have decided to keep the kernel version for the duration of the version of their distribution. This is not demanded by anything technical, it is just a decision that was made by the distro. nVidia works better here because they have decided to use DKMS which allows they do build for a number of kernel versions at the same time while e.g AMD have decided not to use DKMS and instead only supporting the current kernel version that they happen to work on at the moment. So again it all boils down to decisions and not to a limit in the design, the design supports both approaches. Some distros, like Ubuntu, have since moved on past this original decision and have added newer kernels to old LTS disto versions if the user so chooses (aka the HWE kernels). And on top of that you the end user can always compile your own Kernel, there are no technical- (or should we say design-) reasons that prevents it.

The reason why LTS is even a thing is because the kernel design is monolithic, that is under you explicitly backport features into older kernel versions (i.e. pay for RH) if you want stability you need to freeze an older kernel version.

The second issue have zero to do with any "outdated technical design thoughts". A micro kernel is not more or less outdated than a monolithic kernel and the differences between the two is also not why you don't see a stable kernel ABI for things like graphics drivers (e.g had Linux been a micro kernel then the micro kernel interface for the graphics drivers would have changed between versions anyway).

Which is why the most often used desktops (MacOS/Windows) have hybrid kernels because in cases (ergo graphics drivers) it makes sense to get the latest version of these regardless of what version of Windows/Mac you happen to be running (within reason, with Windows its over a decade)

No this is yet a decision, and the decision here is that the internal ABI for the most part is fluid which allows it to be changed if and when a better solution is found combined with a decision that external modules is frowned upon regardless of if they are open or closed.

I know that this decision is made, unfortunately the problem is that it was done in the 90's when graphics drivers were far simpler then they are today. While it makes perfect sense to have keyboard drivers in the tree, as is evident putting complete graphics drivers (i.e. not the basic VESA ones to bring up a display) into the tree is causing more problems than its solving.

Now for the Joe Average end user none of this matters, he or she just want to use their new shiny hardware. My end take on all of this is that solving that part lies in the hands of the distributions, and they should simply enable and allow more rapid and recent versions of both the kernel and mesa if you are a distro vendor that wants to reach out to the public at large and not just to office workstations where the IT department certifies certain hw that they then will run on for the next 10/15 years (e.g Red Hat belongs here but the issue is that far too many distros looks at Red Hat to see how things should be done).

Oh of course it matters, for the average joe the situation is complete shit. They have some Linux distro (they may not even know what distro they have, or what a distro even is), they buy a new graphics card and they cannot even use it properly. Or they release they need valve's new shaders to run some steam game but those fixes have only been upstream into the latest AMD graphics driver. On Windows? They just go to AMD/NVidia website, click on download drivers and everything is setup for you. I also just checked NVidia's driver download page, and their latest drivers support Windows 7 which was released 14 years ago.

Stop putting the blame on Linux distro's, this problem is entirely the fault of the Linux kernel. The whole design of Linux kernel allows distro's of different types since it just a kernel, with some distros being rolling and some being LTS and because of this its Linux's responsibility to be flexible unless Linux wants to go down the BSD route which means that its not only its own kernel but also its own distro and I don't see this happening anytime soon.

This reason is one of the major ones why Linux desktop (and by desktop I mean using Linux on a standard desktop PC, not Steam Deck which is a vertically integrated device with a custom Linux fork) will not make it and its also one of the main reasons why I don't recommend Linux to a lot of users, especially gamers unless they ironically have an NVidia device.

[horizonbrave]

I'd love them to be bold and grab under sponsorship one big project that we love, that is essential and need completion and inject a billion money into them: like Wayland or Gnome. Some real philanthropy

[Sonadow]

Oh of course it matters, for the average joe the situation is complete shit. They have some Linux distro (they may not even know what distro they have, or what a distro even is), they buy a new graphics card and they cannot even use it properly. Or they release they need valve's new shaders to run some steam game but those fixes have only been upstream into the latest AMD graphics driver. On Windows? They just go to AMD/NVidia website, click on download drivers and everything is setup for you. I also just checked NVidia's driver download page, and their latest drivers support Windows 7 which was released 14 years ago.

Stop putting the blame on Linux distro's, this problem is entirely the fault of the Linux kernel. The whole design of Linux kernel allows distro's of different types since it just a kernel, with some distros being rolling and some being LTS and because of this its Linux's responsibility to be flexible unless Linux wants to go down the BSD route which means that its not only its own kernel but also its own distro and I don't see this happening anytime soon.

This reason is one of the major ones why Linux desktop (and by desktop I mean using Linux on a standard desktop PC, not Steam Deck which is a vertically integrated device with a custom Linux fork) will not make it and its also one of the main reasons why I don't recommend Linux to a lot of users, especially gamers unless they ironically have an NVidia device.

That's actually the sole reason I have not bought an Alder Lake laptop yet.

I built my own custom 5.15 kernel to support my Jasper Lake laptop because the distribution I am using came with 5.10 and it was a real drag. While I can definitely do it, I really don't want to waste another three hours of my life building a 6.3 kernel with the latest libdrm and Mesa just to support an Alder Lake laptop.

[mdedetrich]

That's actually the sole reason I have not bought an Alder Lake laptop yet.

I built my own custom 5.15 kernel to support my Jasper Lake laptop because the distribution I am using came with 5.10 and it was a real drag. While I can definitely do it, I really don't want to waste another three hours of my life building a 6.3 kernel with the latest libdrm and Mesa just to support an Alder Lake laptop.

Yeah and it will be a cold day in hell when average Joe's start building/compiling their own kernel.

[F.Ultra]

The reason why LTS is even a thing is because the kernel design is monolithic, that is under you explicitly backport features into older kernel versions (i.e. pay for RH) if you want stability you need to freeze an older kernel version.

No the reason why LTS is a thing is because IT departments at big enterprises wants to certify a set of sw+hw and then run on that for 10-20 years while still getting bug fixes. Micro of Monolithic have zero to do with this. I develop and sell software/services to such enterprises and they do not even want new versions of user space applications, let alone drivers/kernels.

Which is why the most often used desktops (MacOS/Windows) have hybrid kernels because in cases (ergo graphics drivers) it makes sense to get the latest version of these regardless of what version of Windows/Mac you happen to be running (within reason, with Windows its over a decade)

All such enterprises that I have seen and supported over the years that use Windows have never upgraded the graphics drivers past the initial install, once their IT department certified version x.y.z of amd/nVidia/matrox then that was the version of the drivers that they used, this is why MS offers a way for such departments to lock down windows update so people can blacklist updates for their entire enterprise.

Oh of course it matters, for the average joe the situation is complete shit. They have some Linux distro (they may not even know what distro they have, or what a distro even is), they buy a new graphics card and they cannot even use it properly. Or they release they need valve's new shaders to run some steam game but those fixes have only been upstream into the latest AMD graphics driver. On Windows? They just go to AMD/NVidia website, click on download drivers and everything is setup for you. I also just checked NVidia's driver download page, and their latest drivers support Windows 7 which was released 14 years ago.

I never said that it doesn't matter for the average Joe, I said that the reasons behind at doesn't matter to the average joe since all they want is working hardware.

Stop putting the blame on Linux distro's, this problem is entirely the fault of the Linux kernel. The whole design of Linux kernel allows distro's of different types since it just a kernel, with some distros being rolling and some being LTS and because of this its Linux's responsibility to be flexible unless Linux wants to go down the BSD route which means that its not only its own kernel but also its own distro and I don't see this happening anytime soon.

So the fault is the kernel while the design of the kernel at the same time allows distro's of different types? Both cannot be true at the same time...

There is also a third distro option here between rolling and LTS and that is being somewhat LTS for the userspace but rolling for the Kernel and mesa. And here Ubuntu comes quote close, even on 22.04LTS I can go as high as 6.1 of the kernel without compiling one myself and default mesa is at 22.2 so no problem using the latest graphics cards from either nVidia or AMD.

[Sonadow]

Yeah and it will be a cold day in hell when average Joe's start building/compiling their own kernel.

And the worst part of it? You can really only compile your own kernel on x64 hardware because it's so standardized and homogenous.

On heterogeneous platforms like ARM and PPC64LE you are basically shit out of luck if you need a new kernel for newer drivers. Your dtbs won't load in any other kernel outside of the heavily-patched vendor-supplied kernel. Have fun booting up to a dead kernel. Hell, even Microsoft's Project Voltron ARM developer kit that uses standard UEFI with ACPI couldn't even boot a random ARM Linux distribution.

[WiR3D]

So basically what I'm getting is that Linux needs to become a hybrid kernel.

It can have everything in tree, but SOME parts of the tree should be able to update independently without breaking the whole thing. And the key one here is basically the DRM subsystem.

So basically for good user experience on desktops we want:
Linux
Linux-headers
Linux-firmware
Linux-drm

I know enough to be dangerous here, I read the dri-devel list because it merely interests me, but surely this is possible and still aligned with Linux goals, and users will benefit.

Surely this is something the ecosystem should be pushing for?

Unless there's an ideological nuke I'm avoiding in my bunker.

[WiR3D]

I'd even be cool with some features of new drm drivers being disabled because the main kernel isn't new enough.

And atleast this gives major Linux versions a real reason for their existence other than just the minor version being too high

[Sonadow]

So basically what I'm getting is that Linux needs to become a hybrid kernel.

It can have everything in tree, but SOME parts of the tree should be able to update independently without breaking the whole thing. And the key one here is basically the DRM subsystem.

So basically for good user experience on desktops we want:
Linux
Linux-headers
Linux-firmware
Linux-drm

I know enough to be dangerous here, I read the dri-devel list because it merely interests me, but surely this is possible and still aligned with Linux goals, and users will benefit.

Surely this is something the ecosystem should be pushing for?

Unless there's an ideological nuke I'm avoiding in my bunker.

Add a Linux-networking, and even better; split it up to Linux-wired and Linux-wireless

And for all of this to even work, you need a dead-stable kernel ABI such that anything that goes on in Linux-drm and Linux-networking will always be able to communicate properly with the kernel regardless of how much they have changed, AND Linux-drm and Linux-networking must also have a dead-stable ABI such that any binary drivers released for Linux-drm and Linux-networking have at least a 10-year shelf life.

[WiR3D]

I guess what I'm ignorantly wondering is how big that ABI would be for this.