Canonical To Focus On A New, More Modular Snapcraft - Current Codebase Goes Legacy

[oiaohm]

Wrong. Most of the ABI/API is stable, it's just small differences in behavior. Wine has only a couple dozen DLLs in winsxs and works just fine for 95% of apps, so stop this bullshit.

Maybe there's 5% rare cases but you know what? When Linux libs do not break 95% of the apps using them, you can start talking about Windows. Got it?

No this is you who don't want to look at the data.

https://abi-laboratory.pro/index.php?view=abi-tracker

This here has been running for over a decade tracking the stability of the Linux userspace ABI. Just like the wine test project. When you compare compare the collected data Linux user space is more stable with less small differences in behaviour.

This is also shown with flatpak where you can force applications to run with a different runtime to what they were made with an as long as the runtime is newer over 99% of the time it works without issue as long as a particular handful of .so files are not in the mix. Yes there is a white paper out there were a person test this. They went in with the mistake idea they would prove Linux was totally faulty and I give them credit for being truthful.

Now this changes the question. If Linus userspace is technically more ABI stable than Windows why is it worse for application developers?

1) Segregated Dynamic Linking is default under windows. Under Linux its there but developers have to go out of their way to have it by creating wrapper libraries. Yes glibc provide dlmopen that is core functionality to-do this.
2) Lack of a SXS system to allow handful of .so files to have multi versions.

ABI stability is high on Linux. The toolkit on Linux to handle the cases when its not stable is very poor. Windows ABI is not as stable that kind of surprise to lot but it has great tools to handle the cases when its not as long as people follow the build instructions.

Yes there is even tooling built into the Microsoft MSVC C++ runtimes this is why if you application is built with newer version of C++ runtime it has less dealing with libraries using MSVC older but that does not hold for the reverse.

Its really annoying when from the data you can see how close Linux user space ABI is to developer friendly one yet its not one. Its over 90% there. Its the key 10% that is missing causing 90% of the problem. Yes that key 10% is the little bit of toolkiting to handle the cases of incompatibility well.

[benjiro]

And a huge whack of that 32GiB is WinSxS...

I just checked my Windows installation. Windows itself is only 19GB after 3 years of constant using, apps installing and upgrades. So its not a clean system but heavily "populated". This includes a bunch of older programs, .net, game ( inc old ones with old support files ). Fyi: WinSxS is 7GB, System32 is 5GB and the rest is divided amongst different folders like 700MB of fonts. Your Linux distro install is also going to be less clean after a few years

Windows itself does NOT use 32GB for the installation, it never has. What does use a lot of space are the pagefile, swapfile, hybernation, things that people also conveniently forget when talking about any Linux installation size. Like if its a separate partition, yea, that 16GB swap partition somehow does not count

Ironically, in regards to sandbox packages, MS is going more and more sandboxed with W11 ( they have for a long time ). But unlike the above mentioned package managers, MS always tries to integrate the experience to the point it seems native. A good example is how WSL2 despite being a literal VM, does not feel like a VM with CLI-WSL2/Explorer-WSL2/WSL2-FS/VSC-WSL2 etc integration.

Every time its some fights about snap, appimages, flatpak, bla bla... instead of working on a standard solution, it always needs to take years upon years, for a "winner" to be decided while everybody fights like toxic beavers. Every freaking change or improvement, its like watching old grumpy men, all oiled up fighting it out in bikini strings. *ieeeewww*... People with way too much free time on their hands.

[Weasel]

No this is you who don't want to look at the data.

https://abi-laboratory.pro/index.php?view=abi-tracker

This here has been running for over a decade tracking the stability of the Linux userspace ABI. Just like the wine test project. When you compare compare the collected data Linux user space is more stable with less small differences in behaviour.

This is also shown with flatpak where you can force applications to run with a different runtime to what they were made with an as long as the runtime is newer over 99% of the time it works without issue as long as a particular handful of .so files are not in the mix. Yes there is a white paper out there were a person test this. They went in with the mistake idea they would prove Linux was totally faulty and I give them credit for being truthful.

Now this changes the question. If Linus userspace is technically more ABI stable than Windows why is it worse for application developers?

I don't think you understand what ABI stability means. Test suite on wine for example verifies behavior, changes/failures there are usually subtle and in very rare cases (possibly not real-world at all, no app should care).

ABI instability on Windows would mean, for example, dropping gdi32 because it's "old" and replacing it with something else, like dwrite. dwrite already exists, but gdi32 is still there and exports the same fucking symbols (if not more than before). This is the instability in Linux userland.

GTK2 for example is "ABI stable" but the problem is that you have to compare it to its successors that replace it. GTK2 and GTK3/4 are completely incompatible, so much so that even if they share symbols, you'd end up crashing the motherfucking app when calling them because of completely different signatures etc (trashing the stack in the process).

And then on Linux userland you have moron distributions who "drop" old versions of libraries (again, same as MS dropping gdi32, can you imagine the fallout?), this is why it's hostile to developers compared to Windows.

They need to STOP REMOVING the fucking libraries or symbols and, as things progress, add more instead, but keep the old ones, even if deprecated.

You can't tell me with a straight face by looking at the horrible mess graphics toolkits on Linux, or ffmpeg, or whatever other trash project that changes ABI more often than some people change socks. "Stick with older version" does not work when the distribution retards remove them from their repositories. That's the problem with Linux userland.

[ssokolow]

Every time its some fights about snap, appimages, flatpak, bla bla... instead of working on a standard solution

Where do you think the XDG portal system used by both snappy and Flatpak and built into GTK and Qt came from? Hint: Flatpak used to be called xdg-app.

[ssokolow]

I just checked my Windows installation. Windows itself is only 19GB after 3 years of constant using, apps installing and upgrades. So its not a clean system but heavily "populated". This includes a bunch of older programs, .net, game ( inc old ones with old support files ). Fyi: WinSxS is 7GB, System32 is 5GB and the rest is divided amongst different folders like 700MB of fonts. Your Linux distro install is also going to be less clean after a few years

I don't dispute that. I was never talking absolute numbers. 7GB out of 19GB is still 36.8%, which I'd call "a huge whack" of it.

[oiaohm]

I don't think you understand what ABI stability means. Test suite on wine for example verifies behavior, changes/failures there are usually subtle and in very rare cases (possibly not real-world at all, no app should care).

I do understand the difference.

ABI instability on Windows would mean, for example, dropping gdi32 because it's "old" and replacing it with something else, like dwrite. dwrite already exists, but gdi32 is still there and exports the same fucking symbols (if not more than before). This is the instability in Linux userland.

GTK2 for example is "ABI stable" but the problem is that you have to compare it to its successors that replace it. GTK2 and GTK3/4 are completely incompatible, so much so that even if they share symbols, you'd end up crashing the motherfucking app when calling them because of completely different signatures etc (trashing the stack in the process).

Except that is happens and oranges. gdi32 equal would be xlib in Linux that does not break ABI. GTK2 is closer to MFC and yes that has broken a lot. Not all libraries for ABI stability are created equal even under Windows.

You can't tell me with a straight face by looking at the horrible mess graphics toolkits on Linux, or ffmpeg, or whatever other trash project that changes ABI more often than some people change socks. "Stick with older version" does not work when the distribution retards remove them from their repositories. That's the problem with Linux userland.

Graphical toolkits. This is interesting point. You have applications that use Qt under windows that ship with qt right. You have applications that use ffmpeg they ship with ffmpeg. Starting to see a trend here. There bundling is expected as part of making windows applications this need to remain the same with Linux ones. I will give you this is not the only problem so I will show you deeper into ffmpeg.

https://abi-laboratory.pro/index.php?view=timeline&l=ffmpeg

This one is interesting. ffmpeg the breaking changes on API/ABI that not internal only breaks when the Soname changes. As in dll/so name changes. So this is in fact design that you can install multi versions on a system and have it work. Then when you look at distributions you will find ffmpeg packaged in way to install multi versions. So far no sign of a API/ABI problem.

Yes the lack of Segregated Dynamic Linking at this point gives you a problem that new symbols added to a .so might conflict with one of your own. This is quite a major API/ABI problem here. But this is not API/ABI unstable. Yes if you look at wine imports you will see multi sonames lists and this is where the functions wine needs is provided by different versions of the same library yes those functions happen to be 100 percent API/ABI compadible. Wrapper .so that libcapsule guides you to make for Segregated Dynamic linking allows this.

"Stick with older version" does not work when the distribution retards remove them from their repositories

This is linked to SXS and Segregated Dynamic Linking being missing. SXS allows Microsoft to have a libraries on the system that unless your application manifest says they need it you will not link against those libraries. This fixes a problem.

Now lets look at a Linux Distribution. How does a Linux distribution prevent linking to a old library when building a new application. That right the delete the old library. as soon as none of the application the distribution ships needs that library any more.

Microsoft does remove old versions of dll from the system32 and into SXS and when Microsoft finally thinks the library is no longer used they then delete it from the SXS this has results in applications not working on windows 10 that work on windows 7 and applications that work on windows 10 not working on windows 11. This functionality is totally missing when you get on Linux distributions so there is not a delay with the removal. Thing to remember Microsoft delay from when the library was moved into SXS and its final delete is roughly 6 years.

Linux distributions you have no delay on removal as soon as they believe you are .so is no longer need it gone. This is not the API/ABI being unstable. This is a runtime provision by distribution problem that links back to SXS and Segregated Dynamic Linking problems.

Weasel like it or not the API/ABI stability exist on linux. Problem here libraries can coded to provide all the API/ABI stability you need https://abi-laboratory.pro shows this to be the case yet it not end up usable delivered due to really two things.

Yes SXS manifest is really way to guide Segregated Dynamic Linking

https://docs.microsoft.com/en-us/windows/win32/sbscs/about-side-by-side-assemblies-

Yes SXS is a feature of the windows dynamic loader that the glibc dynamic loader does not have. This is not something the elf format forbids either. Just the glibc and musl elf loaders have never implemented it.

Yes that manifest of SXS allows the windows dynamic loader to be smart as well. So application linked against old version of library and its noted in the mainfest if SXS knows new version of that libraries is ABI compatible it can have the application use the new version of the library and if its not ABI compadible and the SXS does not have the old library the program fails to run so no data losses.

Weasel the reality you called the project providing the libraries trash they are not the problem in 99.9% of the cases with Linux. Yes 99.0% problem is in the dynamic loader.because its missing functionality with Linux distributions and 0.9 percent is really distribution management,

[Weasel]

Except that is happens and oranges. gdi32 equal would be xlib in Linux that does not break ABI. GTK2 is closer to MFC and yes that has broken a lot. Not all libraries for ABI stability are created equal even under Windows.

xlib is one of the few sane libraries, yeah. MFC is a redistributable library; it lives standalone and can be supplied with an app. Contrast this with Linux where libraries come from a central location (either flathub or a distro repository or whatever). See the difference?

Graphical toolkits. This is interesting point. You have applications that use Qt under windows that ship with qt right. You have applications that use ffmpeg they ship with ffmpeg. Starting to see a trend here. There bundling is expected as part of making windows applications this need to remain the same with Linux ones. I will give you this is not the only problem so I will show you deeper into ffmpeg.

https://abi-laboratory.pro/index.php...eline&l=ffmpeg
This one is interesting. ffmpeg the breaking changes on API/ABI that not internal only breaks when the Soname changes. As in dll/so name changes. So this is in fact design that you can install multi versions on a system and have it work. Then when you look at distributions you will find ffmpeg packaged in way to install multi versions. So far no sign of a API/ABI problem.

Yes the lack of Segregated Dynamic Linking at this point gives you a problem that new symbols added to a .so might conflict with one of your own. This is quite a major API/ABI problem here. But this is not API/ABI unstable. Yes if you look at wine imports you will see multi sonames lists and this is where the functions wine needs is provided by different versions of the same library yes those functions happen to be 100 percent API/ABI compadible. Wrapper .so that libcapsule guides you to make for Segregated Dynamic linking allows this.

Yes ffmpeg and Qt are a mess on Windows too but they're standalone. They are built on top of the Windows APIs, they don't pull extra dependencies, and so on.

Microsoft does remove old versions of dll from the system32 and into SXS and when Microsoft finally thinks the library is no longer used they then delete it from the SXS this has results in applications not working on windows 10 that work on windows 7 and applications that work on windows 10 not working on windows 11. This functionality is totally missing when you get on Linux distributions so there is not a delay with the removal. Thing to remember Microsoft delay from when the library was moved into SXS and its final delete is roughly 6 years.

Sorry but this is complete nonsense, can you give some proof or you just rambling?

Give examples of DLLs removed and which apps it affected.

Weasel the reality you called the project providing the libraries trash they are not the problem in 99.9% of the cases with Linux. Yes 99.0% problem is in the dynamic loader.because its missing functionality with Linux distributions and 0.9 percent is really distribution management,

Don't get me wrong, I agree the dynamic loader is also a huge problem. In fact, remember how I used to diss ELF in the past and praise why DLLs are superior?

That's because DLLs force the use of symbols attached to a library name when looking them up, instead of having the option to load "global symbols", which is why ELF is trash because almost everyone uses global symbols with ELF. It's literally like trash code by beginners that uses global variables because "easier than using classes" and having it be a time bomb in the future when clashes are bound to happen.

The dynamic loader should have outright banned all ELF files that use global symbols with no library attached as "malicious".

[tildearrow]

xlib is one of the few sane libraries, yeah. MFC is a redistributable library; it lives standalone and can be supplied with an app. Contrast this with Linux where libraries come from a central location (either flathub or a distro repository or whatever). See the difference?

Yes ffmpeg and Qt are a mess on Windows too but they're standalone. They are built on top of the Windows APIs, they don't pull extra dependencies, and so on.

Sorry but this is complete nonsense, can you give some proof or you just rambling?

Give examples of DLLs removed and which apps it affected.

Don't get me wrong, I agree the dynamic loader is also a huge problem. In fact, remember how I used to diss ELF in the past and praise why DLLs are superior?

That's because DLLs force the use of symbols attached to a library name when looking them up, instead of having the option to load "global symbols", which is why ELF is trash because almost everyone uses global symbols with ELF. It's literally like trash code by beginners that uses global variables because "easier than using classes" and having it be a time bomb in the future when clashes are bound to happen.

The dynamic loader should have outright banned all ELF files that use global symbols with no library attached as "malicious".

The solution to this problem is not on ELF. It's on ld.

Here, something that would work, on the symbol table:
- libA.so
- func1
- func2
- func3
- libB.so
- func4
- func2 (conflict?)

Under normal conditions symbols would be stored as:
func1
func2
func3
func4

and crash since libB.so func2 is missing.

But my proposal is to store them like this:
libA_so/func1
libA_so/func2
libA_so/func3
libB_so/func4
libB_so/func2

No need to change or drop ELF. Just change the loader a bit to handle this.

[oiaohm]

xlib is one of the few sane libraries, yeah. MFC is a redistributable library; it lives standalone and can be supplied with an app. Contrast this with Linux where libraries come from a central location (either flathub or a distro repository or whatever). See the difference?

No there is not a difference. MFC different versions is provided with windows this is why some applications that don't include the re-distributable for MFC still work.

Lack of SXS makes providing own runtime more tricky. flathub core freedesktop runtime does not contain gtk or qt. A user with a full KDE desktop may not have GTK installed same with a user with a full Gnome desktop. Neither Qt or Gtk are in fact core Linux api for a graphical desktop.

Yes ffmpeg and Qt are a mess on Windows too but they're standalone. They are built on top of the Windows APIs, they don't pull extra dependencies, and so on.

Both ffmpeg and qt can in fact be built standlone on Linux and package this is demoed by a few flathub packages using only the freedesktop runtime.

That's because DLLs force the use of symbols attached to a library name when looking them up, instead of having the option to load "global symbols", which is why ELF is trash because almost everyone uses global symbols with ELF. It's literally like trash code by beginners that uses global variables because "easier than using classes" and having it be a time bomb in the future when clashes are bound to happen.

Sorry PE format does not in fact absolutely force it. DLL built for different UEFI solutions simple null out the library name for global symbols that could be from any library the UEFI has loaded. Since the loader in different UEFI solutions support this it happens to work.. Why would they implement this symbol overloading so that the load library 1 with func x lib 2 with func x and now run application and it sees library 2 func x.

The dynamic loader should have outright banned all ELF files that use global symbols with no library attached as "malicious".

global symbols have a use case like it or not. Bigger problem is that glibc dynamic elf loader does not support STT_FILE in the dynamic symbol table and the common linkers don't populate this for dynamic linking. Yes the populate it when static linking...... So a static linked elf you can tell what library a symbol come from because there are STT_FILE entries with the static libraries name in the local symbols table. Please note Solaris OS where elf in fact comes dynamic loader does in fact support using STT_FILE to declare .so name in the dynamic table and when you do it functions exactly like PE does on Windows include since STT_FILE was declared segregated linking. Since segregated linking was a originally feature of ELF this is why implement dlmopen was not that bad. But they did not go on to implement the STT_FILE bit.

The elf you see on Linux is a very bad clone of the original ELF implementation with many limitations that should not be there.

So the elf format has entry for declaring this symbol comes from X library just it not being populated. Yes this is the same has what happens with PE with UEFI where they decide not to populate the dll name entry..

Really a dynamic loader that does not support both use cases put limitations on how it can be used.

The reality here is the ELF format was very well designed at Sun. Just we are dealing with incomplete implementations on Linux and BSD this does create some head aches that should not be. Core to fixing this limitations is fixing the dynamic loader. Yes the segregated linking done by libcapsule/valve that to half implemented compared to what the old Solaris implementation offered.

Weasel windows has compatible mode to allow old applications to work this is also about having a collection of old out of date librares that applications can be exposed to. Yes people have complained about applications no longer working on Windows 10 and Windows 11.... Legacy libraries do get removed from windows like it or not. Just the speed is slower. This is slower because you SXS and compatible mode stuff in the windows dynamic loader.

There is one major problem child parts causing majority of the problem the elf dynamic loaders that Linux and BSD platforms use. Yes you incorrectly presumed it was the ELF format because you never looked at the original to see how it was implemented. Like if I used UEFI PE from one of the cursed that use nulled out names I would be able to claim all the same things as PE limitations that you have been claiming against ELF Weasel there is no difference here really both the UEFI PE loaders and the Linux ELF dyanmic loaders are poor copies of something else.

[Weasel]

The solution to this problem is not on ELF. It's on ld.

Here, something that would work, on the symbol table:
- libA.so
- func1
- func2
- func3
- libB.so
- func4
- func2 (conflict?)

Under normal conditions symbols would be stored as:
func1
func2
func3
func4

and crash since libB.so func2 is missing.

But my proposal is to store them like this:
libA_so/func1
libA_so/func2
libA_so/func3
libB_so/func4
libB_so/func2

No need to change or drop ELF. Just change the loader a bit to handle this.

Yes that's exactly what I mean. Although, it doesn't crash for not finding it, it simply uses func2 from the first library it resolved it from, so it will call the wrong function, which arguably is even worse.

The thing is that it's not just ld that's the issue here. These ELF files literally don't store the library near the symbol, because they're built that way. They store it like this:

List of imported libraries:
libA.so
libB.so

List of symbols to be resolved:
func1
func2
func3
func4

That's it. You can use LD_PRELOAD and get the symbols from another library instead (since it's loaded first), which is actually a hack but whatever. The fact they don't come with STT_FILE (to tell what library it comes from) is the problem here. In fact this should be outright banned.

You can't do that with PE, which stores the symbol names as a list from the specific library's table, i.e. it's sanely designed. The loader could ignore the library name, of course, but that's more a problem with the loader being retarded. In this case, the loader simply does not have the information because the ELF files suck and are built without STT_FILE.

Those kind of ELF files should be banned/flagged as malicious so people actually start using it properly and attaching symbols to libraries instead.