Canonical To Focus On A New, More Modular Snapcraft - Current Codebase Goes Legacy

[Weasel]

Eh, unapproved post for tildearrow even though it contains no vulgarity, the hell?

[oiaohm]

List of imported libraries:
libA.so
libB.so

List of symbols to be resolved:
func1
func2
func3
func4

This is what happens when someone has not looked at the original LD ELF from solaris

List of imported libraries. Is not a strait forwards on the orgianl. You have shim libraries. Yes these are libraries of unique so name with directive to overload.

libc.so
shimlibc.so

So the list of imported libraries application order in case you are using more than shim as well as list of libraries. This list to be run though in the original when the init .so functions and this allows shims to in face de-register symbols as well. (don't try deregister of symbols with glibc limited loader it really hate you)

Yes symbol table.
STT_FILE libc,so
STT_FUNC func1
With the imported libraries like I listed func1 might in fact be from shimlibc.

That's it. You can use LD_PRELOAD and get the symbols from another library instead (since it's loaded first), which is actually a hack but whatever.

This on the original is not a simple hack. LD_PRELOAD is how solaris had you loaded compatible mode shims. Remember this is to work in the case you have used STT_FILE as well.

Weasel you have to have method to load compatible mode shims.

Th original elf loader has a heck load of functionality.

List of support missing
1) segregated linking be able to be declared in the symbol table. That the STT_FILE bit.
2) ability to handle shims/filters properly yes this incldue shims that target a particular .so name that can delete and replace functions.

Filters that plug into the ld.so we don't have with glibc provide ld.so either. Yes global symbols can cause problems. There is second side here that we are lacking the tools to address these problems.

Lets take your libA/B example and say that func1 exists in both. ld.so filters/shims solaris supports you can LD_PRELOAD and prevent on of the funct1 from being registered for the main application and segregated load one or both of those libraries This ld.so filters are directly connect the solaris loader.


Thing is ld.so solaris had systems to handle every problem global symbols could cause. Glibc ld.so and Musl ld.so does not.

Weasel you want segregated linking all the time. Please note segregated linking is not free of cost in memory or disc space. UEFI using PE and nulling out the dll names on functions same reason space. Global symbols have space saving not to be ingored.

SUN ELF was design to have global symbols and have the tools to deal with global symbols problems. Segregated linking is a core part. Next is STT_FILE and next is the ablity to put shims/filters into the ld.so from the preload to allow correct any symbol problem by basically taking control of the loader.

On top of this you are still needing the SXS manifest information for dealing with applications from where ever.

[ssokolow]

They should just reboot Snapcraft as a Canonical-managed Flatpak repo, same as they did with Mir and Wayland.

I missed this before but, given what they're doing with snappy, it'd be more correct to say they should reboot it as a Canonical-managed OSTree repo. Flatpak makes the runtimes mandatory so, for OS infrastructure, you need to drop down to the OSTree system it's built on, as distros like Fedora Silverblue do. Snappy blurs those two concerns.

[Vermilion]

Flatpak makes the runtimes mandatory so, for OS infrastructure, you need to drop down to the OSTree system it's built on. Snappy blurs those two concerns.

Snappy also makes runtimes mandatory. But while Flatpak gives you the choice (Freedesktop, GNOME, KDE, or even make your own like elementary), Snap binds you to an Ubuntu base (i.e. core18, core20, ...)

[ssokolow]

Snappy also makes runtimes mandatory. But while Flatpak gives you the choice (Freedesktop, GNOME, KDE, or even make your own like elementary), Snap binds you to an Ubuntu base (i.e. core18, core20, ...)

So you're saying that Flatpak is unsuitable for core system components, Snappy is suitable but tied to an Ubuntu runtime for it, and then OSTree goes further by basically just being an infrastructural component that can be used as a building block for any Nix/Guix-ish distro you want to make?

[Vermilion]

So you're saying ...

No, I said none of this gibberish.

[ssokolow]

No, I said none of this gibberish.

Well, then I don't know what you're saying... because the Flatpak people and Canonical agree that the difference between Flatpak and Snappy is that Snappy can be used for infrastructural components while Flatpak can't, because you're supposed to go direct to the underlying OSTree if you want to use that system for building the OS Flatpak depends on, sort of like std (requires OS services like threading, a filesystem, etc.) vs. core (can be used on bare metal) in the Rust standard library.

[Weasel]

This is what happens when someone has not looked at the original LD ELF from solaris

List of imported libraries. Is not a strait forwards on the orgianl. You have shim libraries. Yes these are libraries of unique so name with directive to overload.

libc.so
shimlibc.so

So the list of imported libraries application order in case you are using more than shim as well as list of libraries. This list to be run though in the original when the init .so functions and this allows shims to in face de-register symbols as well. (don't try deregister of symbols with glibc limited loader it really hate you)

Yes symbol table.
STT_FILE libc,so
STT_FUNC func1
With the imported libraries like I listed func1 might in fact be from shimlibc.

You're missing the point. It's not important where the symbol is from, what matters is that the loader is not given this information in bad ELF files using global symbols. There's nothing the loader can do here because it simply does not have this information, it's not the loader's fault. Well, other than not outright rejecting such files and flagging them as broken, refusing to load them. That should teach them a lesson.

Of course the loader can ignore the library name in PE too, that would be the loader's fault. But the point is that the information of what library the symbol is from is not lost, it's always recorded in PE, because that's how it is structured (i.e. it's mandatory).

On top of this you are still needing the SXS manifest information for dealing with applications from where ever.

SXS manifest is a perfect example of what i mean. It's just extra information, i.e. it supplies the exact library version it wants (not just the library name), with checksum and all that. The loader can ignore this. For example, if it can't find the exact version of the library, it tries the closest one that it can find, or if none in WinSxS, it loads the standard one from system32.

But the thing is that it has the information available so it can make the proper choice. Without the information, it's guesswork at best. There's no magic here.

Global symbols are cancer because they strip out the library information.

[ssokolow]

Of course the loader can ignore the library name in PE too, that would be the loader's fault. But the point is that the information of what library the symbol is from is not lost, it's always recorded in PE, because that's how it is structured (i.e. it's mandatory).

[...]

Global symbols are cancer because they strip out the library information.

Honestly, it sounds like the GNU and BSD people would have just written zero-length content into that field if you'd given them PE instead of ELF. You can't fix coders determined to make bad decisions.

[oiaohm]

You're missing the point. It's not important where the symbol is from, what matters is that the loader is not given this information in bad ELF files using global symbols. There's nothing the loader can do here because it simply does not have this information, it's not the loader's fault. Well, other than not outright rejecting such files and flagging them as broken, refusing to load them. That should teach them a lesson.

Weasel there is a lot extra information on elf.

Honestly, it sounds like the GNU and BSD people would have just written zero-length content into that field if you'd given them PE instead of ELF. You can't fix coders determined to make bad decisions.

Yes ssokolow is right on the money but its not just what they would do to PE but what they have in fact done to elf. Yes those using PE in UEFI have nulled out the value.

https://docs.oracle.com/cd/E26502_01...er6-54676.html this is solaris elf symbol versioning.
https://refspecs.linuxfoundation.org...ymversion.html This is Linux elf symbol versioning.

vn_file Offset to the file name string in the section header, in bytes.

Guess what is populated with NULL. Yes this vn_file should be the soname of the of where the symbol is from. Yes this should be a reuse of what is in the list of so the program needs in elf. Yes this consumers 4 bytes if null and 4 bytes of pointing the the soname.

vn_file being populated did not automatically trigger segregated linking.

Of course the loader can ignore the library name in PE too, that would be the loader's fault. But the point is that the information of what library the symbol is from is not lost, it's always recorded in PE, because that's how it is structured (i.e. it's mandatory).

No when you get to UEFI PE they null it out so the library the symbol is from is not recorded and this is not recorded just as much as ELF is. When you use modern Glibc using a modern linker you will always have ELF with the version tables.

So in elf you have STT_FILE that for ELF pre using version symbols and to declare this must be segregated loaded if using a Solaris elf loader . Then you have vn_file in the versioned synbol table of "Version Needed Entries" if you are using versioning should be populated but you vn_file find it populated with NULL that glibc and musl loader accepts this Solaris elf loader on the other hand throws error for incomplete symbol versioning and fail to load the program without shim to replace those tables..

Weasel its a different layout to PE true. But what you are request is in the ELF format if those implementing linkers and loaders decide to support it.

Yes Solaris loader supports supports preloading a .so file with replacement symbol version tables for the application. So quite powerful shimming ability here that we also don't have with glibc and musl elf loader. Yes form of solaris shim you could replaced the list of needed so files as well completely. Lot more functional loader lot less total road block problems.