A Look At Flatpak vs. Snap Adoption In Various 2018 Linux Distributions

[oiaohm]

I did not really try to think of a solution right now (not sitting on the designer seat) but yeah, thanks, of course, I am in favour of relative paths, and yes, that does agree (perhaps) with sandboxing,

and might always converge towards everything-in-one-package,

and I think the MacOS model for applications should be the model for how to approach this.

And I don't know everything, but in this case my Calligra build used libraries out of its own directory structure, so could have been relative just as easily.

And I am not here to suggest a total solution, but I am just saying that I know that a good approach to "full packages" first has to solve the relocation problem.

And solve it in a way that it does not just affect said sandboxing technology, but everything in Linux, ie. a standard feature you can use even without Snappy or Flatpak or whatever; something generic and in a sense universal.

Sandboxing allows you to use sandbox to control the paths application can see. So if application code is not using relative paths and the sandbox shows it the path it wants to see everything is ok. Problem you got with attempt to make application relative is the number of places that can be fixed. There is no standard for how overrides to this will be provided in the binary.

MacOS is not exactly great example. Rename dmg files and attempt to run the quite a few fail because the installer is in fact using fixed path of /volume/"dmg file name -.dmg"

https://github.com/flatpak/flatpak/wiki/Sandbox

Flatpak application to it self due to sandbox is in fixed path /app and the runtime it wants is /usr but reality on disc its in a ostree somewhere https://github.com/flatpak/flatpak/wiki/Filesystem

https://www.mankier.com/1/flatpak

You change the ostree location with environmental vars.

[Mavman]

Don't be too excited. You also have to install the runtime from somewhere (at least 0.5GB).

isn't that already included in most distro's?

[jm111]

Sandboxing allows you to use sandbox to control the paths application can see. So if application code is not using relative paths and the sandbox shows it the path it wants to see everything is ok. Problem you got with attempt to make application relative is the number of places that can be fixed. There is no standard for how overrides to this will be provided in the binary.

This is exactly what I'm saying: you are incapable of solving the lesser problem, so you just throw the bigger solution at it; ensuring the lesser problem will keep plagueing systems that don't use the bigger solution.

Thus, you get a huge split; all of sandboxing's benefits are only available in a sandbox, everything else has nothing.

And this is because the entire thing is still just a huge mess that you just manage to deal with because you separate it from other huge messes.

Making stuff relative to the ordinary root would be easy; anything in /usr/lib could reach /usr/bin, /usr/share with easy ../lib paths.

At that point, you can already move stuff to /usr/local, but you need a provision to resolve paths relative to __FILE__, so to speak.

I mean, someone's just gotta do it you know, it's not that difficult.

If your application (and the linker/loader whatever) resolves all /lib, /share, /doc, /sbin, /bin and /etc paths as relative links,

then the entire problem is already solved; provided you stick to that "subtree-format".

You can now move the subtree anywhere you want.

What about other libraries? Not important, my Calligra problem from before is now already solved.

[jm111]

Only way that non sandbox is really going fly is if the ABI difference between distributions can be fixed and ABI and Interface(dbus socket...) breakage can be stopped. Problem is I don't see that happening any time soon that being inside the next 10 years(I might be a little bit to pessimistic here but I don't think so). Sandbox path is method that is open to us in the short term.

The problem is that lesser problems have not yet been solved; not all applications require the full solution set, but even lesser demands are not met.

This means that competing sandbox solutions cannot use common 'base' solutions as a starting point, which leads to this whole "war of the sandboxes" thing, because there is no common solution anywhere.

For there to be a fully open, compatible, and widely supported solution, as much of it has to be 'common solution' as possible;

not something 'privately owned' by Canonical, and although flatpak had a better feel to me, it still is too big a solution for many common problems,

because the schism between regular packages is too big.

It's an all-or-nothing approach, and that's why it won't work.

Do you really think Flatpak could replace ALL of a distribution's packages? If not, why would you even attempt 10% unless absolutely necessary?

And if you wouldn't, it means flatpak remains relegated to independent distributors, a bit of a "compile once, run everywhere" sentiment (Java) --- the only attractiveness is multi-distro packages, but the host systems themselves end up a mess.

Distros will keep packaging their own primary packages while some upstream distributors will also provide flatpaks for those who don't have distro versions.

The amount of applications that really need sandboxing is limited; hence, Flatpak will only be used when this is totally required.

That's why it will always remain a fringe thing; it does not integrate well with the host system itself.

I don't want half my desktop applications running in a Flatpak, please no.

[oiaohm]

Do you really think Flatpak could replace ALL of a distribution's packages? If not, why would you even attempt 10% unless absolutely necessary?

Endless OS use OStree for core OS and flatpak for all non core applications. What endless shows is over 99 percent of distribution packages could be flatpak.

So there is basically a working example.

And if you wouldn't, it means flatpak remains relegated to independent distributors, a bit of a "compile once, run everywhere" sentiment (Java) --- the only attractiveness is multi-distro packages, but the host systems themselves end up a mess.

This is not the case with flatpak you have missed there is Endless and work with debian looking into how far flatpak packaging can go.

Distros will keep packaging their own primary packages while some upstream distributors will also provide flatpaks for those who don't have distro versions.

The amount of applications that really need sandboxing is limited; hence, Flatpak will only be used when this is totally required.

Totally required is the case with Endless os.

That's why it will always remain a fringe thing; it does not integrate well with the host system itself.

Really flatpak integration with host is always improving.

I don't want half my desktop applications running in a Flatpak, please no.

Those running endless already have 90 percent of their desktop applications running inside flatpak. And you are only talking about half.

The reality here is flatpak could force the host mess to sort out. If 90 percent of the application a user of distribution is using is coming from flatpak distributions breaking it comes not possible.

The difference between facing a collective vs a individual. Distribution is able to ignore 1 lone developer with a problem its far harder if they have many thousands in the one group that their alteration or lack of alteration has broken..

[jm111]

Endless OS use OStree for core OS and flatpak for all non core applications. What endless shows is over 99 percent of distribution packages could be flatpak.

Someone in this thread made the distinction between "could be" and "should be".

This is not the case with flatpak you have missed there is Endless

Personally if I had to give a complete computer nono a computer I would start with MacOS, then Windows, then Linux Mint, and only then something like Endless OS.

Of course, these days I would give a tablet instead; probably an iPad mini.

I am just saying you are not using an inspiring example.

Really flatpak integration with host is always improving.

And the sun is always shining, doesn't say a lot.

Those running endless already have 90 percent of their desktop applications running inside flatpak. And you are only talking about half.

Which is how many people?

The reality here is flatpak could force the host mess to sort out. If 90 percent of the application a user of distribution is using is coming from flatpak distributions breaking it comes not possible.

That's exactly what I said in the beginning: you do not solve the mess, you avoid it.

Not using a computer also solves Linux' mess.

But now you probably have new problems; although I have no flatpak experience, only snappy.
Snappy of course runs everything inside its own loopback mount; not ideal.

[oiaohm]

That's exactly what I said in the beginning: you do not solve the mess, you avoid it.

Not using a computer also solves Linux' mess.

But now you probably have new problems; although I have no flatpak experience, only snappy.
Snappy of course runs everything inside its own loopback mount; not ideal.

Maybe the best solution is avoid it.

Making applications relocatable in path does cause overhead in application code. Flatpak is using bind mounts and mount namespaces. The same stuff systemd uses to give services private temp directories. So they are quite well optimised.

Please also remember the file system you see as a user under Linux is called a "virtual file system" because it virtual is not based on hardware. All bind mounts and mount namespace does is give application their own private virtual file system. Other than start up overhead of creating the extra virtual file system the over head is basically non existent using bind mounts and mount namespace.

Coming from snap using loop back this is quite bad. Loop back results in requests having to be processed twice. One by the file system inside the loop back and once by the file system hosting the loop back. Yes mount namespace and bind mounts cost at start up general processing no real change.

https://lwn.net/Articles/747633/

If you were looking at loopback like you would be look to how snapshots are done in btrfs and are planing to be done in xfs to avoid insane overhead.

[jm111]

Making applications relocatable in path does cause overhead in application code.

As someone else said, that's insignificant. You have really bad arguments you know.

The MacOS argument was also irrelevant.

The "integration is always improving" was also a bad argument.

Flatpak is using bind mounts and mount namespaces.

Right, that's better than the loopback of Snappy I think.

The same stuff systemd uses to give services private temp directories.

But I absolutely hate systemd private temp directories, they are unnecessary, do not solve anything, leave junk behind, systemd doesn't clean it up, etc. etc. etc.

AND I DO NOT WANT STUFF NAMED SYSTEMD IN MY SYSTEM

But that's just me.

Private temp directories is just a huge ego trip for Mr. Poettering ---- systemd everywhere.

The directories are not cleaned up because it's supposed to be tmpfs, but /var/tmp gets littered because running systemd-tmpfilesd on it is way too aggressive for most distributions.

C processes can create unlinked, privately owned tmp directories with no issue, I really don't think we need "extra security" by hiding stuff from processes on the mount level, access rights are sufficient.

The private temp directories make it hard to troubleshoot stuff because even root can't see the files, and using namespaces is difficult.

Please also remember the file system you see as a user under Linux is called a "virtual file system" because it virtual is not based on hardware. All bind mounts and mount namespace does is give application their own private virtual file system. Other than start up overhead of creating the extra virtual file system the over head is basically non existent using bind mounts and mount namespace.

I never complained about efficiency overhead, but usability problems.

Coming from snap using loop back this is quite bad. Loop back results in requests having to be processed twice. One by the file system inside the loop back and once by the file system hosting the loop back.

I don't think the efficiency matters too much.

But unmounting stuff is another unsolved problem in Linux with many issues.

I like containment but I don't think applications should sit in their own filesystems.

Yes mount namespace and bind mounts cost at start up general processing no real change.

Agree with that.

[oiaohm]

But I absolutely hate systemd private temp directories, they are unnecessary, do not solve anything, leave junk behind, systemd doesn't clean it up, etc. etc. etc..

I agree with the mess clean up should be better. But private temp directories with particular services that happen to be closed source java they are a god send. Yes you have two services that create exactly the same temp files because they are based on exactly the same toolkit so only 1 would run without private temp directories.

But there is a more serous side.

https://nvd.nist.gov/vuln/detail/CVE-2018-1053

Yes postgresql this year was caught creating tempfile with wrong permissions. But every year there have been a few hundred programs making flaws like this.

So do not solve anything is not exactly true. SystemdPrivate temp does limit how exploitable tempfile creation stuff ups are.

C processes can create unlinked, privately owned tmp directories with no issue, I really don't think we need "extra security" by hiding stuff from processes on the mount level, access rights are sufficient.

If this was reliability done we still would not have CVE numbers over files in tmp directory being done with wrong privilege and other horrible mistakes.

The private temp directories make it hard to troubleshoot stuff because even root can't see the files, and using namespaces is difficult.

.
Root cannot see unlinked files directly anyhow even if you don't have private temp from systemd. But the fact application has a private temp namespace made with the way systemd temp works means it did create a temp file and you do need to look at /proc to see it.

https://www.freedesktop.org/software/systemd/man/tmpfiles.d.html

By the way systemd-tmpfiles is quite control-able. Even without private temp directories it can be used to patch up some application stuff ups on temp file permissions.

Private temp directories is just a huge ego trip for Mr. Poettering ---- systemd everywhere.

.
Like it or not Private temp in systemd is not a huge ego trip. Its something that was created when a developer who was not Poettering looked at recurring CVE issues. Yes applications with bad temp files is quite a recurring CVE issue. Then how can we make this less useful to attackers. Yes the result is little more painful for administration but there is a good reason for the pain. Now if you can work out a method that protects against the recurring CVE issues around tempfiles is more administration friendly love to hear it..

But unmounting stuff is another unsolved problem in Linux with many issues.

.
This is why namespace and bind mounts works so well. Bind mounts gets to play the pretend you deleted the directory card. Because a bind mount is not a real file system or have a fake block device behind so it can pull the magical disappearing directory. Yes the cached files are linked to the real file system hosting the ostree with flatpak like a normal ostree based OS that way.

Yes snap with loopback has umounting problems this is not a flatpak issue using bind mounts. Basically flatpak was smart here did not reinvent the wheel and picked up a already working solution. Of course this pushed flatpak min kernel version up. Higher min kernel version was the price for a working system.

ostree can be used to run a complete operating system with stacks of bind mounts https://ostree.readthedocs.io/en/lat...ting-existing/. Yes the fact ostree was used to run complete OS before flatpak used it means it was already very well tested with the mounting problems. Ostree still sorting out connection handling. Yes flatpak did not even really make their own bundle update system instead it using the Ostree update system.

I like containment but I don't think applications should sit in their own filesystems.

.

If applications are seeing their own file system it better be like how flatpak/ostree does it where its truly virtual and truly able to be destroyed when application closes without major issues. Heck you can destroy it before application closes if you don't care about upset application. Yes the ablity to handle it like deleting a directory instead of complex file system or block device unmount with cache flushing and the like makes everything lot nicer.

[the_scx]

isn't that already included in most distro's?

No, no distribution provides it. This is something that users must install on their own (using flatpak as package manager).

And now, for x86, we have at least the following runtimes:
org.freedesktop.Sdk/i386/1.6
org.freedesktop.Sdk/x86_64/1.6
org.gnome.Platform/i386/3.24
org.gnome.Platform/x86_64/3.24
org.gnome.Platform/i386/3.26
org.gnome.Platform/x86_64/3.26
org.gnome.Platform/i386/3.28
org.gnome.Platform/x86_64/3.28
org.kde.Platform/i386/5.9
org.kde.Platform/x86_64/5.9
org.kde.Platform/i386/5.10
org.kde.Platform/x86_64/5.10
org.kde.Platform/i386/5.11
org.kde.Platform/x86_64/5.11

I only mentioned currently supported environments. You can multiply this by two, because IDE will use the SDK instead of Runtime (e.g. GNOME Builder uses org.gnome.SDK). To be honest I have to admit that for games Freedesktop should be just enough (but of course, developers must provide tons of libraries on their own).