Announcement

Collapse
No announcement yet.

A Look At Flatpak vs. Snap Adoption In Various 2018 Linux Distributions

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Originally posted by Weasel View Post
    I don't see the problem. Plugins are already loaded with dlopen/dlsym, it's the only way since they "violate" the one-definition-rule by design. They won't be impacted in the slightest.

    What you linked is just an optimization to load functions on demand as they are used. You can do that with stubs already (i.e. patching code on the fly when it gets called the first time) and AFAIK it's even done in some cases. I'm saying that you should just preload everything (as long as it's not a plugin dynamically loaded by design, because it shares functions and interfaces). Negligible impact in memory usage compared to sandboxing..
    The overhead from Linux kernel namespaces on a systemd system is quite Negligible.

    Originally posted by Weasel View Post
    There's no confusing if all libraries used proper versioning, it will never load from the wrong library. Simple as that.

    Of course, distros name their stupid libraries arbitrarily and not like upstream, so you can't use the base packages. However, you CAN just bundle all dependencies and if some "base" library is used, and it depends on some other "base" package, it will use that without issue. Only thing is you'll have to be cautious how you name the "libraries deployed with the app" in this case, maybe add a prefix to them, so they will never conflict with the retarded distro decisions.

    Let's say your app depends on the distro's libc, so it loads that. Somehow the base libc depends on a lot of other libraries, let's say libfoobar, which get loaded from the system, and will never conflict with your app. All the other deps "bundled" with your app can be prefixed or whatever. This avoids sandboxing as there will be no conflicts. If your app has ALSO a different version of foobar that it depends on then just call it Lfoobar and bundle it with the app.
    https://github.com/AppImage/AppImage...latform-Issues << read this carefully.

    Reality is you cannot trust the distribution libc even because glibc deprecates and adds new functions. Depend on how glibc built will effect if you application runs or not. Distributions can do stupid things it library naming and library versioning the reality since they can they do. Basically you end up needing a full runtime independent to distribution to have the least random issues.

    The reality is the runtime size of flatpak and the overhead it causes does not change even you have solution that avoids a sandbox.


    Originally posted by Weasel View Post
    Since Lfoobar is different name than libfoobar (from system) it will never conflict and the linker magic makes it so your app loads from Lfoobar, while also indirectly loading from libfoobar (via system's libc).
    The saying never conflict is wrong. What if libfoobar and Lfoobar attempt to create the same dbus entry or socket or any other resource..

    The benchmarks on sandboxing overhead is not that high. Fixing the was done in linking LSB by the loader. This did not stop loading two different versions of a library loaded in an application managing to conflict with each other. The bad part is you test one distribution and everything seams fine and your users run a different one and everything goes to hell.

    Weasel you are being way to optimistic on how bad the problem is. Reality is to be safe you application should only ever have loaded 1 version of a particular library. If you want to load two you need to be absolutely sure of the library that you don't have a resource conflicts or function name conflicts. Resources conflicts are not fixed/prevented by linker trickery. Please note it gets worse. Lets say your application needs a newer dbus service than what host applications can handle so either your application fails or all the host ones do if you don't have a sandbox option on dbus.

    The basic nightmare that is simple to forget is that the distribution can have old version in some cases of the program you are needing to run to support your application and newer and old resource conflict. Sandbox is not just preventing linking issues it also about allowing resource allocation separation so preventing resource conflict issues as well..

    If it was as simple as linker fix it would have been done by now.

    Comment


    • Originally posted by oiaohm View Post
      The overhead from Linux kernel namespaces on a systemd system is quite Negligible.



      https://github.com/AppImage/AppImage...latform-Issues << read this carefully.

      Reality is you cannot trust the distribution libc even because glibc deprecates and adds new functions. Depend on how glibc built will effect if you application runs or not. Distributions can do stupid things it library naming and library versioning the reality since they can they do. Basically you end up needing a full runtime independent to distribution to have the least random issues.

      The reality is the runtime size of flatpak and the overhead it causes does not change even you have solution that avoids a sandbox.



      The saying never conflict is wrong. What if libfoobar and Lfoobar attempt to create the same dbus entry or socket or any other resource..

      The benchmarks on sandboxing overhead is not that high. Fixing the was done in linking LSB by the loader. This did not stop loading two different versions of a library loaded in an application managing to conflict with each other. The bad part is you test one distribution and everything seams fine and your users run a different one and everything goes to hell.

      Weasel you are being way to optimistic on how bad the problem is. Reality is to be safe you application should only ever have loaded 1 version of a particular library. If you want to load two you need to be absolutely sure of the library that you don't have a resource conflicts or function name conflicts. Resources conflicts are not fixed/prevented by linker trickery. Please note it gets worse. Lets say your application needs a newer dbus service than what host applications can handle so either your application fails or all the host ones do if you don't have a sandbox option on dbus.

      The basic nightmare that is simple to forget is that the distribution can have old version in some cases of the program you are needing to run to support your application and newer and old resource conflict. Sandbox is not just preventing linking issues it also about allowing resource allocation separation so preventing resource conflict issues as well..

      If it was as simple as linker fix it would have been done by now.
      This is a great post, thanks. You've convinced me that to solve this problem properly (making truly cross-distro, portable applications), you need sandboxing/virtualization/containerization of applications.

      There are clear downsides to virtualization which many have pointed out though including:
      • additional complexity
      • additional storage space used due to duplication of data and general bundling of dependencies
      • performance overheads (there will always be some, even if they are small)
      • allowing the sandboxed app to as seemlessly as possible interact with the non-sandboxed host system
      • apps being held back by the limitations of the available runtimes to maximise compatibility with many distros.
      I think the solution to these problems above is to always complement portable apps (eg flatpak & snap apps) with non-portable, distro specific apps from an old school software repo.

      Personally I think games are the ideal apps to be made portable with tech like flatpak and snaps. However, I'm perfectly happy using the version of my window manager, my GNU userland tools and my bittorrent client that comes supplied with my distro from a repo (eg from deb or rpm packages).

      This is pretty exciting really. Flatpak and snap may, for the first time be bringing GNU/Linux, practical, secure and easy to use cross-distro software packages. This has been a huge barrier for users of our system and it might finally almost be solved. How amazing will it be if whenever you go to a game or program home page, there is a flatpak file there (or some similar tech) which you know will work on your distro? How amazing will it be if the various places that sell games all provide a secure and easy to use GNU/Linux installer (again: flatpak or similar) that just-works on your system.

      When you tie this tech with the upcoming advancements being made with Wayland and also the great Open Source driver work being done by AMD, I think GNU/Linux has a bright future ahead on the desktop.

      Comment


      • Originally posted by oiaohm View Post
        The saying never conflict is wrong. What if libfoobar and Lfoobar attempt to create the same dbus entry or socket or any other resource..

        The benchmarks on sandboxing overhead is not that high. Fixing the was done in linking LSB by the loader. This did not stop loading two different versions of a library loaded in an application managing to conflict with each other. The bad part is you test one distribution and everything seams fine and your users run a different one and everything goes to hell.

        Weasel you are being way to optimistic on how bad the problem is. Reality is to be safe you application should only ever have loaded 1 version of a particular library. If you want to load two you need to be absolutely sure of the library that you don't have a resource conflicts or function name conflicts. Resources conflicts are not fixed/prevented by linker trickery. Please note it gets worse. Lets say your application needs a newer dbus service than what host applications can handle so either your application fails or all the host ones do if you don't have a sandbox option on dbus.

        The basic nightmare that is simple to forget is that the distribution can have old version in some cases of the program you are needing to run to support your application and newer and old resource conflict. Sandbox is not just preventing linking issues it also about allowing resource allocation separation so preventing resource conflict issues as well..

        If it was as simple as linker fix it would have been done by now.
        Yeah, I hadn't thought of anything except for just pure library functions/APIs, you're definitely right with any other service... I suppose that upstream devs who make a newer version of a library could make it use a totally different naming for dbus or whatever socket etc than a prior version, but that needs to be somehow enforceable since people do stupid things if not forced, most of the time.

        But yeah, I don't think a solution without a sandbox is very viable for complex apps, only simple ones, bah. I just dislike the idea of an entire sandbox for one application even if the app is trustworthy, it just seems too overkill, but I give up, can't find a simple solution to the above.

        Thanks for the info.

        Comment


        • Originally posted by cybertraveler View Post
          There are clear downsides to virtualization which many have pointed out though including:
          • additional complexity
          • allowing the sandboxed app to as seemlessly as possible interact with the non-sandboxed host system

          This is not as straight forwards as you would think on these two. There have been existing downsides to the current system. Most of your requirements to fix seamlessly with non-sandboxed applications is also required for another reason.

          https://flatpak.github.io/xdg-deskto...rtal-docs.html
          If you read over flatpak portals you will notice this provides like unified open/close dialogues, print dialogues and so on. The issue with resource conflict is why putting a qt open dialog in a gtk application directly sometimes does not work or doing the reverse had the same problems so lack of common dialogues. Hard part is there has been a lack of complexity so the Linux Desktop has never been been as good as it could have been. This is like the theme problem as well.

          So really its not to allow sandboxed app to seamlessly interact with non-andboxed applications its mostly to make all application seamlessly interact with each other and give users a common experience. To give that common experience a percentage of the additional complexity is required. The portal work in flatpak is helping sort out a problem that has effected non-sandboxed applications since the start of X11 so even if flatpak dies we don't really want the portal work to die. It also explains why snap, gtk, qt... is also picking up portal work because the problem is universal.

          Originally posted by cybertraveler View Post
          I think the solution to these problems above is to always complement portable apps (eg flatpak & snap apps) with non-portable, distro specific apps from an old school software repo.
          I do agree there will keep on being a place for non portable applications. But for those providing games and the like non portable is a nightmare so a portable form has to exist. Also we do need to face up to the problems that effect non portable and deal with them. Like why in heck does non portable lack common theme storage for even the basics like window colours and font selection. Most of the problems in non portable are not hidden by software hacking once you do sandbox so make us face up to those problems.

          I will admit there are downsides to sandbox applications and there have been a lot of other issues that we have been sweeping under rug and not deal with. This include how to make drivers from third parties that safely migrate to newer Linux Kernels as well. Its about time we stop hiding the problems and fix the problems.

          Comment


          • Originally posted by Weasel View Post
            Yeah, I hadn't thought of anything except for just pure library functions/APIs, you're definitely right with any other service... I suppose that upstream devs who make a newer version of a library could make it use a totally different naming for dbus or whatever socket etc than a prior version, but that needs to be somehow enforceable since people do stupid things if not forced, most of the time.

            But yeah, I don't think a solution without a sandbox is very viable for complex apps, only simple ones, bah. I just dislike the idea of an entire sandbox for one application even if the app is trustworthy, it just seems too overkill, but I give up, can't find a simple solution to the above.

            Thanks for the info.
            You have to remember I have 20+ years of experience trying to beat problem. I started off like you that thinking should be able to-do this without a sandbox only to find out every single time I think I have solved I have walked myself into a trap.

            Also a sandbox is not the worst outcome. Think how you test different versions of internet explorer against a website,
            https://developer.microsoft.com/en-u...dge/tools/vms/
            That right a virtual machine per version of internet explorer that is not the host copy. You are needing to run multi versions of MS office and not have them conflict hello multi virtual machines. So things can be way worse than a sandbox with even worst desktop integration. Of course you see this also in enterprise using hypervisors.

            Only way that non sandbox is really going fly is if the ABI difference between distributions can be fixed and ABI and Interface(dbus socket...) breakage can be stopped. Problem is I don't see that happening any time soon that being inside the next 10 years(I might be a little bit to pessimistic here but I don't think so). Sandbox path is method that is open to us in the short term.

            Comment


            • Originally posted by oiaohm View Post
              But for those providing games and the like non portable is a nightmare so a portable form has to exist.
              Thanks for the full reply. I fully agree with the above. It's a pretty exciting time. I hope Flatpak, snap or something similar takes off in a big way within a couple of years. It would be amazing for GNU/Linux to be as easy as OS X for installing software.

              Comment


              • Very pleasant to read some experiences. I am rather 'hostile' to snapd, not necessarily from experience, but just concept.

                First and foremost, the nomenclature, "snap" in the bigger world means "snapchat screenshot", so they are basically repurposing existing 'trademark' or jargon, and "snappy" is a nonsensical world to begin with.

                So the branding is totally off from the get go, it sounds like a toy system, and that's probably what it is.

                Next up, the only snap I tried was RocketChat. Didn't launch.

                My first snap experience, the app didn't even launch.

                The way to solve Linux library problems is to foremost make sure all libraries are relocatable, and while I do not know the incessant details, I have had enough compiled binaries that couldn't find their libraries to know the headaches.

                I once compiled Calligra and found I couldn't move the compiled package to a different directory, it would stop working because its own library paths were absolute and hardcoded.

                I say: first make sure regular apps are portable within the same system, THEN you can think about packaging them for other systems!

                Comment


                • Originally posted by jm111 View Post
                  The way to solve Linux library problems is to foremost make sure all libraries are relocatable, and while I do not know the incessant details, I have had enough compiled binaries that couldn't find their libraries to know the headaches.

                  I once compiled Calligra and found I couldn't move the compiled package to a different directory, it would stop working because its own library paths were absolute and hardcoded.

                  I say: first make sure regular apps are portable within the same system, THEN you can think about packaging them for other systems!
                  Did you try to use LD_LIBRARY_PATH? For me it usually works.

                  Comment


                  • Originally posted by jm111 View Post
                    The way to solve Linux library problems is to foremost make sure all libraries are relocatable, and while I do not know the incessant details, I have had enough compiled binaries that couldn't find their libraries to know the headaches.

                    I once compiled Calligra and found I couldn't move the compiled package to a different directory, it would stop working because its own library paths were absolute and hardcoded.

                    I say: first make sure regular apps are portable within the same system, THEN you can think about packaging them for other systems!
                    Sorry no this is not the only problem.
                    1) limited devices/resources. Libraries attempt to start more than 1 opengl primary context on a single window causes failures or libraries attempt to use same dbus, serial... socket other libraries loaded by application, exclusive grab keyboard/mouse conflict between libraries. When you have huge toolkits when they ABI break they change where lot of this stuff is done.
                    2) items like absolute paths and file locking issues.
                    3) linking function name conflicts.
                    4) needing particular versions of programs to run to perform some task and if wrong version strange failures failed because command line option X is missing.

                    Sandboxing the application allows you to work around the lot. Having a full runtime with the application deals with 3 out of the 4 so application never gets library it was not designed for or applications it was never designed for. Yes absolute paths problem is one of the items that you have to address if you want to do this without sandboxing.

                    Originally posted by Weasel View Post
                    Did you try to use LD_LIBRARY_PATH? For me it usually works.
                    This does not work with all applications.

                    Like Krita you have to set KRITA_PLUGIN_PATH as well because it has a hard coded path where it attempts to load plugins from they were good and include a environmental variable over ride. Yes these per application path over rides do get annoying. This is nicer than something like wine where you have to set 8 things in environmental variable as well as altering LD_LIBRARY_PATH or have wine not run or die in strange ways because you relocated it. Please not wine was designed to be relocatable but is not friendly.

                    Some ways it would be handy if there was some for values that need to be changed for relocated could stored that applications would auto load so not requiring environmental var overrides.

                    Comment


                    • Originally posted by oiaohm View Post

                      Sorry no this is not the only problem.
                      I have never said it was the only problem, so there is no need to be sorry (or to say no).

                      I am saying that if you do not solve this problem first, your other solutions are going to try to bridge too big a gap, ie. cannot use your first solution to solve the other ones.

                      Comment

                      Working...
                      X