Announcement

Collapse
No announcement yet.

Would Phoronix do Gentoo Linux versus Ubuntu Linux benchmarks?

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Would Phoronix do Gentoo Linux versus Ubuntu Linux benchmarks?

    Linux Magazine recently did benchmarks of Gentoo Linux and Ubuntu Linux alongside one another, but practically everyone on the Gentoo forums agrees that the comparison is flawed:



    In the past, Phoronix has done benchmarks of alpha releases of distributions (e.g. Ubuntu 10.04), but has had a long standing omission of benchmarks involving Gentoo Linux, despite Gentoo Linux being a mainstream distribution that has a great deal to offer to the Linux community in ways that other distributions cannot.

    Would Phoronix be so kind as to do its standard benchmarks in as well as benchmarks similar to those done by Linux Magazine with Gentoo users' concerns addressed in a Gentoo Linux versus Ubuntu Linux comparison? Phoronix does excellent Linux benchmarks and it would be wonderful to see a meaningful comparison between the two distributions. It also would be wonderful to see Phoronix rectify its long standing omission of Gentoo Linux in its benchmarks.

    Before anyone mentions Sabayon Linux, benchmarks of Sabayon Linux do not count as benchmarks of Gentoo Linux, in particular because installing Sabayon Linux ignores all of the customization that is done on a typical Gentoo Linux System. While running Gentoo Linux does not automatically translate into higher performance, that customization often yields higher performance, which is something that is Sabayon Linux does not have.

    In case Phoronix is willing to do benchmarks of Gentoo Linux, here are some instructions:

    Code:
    The typical Gentoo user runs Gentoo Linux with packages from Gentoo's testing tree (software that is stable upstream, but has not had the formality of being declared stable by Gentoo's package maintainers), so a proper comparison between Gentoo Linux and Ubuntu Linux would involve using packages from Gentoo's testing tree (for x86_64, this would involve running 'echo "ACCEPT_KEYWORDS=\"~amd64\"" >> /etc/make.conf') and then masking any major software newer than the software that Ubuntu Linux uses. That would involving doing 'echo ">=sys-kernel/gentoo-sources-2.6.32" >> /etc/portage/package.mask', among similar commands for X, GCC, etcetera when comparing Gentoo Linux to Ubuntu Linux 9.10, such that all of the major software versions are identical. A list can be found at wikipedia:
    
    [url]http://en.wikipedia.org/wiki/List_of_Ubuntu_releases#Version_history_of_common_programs[/url]
    
    The sys-kernel/gentoo-sources, sys-devel/gcc, x11-base/xorg-server and x11-drivers/nvidia-drivers are likely the only software on the system that would require this treatment.
    
    The kernel should be compiled for the system's architecture, based off a .config from [url]www.kernel-seeds.org[/url]. This should accurately simulate the level of kernel customization that is done by the typical Gentoo user, as the owner of that site has taken most of the things Gentoo users do, did them by default for us and published instructions regarding the things that tend to vary from system to system.
    
    Lastly, the recommendations for Gentoo involve having all software on the system compiled with "CFLAGS=\"-O2 -march=native -pipe -fomit-frame-pointer\"" and "CXXFLAGS=\"${CFLAGS}\"" in /etc/make.conf, although you could vary this if you want to try testing different optimization levels in your benchmarks, like Linux Magazine did.

  • #2
    Such a comparison would be of little value. The major reason for using Gentoo is not performance but rather its flexibility.

    Besides, a benchmark run would be very hard to make fair and relevant at the same time:
    If you adjust the package versions in Gentoo to what you find in Ubuntu, this would be unfair to Gentoo, as you are not using what the developers recommend you to use. The other way round is not feasible, even. So you would penalize Gentoo for its flexibility.
    If instead you simply go with the latest stable or latest unstable releases, you will only compare performance of different X servers / kernels etc., and as these versions are constantly changing, chances are that the results are already outdated by the time they are published.

    Add to this that installing Gentoo requires reading and careful following of documentation, and you can probably see why no such benchmark comparison has been posted yet.

    Comment


    • #3
      Originally posted by Shining Arcanine View Post
      Linux Magazine recently did benchmarks of Gentoo Linux and Ubuntu Linux alongside one another, but practically everyone on the Gentoo forums agrees that the comparison is flawed
      Of course they did, all four of them, at the time of this writing!

      One of them referred to optimizations, but did not specify which (s)he preferred. Which do you prefer?

      Which optimizations should be done for Ubuntu, as you, after all, can compile your own package for Ubuntu (and Debian) too, using about the same optimizations as for Gentoo.

      All comparisons are flawed, and most comparisons are more flawed than 50% of them!

      Comment


      • #4
        Originally posted by sabriah View Post
        of course they did, all four of them
        lol!

        5678910

        Comment


        • #5
          What would it even mean to "benchmark Gentoo"? Any given configuration will almost certainly differ greatly from the majority of Gentoo users, because there's no such thing as a "default install" of Gentoo. The only standard binaries are the ones in stage3, and a bunch of those are routinely replaced during the installation process (which ones? that depends on when you install and what's in your make.conf). The customization, including building a brand new kernel image, begins well before the install is over.

          Comment


          • #6
            Generally I think there is no point for such comparison.

            Gentoo and Ubuntu address completely different audience.

            Gentoo gives you maximum performance and flexibility at the cost of more complex system and much bigger maintenance. Gentoo is generally addressing people who know linux very well and want everythng to be done the way they want it.

            Ubuntu on the other hand is addressing mainstream linux user. Almost everything is prepared to work of the box. Minimal user knowledge is required to run and maintain it.

            Every single system I've got at home is running Gentoo. But I always recommand ubuntu to my friends who want to try linux.
            Rob
            email: [email protected]

            Comment


            • #7
              Originally posted by sabriah View Post
              Of course they did, all four of them, at the time of this writing!

              One of them referred to optimizations, but did not specify which (s)he preferred. Which do you prefer?

              Which optimizations should be done for Ubuntu, as you, after all, can compile your own package for Ubuntu (and Debian) too, using about the same optimizations as for Gentoo.

              All comparisons are flawed, and most comparisons are more flawed than 50% of them!
              The Gentoo forums are much more popular than these forums and at the time of this post, that thread has 316 page views versus 104 for this thread. The reason so far people posted is because they have nothing to add, not because they disagree. From my experience, if someone at the Gentoo forums disagrees about something, they say something.

              As for Ubuntu, Ubuntu is not designed to have the same level of customization that Gentoo has, and its community actively discourages it. Practically every community discourages going outside the scope of their distribution's package manager, Gentoo included, but Ubuntu's package manager and package maintainers attempt to handle the modification of configuration files and selection of preprocessor directives for users, which is something that Gentoo's package manager does not do. Ubuntu should not have any optimizations done for it, unless you consider installing its SMP i686 kernel to be an optimization, because anything more than that is illegal as far as it as a distribution is concerned, and would qualify as a fork of the Ubuntu distribution rather than any actual instance of the Ubuntu distribution.

              Comment


              • #8
                By the way, I would like to add that I listed how a standard Gentoo machine should be configured, which included the various optimizations that would be done, in my first post.

                Comment


                • #9
                  Originally posted by Ex-Cyber View Post
                  What would it even mean to "benchmark Gentoo"? Any given configuration will almost certainly differ greatly from the majority of Gentoo users, because there's no such thing as a "default install" of Gentoo. The only standard binaries are the ones in stage3, and a bunch of those are routinely replaced during the installation process (which ones? that depends on when you install and what's in your make.conf). The customization, including building a brand new kernel image, begins well before the install is over.
                  Gentoo could be configured with Ubuntu's software versions as a Gentoo user might do if he wanted binary compatibility with Ubuntu packages. That would ensure a fair comparison. To date, all other comparisons have been flawed because of performance regressions (or improvements) from newer/older software versions, so it is impossible to compare the impact of the differing philosophy of each distribution on system performance.

                  Originally posted by dagger View Post
                  Generally I think there is no point for such comparison.

                  Gentoo and Ubuntu address completely different audience.

                  Gentoo gives you maximum performance and flexibility at the cost of more complex system and much bigger maintenance. Gentoo is generally addressing people who know linux very well and want everythng to be done the way they want it.

                  Ubuntu on the other hand is addressing mainstream linux user. Almost everything is prepared to work of the box. Minimal user knowledge is required to run and maintain it.

                  Every single system I've got at home is running Gentoo. But I always recommand ubuntu to my friends who want to try linux.
                  Gentoo Linux is not meant for maximum performance (although that usually comes with maximum customizability) and is a mainstream Linux distribution. Once it is running, very little needs to be done to maintain and use it. You need to syncronize its package manager with the mirrors, tell it when you want to install updates, and maintain its configuration files, a process which is semi-automated thanks to a neat tool Gentoo provides its users called etc-update.

                  Issues like the Ubuntu 8.10 upgrade that killed X for a great many Ubuntu users do not happen for Gentoo users unless they start mixing software branches. Gentoo Linux is a mainstream distribution and that shows when you do not mix software branches. When you mix software branches, an upgrade for software X you installed from the testing tree has a dependency on software package Y you installed from the stable tree, but the version in the stable tree is too old for it to work, however, even then, you get some very informative error messages from the package manager that alert you to the problem and how to resolve it.

                  I have found it is usually best to stick with either the stable tree (software that is stable according to both Gentoo's package managers and upstream) or the testing tree (software that is stable according to upstream). As long as you stick to one or the other and you listen to the messages that the package manager gives you after installing upgrades (e.g. kernel option Z is not set, please set it or things will break), things do not break. Since Gentoo lets you select which tree or software versions (the latest stable being the default) from which you want to install software, solving or preventing things kinds of things is really easy to do. In my case, I just put ACCEPT_KEYWORDS="~x86" in /etc/make.conf, so I always have the latest stable software available to Gentoo users from upstream, even if it is in Gentoo's testing tree. Since I did this, my experience with Gentoo has been to just let it take care of everything for me, including which versions to install.

                  So far it seems that the only opportunity I have for things to break is if I intentionally do something stupid, like corrupting the contents of my configuration files.

                  Comment


                  • #10
                    Originally posted by Shining Arcanine View Post
                    By the way, I would like to add that I listed how a standard Gentoo machine should be configured, which included the various optimizations that would be done, in my first post.
                    My apologies. I was in a grumpy mood.

                    Comment

                    Working...
                    X