Announcement

Collapse
No announcement yet.

Gentoo Linux vs. Linaro ARM Benchmarks

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • #16
    Originally posted by GreatEmerald View Post
    Also, I installed 4.7 on my device manually, but even then it installs into a new slot, and by default the old compiler is used.
    One can easily run gcc-config to switch between multiple installed gcc versions in gentoo. But you probably already know this Another interesting trick is to tune gcc configuration, which can be done in the following way:
    Code:
    mkdir /etc/portage/env
    mkdir /etc/portage/env/sys-devel
    echo 'EXTRA_ECONF="--with-cpu=cortex-a9 --with-fpu=neon"' > /etc/portage/env/sys-devel/gcc
    emerge sys-devel/gcc
    The optimization flags are normally specified in CFLAGS variable from /etc/make.conf, but changing gcc configuration is more reliable because it has global effect (even for the packages not built by portage). In particular, this is a good way to deliver CPU specific optimization settings to phoronix-test-suite. Also you can add "--with-mode=thumb" option for a fair comparison with ubuntu/linaro.

    Comment


    • #17
      Originally posted by RealNC View Post
      The article claims that Gentoo Stable is using GCC 4.7. This is not the case. Stable ("arch") is using GCC 4.5.4, while Testing ("~arch") is using 4.6.3. GCC 4.7 is actually hard-masked and marked as experimental. You cannot install it, unless you unmask it (in Gentoo, that means telling the system "I'm about to potentially shoot myself in the foot, and yes, that's what I want.")
      The articles claims that Gentoo stable was compiled with GCC 4.7, which you can do, although as you say, it is unsupported.

      Originally posted by RealNC View Post
      So I'd say this benchmark is bogus. If you installed GCC 4.7 on Gentoo, you should have done the same for Linaro. Since you didn't do that, the benchmark is highly biased.
      How do you propose that be done for Linaro? As far as I know. Linaro distributes binaries while Gentoo distributes source code. Installing a different system compiler on Linaro will not make the binaries that they distribute faster.

      Edit:
      Also, if GCC was upgraded to a non-stable (and even non-testing) package version, then who knows what else was. I suspect the person who made the images for Phoronix didn't even mention this.
      Gentoo developers do this all the time, mainly because we are the people who fix the bugs when things go wrong. If this were a Gentoo developer's system, it would not be surprising for it to use the latest GCC.

      With that said, I would love to know who this Robert Sanders is. We don't have any active or retired developers by the name of Robert Sanders:

      http://www.gentoo.org/proj/en/devrel...l/userinfo.xml
      http://www.gentoo.org/proj/en/devrel...Filter=Retired

      Comment


      • #18
        Originally posted by ryao View Post
        The articles claims that Gentoo stable was compiled with GCC 4.7, which you can do, although as you say, it is unsupported.
        If you have GCC 4.7, it's not Gentoo Stable anymore.

        I don't disagree* with anything else you said, since it's not the point. The point is that Gentoo Stable was not tested in this benchmark.

        * Well, maybe with not installing GCC 4.7 in Linaro. The Phronix Test Suite is built from source, so why wouldn't GCC 4.7 affect the results? I say it would affect them greatly.

        Comment


        • #19
          Originally posted by RealNC View Post
          If you have GCC 4.7, it's not Gentoo Stable anymore.

          I don't disagree* with anything else you said, since it's not the point. The point is that Gentoo Stable was not tested in this benchmark.

          * Well, maybe with not installing GCC 4.7 in Linaro. The Phronix Test Suite is built from source, so why wouldn't GCC 4.7 affect the results? I say it would affect them greatly.
          I assumed that the phoronix test suite was a bunch of scripts that used the system libraries and binaries. Is that incorrect?

          By the way, I really would like to know who this Robert Sanders is. Michael claims that he was given shell access to a "board to be used as the primary build host for Gentoo weekly ARM stages". This makes no sense to me because I was under the impression that the release engineering team cross compiled the stages using catalyst.

          Comment


          • #20
            Originally posted by ryao View Post
            With that said, I would love to know who this Robert Sanders is. We don't have any active or retired developers by the name of Robert Sanders: http://www.gentoo.org/proj/en/devrel...l/userinfo.xml
            The people on #gentoo-embedded irc channel say that he is 'dagger'.

            Comment


            • #21
              Originally posted by ssvb View Post
              The people on #gentoo-embedded irc channel say that he is 'dagger'.
              That solves that mystery. It would seem that our records are inconsistent. The official record says that his name is "Robert Piasek", but he is using "Robert Sanders" on freenode.

              Comment


              • #22
                Originally posted by ryao View Post
                I assumed that the phoronix test suite was a bunch of scripts that used the system libraries and binaries. Is that incorrect?
                AFAICT, the scripts download the sources of each individual benchmark, build them, and then run them. So in that case, GCC 4.7 will probably always win, regardless of whether you're using Gentoo or Linaro.

                Comment


                • #23
                  Originally posted by RealNC View Post
                  If you have GCC 4.7, it's not Gentoo Stable anymore.

                  I don't disagree* with anything else you said, since it's not the point. The point is that Gentoo Stable was not tested in this benchmark.

                  * Well, maybe with not installing GCC 4.7 in Linaro. The Phronix Test Suite is built from source, so why wouldn't GCC 4.7 affect the results? I say it would affect them greatly.
                  Whatever. In any case, "Gentoo Stable" phrase feels unnatural to me and I don't see why anyone would insist on using it. "Gentoo" itself is a highly configurable distro where you can easily mix and match various versions of the packages, use different versions of gcc or even use different non-gcc compilers for some selected packages. Also keep in mind that upstream FSF gcc-4.7.1 is even not a point-zero release, that's a compiler with a bunch of bugs ironed out already. I don't understand your prejudice against it.

                  The plain fact here is that Ubuntu clearly underperformed in this test. It's up to somebody to run the benchmark on a properly configured linaro/ubuntu system (using 'performance' frequency scaling governor) with a better compiler and share the results.

                  BTW, the first initial Ubuntu 11.10 phoronix benchmark on Pandaboard ES showed such abysmal performance that it even got me motivated to register in this forum (because somebody was wrong on the Internet! ). So I had posted my own results from a similar 1.2GHz dual-core Cortex-A9 board for comparison and it used stable Gentoo with gcc 4.5 at that time (versus newer ubuntu gcc 4.6). There were some Ubuntu performance improvements since that time, which were depicted on phoronix as kinda "Ubuntu being on the forefront of ARM development" and not just "sore losers are catching up" Seriously, linaro is doing a lot of gcc related ARM work which eventually gets accepted in upstream gcc and deserves credit for this. But I get a feeling that the end users are treated as guinea pigs and the effect of a lot of minor incremental improvements gets eclipsed by some stupid bugs which ruin the overall performance in Ubuntu distro.

                  Comment


                  • #24
                    Originally posted by RealNC View Post
                    AFAICT, the scripts download the sources of each individual benchmark, build them, and then run them. So in that case, GCC 4.7 will probably always win, regardless of whether you're using Gentoo or Linaro.
                    Does it build the library dependencies as well? If not, Gentoo would have an edge because its system libraries would still be built with the system compiler, while Linaro's would be whatever Linaro ships.

                    Comment


                    • #25
                      Originally posted by ryao View Post
                      Does it build the library dependencies as well? If not, Gentoo would have an edge because its system libraries would still be built with the system compiler, while Linaro's would be whatever Linaro ships.
                      There is little point speculating Gentoo can be just rebuilt using gcc 4.6 (maybe even in two configurations: with and without thumb2) and benchmarks repeated on ODROID-X. With ondemand governor to make it fair.

                      Comment


                      • #26
                        Originally posted by ryao View Post
                        Does it build the library dependencies as well? If not, Gentoo would have an edge because its system libraries would still be built with the system compiler, while Linaro's would be whatever Linaro ships.
                        Unless the benchmark is redone properly, we'll never know.

                        Note that all I'm saying here is that this particular benchmark is null and void. You can't draw any conclusions from it. It was pretty much a waste of time both for Michael, as well as for us.

                        Comment


                        • #27
                          Originally posted by RealNC View Post
                          Note that all I'm saying here is that this particular benchmark is null and void. You can't draw any conclusions from it. It was pretty much a waste of time both for Michael, as well as for us.
                          Sorry for repeating, but the conclusion is that the performance of Ubuntu-based Linaro 12.08 with ondemand governor is far from perfect. The hardware can do much better than that. Any objections?

                          Comment


                          • #28
                            About Gentoo compilation time. When using Gentoo for desktop (kernel, xorg, kde, firefox, libreoffice and quite a pack of other software) , on a standard quad-core (phenom II, core i first generation), the complete tree reemerge takes about 14 hours. But you hardly ever do this, because since glibc and gcc are usually kept same, packages are emerged or updated, as well as their dependencies, but never the whole system. I found Gentoo to be far easier to maintain than Arch.

                            Comment


                            • #29
                              Originally posted by ssvb View Post
                              Sorry for repeating, but the conclusion is that the performance of Ubuntu-based Linaro 12.08 with ondemand governor is far from perfect. The hardware can do much better than that. Any objections?
                              This was a Linaro vs Gentoo bench. And in that respect, it failed. The only thing you can do then, is to ignore the Gentoo results and look at the Linaro numbers. And in that case, you don't have anything to compare them against. So how can you draw any conclusions about the governor? It takes at least two in order to perform any kind of comparison.

                              Comment


                              • #30
                                Originally posted by RealNC View Post
                                So how can you draw any conclusions about the governor?
                                I'm not drawing any conclusions about the governor. The conclusions are about the "Ubuntu-based Linaro 12.08" as a whole.

                                It takes at least two in order to perform any kind of comparison.
                                You have the results from some other linux distro (which happens to be not "Stable Gentoo", but who cares).
                                Last edited by ssvb; 09-09-2012, 05:01 PM.

                                Comment

                                Working...
                                X