Announcement

Collapse
No announcement yet.

FreeBSD 8.0 Benchmarked Against Linux, OpenSolaris

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • FreeBSD 8.0 Benchmarked Against Linux, OpenSolaris

    Phoronix: FreeBSD 8.0 Benchmarked Against Linux, OpenSolaris

    With the stable release of FreeBSD 8.0 arriving last week we finally were able to put it up on the test bench and give it a thorough look over with the Phoronix Test Suite. We compared the FreeBSD 8.0 performance between it and the earlier FreeBSD 7.2 release along with Fedora 12 and Ubuntu 9.10 on the Linux side and then the OpenSolaris 2010.02 b127 snapshot on the Sun OS side.

    http://www.phoronix.com/vr.php?view=14407

  • #2
    OpenSolaris 2010.02 really ran in front when it came to the OpenSSL RSA 4096-bit performance where it performed more than twice as fast as Ubuntu/Fedora and then three and a half times faster than FreeBSD. FreeBSD struggled in OpenSSL similar to our Mac OS X 10.6 benchmarks where it too lagged behind in this computational test.
    Are you sure Solaris wasn't ran in 64 bits here?

    Comment


    • #3
      All operating systems were left with their default options during the installation process
      Which for those of us who don't happen to memorize each distributions quirks equals...?


      Some interesting results there, though I wonder about your obsession with using development releases to compare against other distributions / OSs.
      Would be interesting to see FreeBSD 8 on UFS vs FreeBSD 8 on ZFS, and comparison to the OpenSolaris benchmarks you did to see what advantages/disadvantages ZFS is providing on the OS. In theory based on how ZFS works it should be a little slower (ZFS's serious advantages are considered worth the trade off.)

      Comment


      • #4
        FreeBSD 8.0 with ZFS

        The comparsion here shows a benchmark of default installs and no optimizations whatsorever - it is not a comparsion of the kernels.

        There are two things that may slow down FreeBSD 8 in this particular comparsion to Ubuntu:

        a) compiler used - FreeBSD uses GCC 4.2.x, Ubuntu GCC 4.4.x
        b) file system - UFS vs EXT4 (vs ZFS)

        First, you can use the GCC44 compiler in FreeBSD:
        Using newer version of GCC and binutils with the FreeBSD Ports Collection

        To may tests, it may provide 20% and higher performance increase for some applications (e.g. complex calculations). On average, it is about 5-10%.

        Second, you can try a full-ZFS based FreeBSD 8.0, just download and install from the mfsbsd SE ISO:
        mfsBSD homepage

        ZFS metadata operations are very fast but that is not one of the main advantages of ZFS. These are:
        • high data integrity (checksumming)
        • integrated volume management (stripe, mirror, raid-z)
        • snapshots and clones of snapshots
        • transfer of snapshots and volumes to remote systems
        • filesystem-level data compression (lzma, gzip)
        • delegation of filesystem rights to users.

        Thanks,
        mm@FreeBSD.org

        Comment


        • #5
          From the OpenSolaris forum:

          The system automatically boots into 64/32 bit mode depending on CPU probe done at boot.
          How did you make to run its kernel in 32 bit mode? Some bootline option?

          Comment


          • #6
            I wish you would stop *describing* the graphs rather than just commenting on interesting ones.

            This is good: "OpenSolaris was not compatible with the MAFFT benchmark that measures the multiple sequence alignment of this computational biology program."

            This is bad: "While Linux was superior with POV-Ray, when it came to the multi-threaded C-Ray, FreeBSD did the best. FreeBSD 7.2 was slightly faster than FreeBSD 8.0, but both were faster than Ubuntu/Fedora and OpenSolaris. OpenSolaris came in third while Fedora and Ubuntu took fourth and fifth, respectively."

            I can see all that much more quickly just by looking at the graph, which is presumably why they're there! The duplicate description is just noise that I have to wade through - the sort of thing you'd write in high school science reports to make them longer.

            Otherwise, keep up the good work.

            Comment


            • #7
              The only real merititious point of OpenSolaris is its superb filesystem, ZFS - at least, when you consider the figures gained from this benchmark. Good work, by the way.

              It's a damn shame that the CDDL remains incompatible with the GPL. The whole point of free software and open-source was to help open communities grow, and develop, and for free and open scientific knowledge, through the medium of software, to get progressively better. Open license incompatibility is totally against the spirit of open-source.

              Read these extracts from Wikipedia about ZFS:

              [ZFS's] name originally stood for "Zettabyte File System". The original name selectors happened to like the name, and a ZFS file system has the ability to store 340 quadrillion zettabytes (256 pebi-zebibytes exactly, or 2128 bytes). Every ZiB is 2 [to the power of] 70 bytes.

              ZFS is a 128-bit file system, so it can address 18 quintillion (1.84 10 [to the power of] 19) times more data than current 64-bit systems. The limitations of ZFS are designed to be so large that they would never be encountered, given the known limits of physics (and the number of atoms in the earth's crust to build such a storage device).
              Last edited by synthil; 11-30-2009, 08:37 AM.

              Comment


              • #8
                @mm_freebsd

                There are things in Ubuntu and esspecially in Fedora which can make them much slower then running with some options disabled like SELinux or some kernel debbuging options.
                Last edited by kraftman; 11-30-2009, 10:18 AM.

                Comment


                • #9
                  Originally posted by synthil View Post
                  The only real merititious point of OpenSolaris is its superb filesystem, ZFS - at least, when you consider the figures gained from this benchmark. Good work, by the way.
                  Just wait till Oracle will buy Sun

                  Comment


                  • #10
                    lovely review

                    @mm_freebsd
                    Originally posted by http://www.freebsd.org/doc/en/articles/custom-gcc/article.html
                    Using GCC version 4.4 with SSSE3 instruction set enabled (if supported by the CPU) may yield up to 10% average increase in binary performance. In certain tests, the results show more than a 20% performance boost (e.g. in multimedia processing).
                    But what u said in regards to the compiler was:

                    To may tests, it may provide 20% and higher performance increase for some applications (e.g. complex calculations). On average, it is about 5-10%.
                    You also mentioned that

                    The comparsion here shows a benchmark of default installs and no optimizations whatsorever
                    Conveniently leaving out that these gains you speak of are by using SSSE3.

                    Comment


                    • #11
                      Originally posted by synthil View Post
                      Open license incompatibility is totally against the spirit of open-source.
                      Of course with this logic GPL3 is against the "spirit of open-source" because it is not backwards compatible.

                      @ kraftman - What makes you think Oracle will change anything?

                      Comment


                      • #12
                        Oracle wants to earn money, which it does by restricting tech to it's products. I hope Oracle keeps everything from SUN open.

                        Regarding OpenSolaris wins some benchmarks here. That is no big deal, I couldnt care less. OpenSolaris is actually in alpha stage, not beta. Every two weeks the source from the development tree is checked out and released as a OpenSolaris build. Later, when all functionality finally has been added, SUN will release OpenSolaris in beta stage and intensive testing will take place. Then Solaris 11 will be released, rock stable, rock solid and fast. The fun thing is that OpenSolaris is actually quite stable as it is today, right from the development source tree. Wait until beta testing begins, then you will see some REAL stability.

                        Right now, OpenSolaris is not optimized at all. It is under heavy construction. Later, SUN will optimize OpenSolaris for speed. Then I would like to see OpenSolaris benched on machines with 64 CPUs or more. That is were Solaris strengths are; big scale computers. Not laptops. Solaris comes from server OS, tailored to big iron. Only now it has been directed to desktop and laptops.

                        For you guys stating that ZFS is the only good thing Solaris has, well you have not heard about DTrace, then. Many developers say that DTrace is THE killer feature of today. Several Firefox developers have switched to OpenSolaris, just because of DTrace. This DTrace will be heavily used for OpenSolaris optimizations, later.

                        Comment


                        • #13
                          Originally posted by the article
                          Summarizing some of the key packages found in each operating system, ... an eye-crossing amount of versions number
                          Here's what would be awesome: a version matrix:

                          Code:
                                                 kernel | filesystem | gcc | x.org | glibc (if applicable) | ...
                          FreeBSD 8.0
                          FreeBSD 7.2
                          OpenSolaris
                          ...
                          It's pretty common to get to a benchmark result that surprises me, and I want to go check what versions of gcc were involved. Since I haven't memorized the paragraph of version numbers, I have to basically read it all again. If it was in a table, I could scan down just the column of the table I wanted to check.

                          And I second the suggestion to not say anything about a graph if you don't have anything useful to add. "oh look Fedora won again" is useless. We have to read all the text you write to avoid missing any interesting observations or explanations of why one score is higher, and it just slows us down to read stuff that isn't useful. It especially slows me down when one of those casual comments seems to be wrong (i.e. my intuition as to the root cause of the result differs from yours, for example :P ); it makes me stop and consider replying in the article... But then I don't bother, because it's usually something relatively minor.

                          High-school science report padding is probably the best description I've seen of the style. Other benchmark sites do this too, writing like it's a play-by-play commentary on a race, with one video card making up for earlier defeats by doing well on this benchmark, and garbage like that.

                          Comment


                          • #14
                            Why not win32/win64?

                            I know this benchmark and analysis probably takes long enough but I do think it would have been interesting to see win32/win64 in the mix somewhere. Is this a problem with porting the suite to use mingw?? ? Or is it just that performance is known to be so bad in Windows that it just doesn't make sense to from a benchmark POV?

                            Comment


                            • #15
                              Originally posted by avuton View Post
                              I know this benchmark and analysis probably takes long enough but I do think it would have been interesting to see win32/win64 in the mix somewhere. Is this a problem with porting the suite to use mingw?? ? Or is it just that performance is known to be so bad in Windows that it just doesn't make sense to from a benchmark POV?
                              Yes. Performance is so terribly bad in Windows that you would have to leave the benchmark running for weeks before completion. Not worth it, trust me.

                              Comment

                              Working...
                              X