Announcement

Collapse
No announcement yet.

FreeBSD 8.0 vs. Ubuntu 9.10 Benchmarks

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • #51
    Originally posted by mtippett View Post
    Accessible was used carefully . For FS tuning, I consider my self as newbie (with a deeper technical understanding).

    The point is that
    "defaults + accessible tuning guide" = argument for "fairer" comparison
    where as
    "defaults + tuning as a black art" ~~ "defaults"
    for the majority of users.

    I don't completely buy into "fairer" because "fairness" is a function of level of expertise, effort in configuration, particularly intent for the results . That is why Phoronix for the most part relies on the upstream's "Best Effort" of what they believe is production worthy.

    (To be fair, I updated my reply slightly between our exchanges .
    I believe I've got your point Your examples can be quite relative IMHO, because black art can be something normal for some people However, it will be the best to see final releases comparison, because what we see now isn't something what end user will use yet To make a completely fair test, which many people expect, ignoring what devs consider being the best for end users someone should set both systems to something like this:

    tune them for maximum performance while both remain identically stable same time, data is safe and there are identical apps versions used. However, doing such comparison is probably impossible.

    "defaults + accessible tuning guide" = argument for "fairer" comparison are IMHO:

    Ubuntu: using ext4 in writeback mode, turning off debugging (if enabled),

    FeeBSD: turning off debugging, using same GCC version as Ubuntu.


    @Risner

    Thank you.
    Last edited by kraftman; 29 September 2009, 04:05 AM.

    Comment


    • #52
      FreeBSD 7.2 kernel with and without debugging options

      OK, just to show how big the difference in performance can be between a kernel with and one without the debugging options, I ran a buildworld on FreeBSD 7.2 (not 8.0, because it's the only one I have).

      I am not sure what are the options used in 8.0 RC, but I used:

      options KDB
      options DDB
      options INVARIANTS
      options INVARIANT_SUPPORT
      options WITNESS
      options DEBUG_LOCKS
      options DEBUG_VFS_LOCKS
      options DIAGNOSTIC

      The machine is a Dual Core 2 at 2GHz with 1GB RAM and a 5400RPM SATA HDD.
      The command used: make -j4 buildworld, after a make clean, of course. -j4 means 4 parallel jobs.

      The results are:
      - without debugging options: 32 minutes and 10 seconds
      - with debugging options: 41 minutes and 55 seconds.
      That's about 30% more time for the one with debugging options on.

      Comment


      • #53
        Originally posted by clau View Post
        The results are:
        - without debugging options: 32 minutes and 10 seconds
        - with debugging options: 41 minutes and 55 seconds.
        That's about 30% more time for the one with debugging options on.
        Nice improvement in this. There also some debugging options enabled in Ubuntu, but I wonder if they have meaningful impact on performance.

        Comment


        • #54
          Originally posted by kraftman View Post
          Nice improvement in this. There also some debugging options enabled in Ubuntu, but I wonder if they have meaningful impact on performance.
          We'll see soon, when both are stable

          Comment


          • #55
            Originally posted by clau View Post

            ...

            The results are:
            - without debugging options: 32 minutes and 10 seconds
            - with debugging options: 41 minutes and 55 seconds.
            That's about 30% more time for the one with debugging options on.
            Yes, I agree that turning on all those options will have a meaningful impact on performance. Effectively you have given an upper and lower bound of what the performance could be like.

            The debugability of the system is a sliding scale with performance at the other end, the assertion is that there is debug options placed on the kernel means that FreeBSD-8.0 will site somewhere on that debugability scale. Where it actually sits will take someone with sufficient FreeBSD-Fu to show how to help move from supposition on the options to fact and place us at a quantified point on the scale.

            Excuse my ASCII art..

            [Debugability]-------=|=-----------------[Performance]

            There also exists a scale for file-system integrity, integrity at one end, performance at the other. Most distributions choose to push the default options up towards integrity, but as the openSUSE "filesystem barrier" benchmarking demonstrated, you can easily push it to far and lose a lot of performance.

            [Data Integrity]-------=|=-----------------[Performance]

            In almost all of these scales (security, memory efficiency, etc) the intent is to always find a balance.

            So I agree that there may be some debug options turned on within the development versions, but I strongly feel that the FreeBSD maintainers would still ensure the slider is at a balanced mid-way point, rather than the debugability at all costs - which appears to be your benchmark numbers.

            Anyone with the sufficient FreeBSD-8.0-fu to help?

            Matt

            Comment


            • #56
              Originally posted by Apopas View Post
              We'll see soon, when both are stable
              So, there were a bunch of messages (including from myself) about this, without actually checking it. I just installed FreeBSD 8.0RC1 in Qemu, and AFAICT the debugging options aren't enabled.
              Therefore, I don't expect big changes in the RELEASE, unless the benchmarking will be done by using a newer GCC version (8.0 has version 4.2.1, although a newer version can be used from ports).

              Comment


              • #57
                Originally posted by clau View Post
                So, there were a bunch of messages (including from myself) about this, without actually checking it. I just installed FreeBSD 8.0RC1 in Qemu, and AFAICT the debugging options aren't enabled.
                Therefore, I don't expect big changes in the RELEASE, unless the benchmarking will be done by using a newer GCC version (8.0 has version 4.2.1, although a newer version can be used from ports).
                Thanks, getting close... FreeBSD-Fu is strong in this one. Anyone care to confirm and expand on clau's research?

                Again, just like Solaris, availability of Sun Studio which is faster doesn't change the fact that Solaris by default will build 32 bit binaries with a slower version of gcc. For most users they won't know what is happening except for Solaris is feeling slow.

                I also stumbled across

                I just saw a new article on Phoronix testing the performance of FreeBSD 8.0 vs Ubunto 9.10. I think a lot of the tests are influenced by the difference of gcc compilator versions that are used - 4.2.1 for FreeBSD (pretty old already) and 4.4.1 for Ubuntu. But I don't think that memory and...


                even the FreeBSD community doesn't *really* know what is turned on.

                Which goes back to one of my other comments that I frequently make. If there isn't an accessible (well written/easy to follow) guide about how to change things for the better, the black art of tuning is held with specialist or the maintainers.

                It is the maintainers who are making the tuning decisions, consequently the benchmark approach of "out of the box" is still valid and still valuable.

                Regards,

                Matthew

                Comment


                • #58
                  Originally posted by mtippett View Post
                  So I agree that there may be some debug options turned on within the development versions, but I strongly feel that the FreeBSD maintainers would still ensure the slider is at a balanced mid-way point, rather than the debugability at all costs - which appears to be your benchmark numbers.
                  Prerelease versions are not for production use, therefore the performance is less important, while finding stability issues is much more important.
                  Besides, a FreeBSD developer can always compile a kernel with whatever options he needs, whenever he needs it.

                  Comment


                  • #59
                    Oh, but there is alot of FreeBSD documentation. One just needs to read. For instance:

                    The Handbook:
                    A constantly evolving, comprehensive resource for FreeBSD users


                    Configuration and Tuning (from Handbook):
                    This chapter explains much of the FreeBSD configuration files, how to enable or disable a service, how to configure the logging system and the power management area.


                    Kernel Debugging (from Handbook):


                    Developer's handbook:
                    For people who want to develop software for FreeBSD (and not just people who are developing FreeBSD itself)


                    The thing is I didn't know when exactly the debugging options are removed from the default kernel, in in RC or in RELEASE versions.

                    Nevertheless, I suppose this subject is closed, since 8.0RC1 seems to have the debugging options disabled.

                    Comment


                    • #60
                      Originally posted by clau View Post
                      Prerelease versions are not for production use, therefore the performance is less important, while finding stability issues is much more important.
                      Besides, a FreeBSD developer can always compile a kernel with whatever options he needs, whenever he needs it.
                      Yes, I agree. The slider moves around throughout the development cycle.

                      BTW, slashdot has FreeBSD-FU..



                      In particular

                      Originally posted by slashdot-OMG
                      Anyone actually familiar with the FreeBSD development and release process would know that a release candidate has a considerable amount of debugging options turned on.

                      On Sep-10, most debugging was disabled: http://lists.freebsd.org/pipermail/s...er/013399.html [freebsd.org]

                      On Sep-17, there was the last commit before 8.0-RC1: http://lists.freebsd.org/pipermail/s...er/013645.html [freebsd.org]

                      Anyone familiar with the FreeBSD development and release process would know that there are no fixed rules rules when certain stuff happens and there are no sweeping changes like turning off debugging between a late RC and the actual release. (Other debugging stuff like kernel and module symbols are kept for the release.)
                      So let's reset the comments at the start of the thread, and have a discussion based on fact . New thread, or let's talk impact and cause .

                      Regards,

                      Matthew

                      Comment

                      Working...
                      X