Announcement

Collapse
No announcement yet.

FreeBSD 8.0 vs. Ubuntu 9.10 Benchmarks

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Originally posted by kraftman View Post
    Don't forget to use Google's malloc in this case. Hehe I thought about this benchmark before :P



    Bsd guys didn't even post configs, apps versions, they write linux not Linux, not nice :P
    Allow me to correct you:

    "As of Linux 2.6.22 (Fedora 8) they have fixed the most serious scaling problem [...]. Linux 2.6.23 incorporated a new "Completely Fair" Scheduler which performs significantly worse than 2.6.22 on this workload. 2.6.24 has not yet been evaluated."

    They *have* written Linux with capital L

    "MySQL 5.0.51 is currently used for performance comparisons."

    There you have the MySQL version.

    Seriously, it seems a fair test to me, though it's rather old and many things might have changed in both OSes. According to the link you posted 2.6.25 was already much improved.

    Arguably, the omitted sysbench version is less important, since the same version was used.

    Bottom line, I'd like to see such a test on newer releases.

    Comment


    • Originally posted by clau View Post
      Allow me to correct you:

      "As of Linux 2.6.22 (Fedora 8) they have fixed the most serious scaling problem [...]. Linux 2.6.23 incorporated a new "Completely Fair" Scheduler which performs significantly worse than 2.6.22 on this workload. 2.6.24 has not yet been evaluated."
      I was according to this:

      http://people.freebsd.org/~kris/scaling/os-mysql.png

      If 2.6.22 was tested here and there (also in benchmark I posted) I wonder what they base on saying some scaling problems were fixed in 2.6.22?

      Ok, now I know:

      I did some tests to compare Linux vs FreeBSD using mysql and sysbench. This graph shows fedora core 6 vs freebsd bleeding edge in cvs 7.0. The test is sysbench with the following parameters: sysbench --test=oltp --num-threads=${i} --mysql-user=root --max-time=60 --max-requests=0 --oltp-read-only=on…


      There's gnu's malloc used and that's why 2.6.21* drops down.

      And with Google's malloc:

      http://ozlabs.org/~anton/linux/sysbench/

      Seriously, it seems a fair test to me, though it's rather old and many things might have changed in both OSes. According to the link you posted 2.6.25 was already much improved.
      It could be like this. However, there's/was known problem with GNU's malloc library and I wonder what library was used (it seems it was used). There's also 2.6.22 test which I'm according to.
      Last edited by kraftman; 01 October 2009, 01:06 PM.

      Comment


      • Originally posted by clau View Post
        Comparing a benchmark ran in a VM with one ran in the host OS is completely pointless.
        If you really want to do the benchmark yourself, you'd have to install both OSes directly on the hardware, no VM involved.
        Without context, you can't make that decision. There may be any number of scenarios that would dictate that sort of configuration. (For example, 3 server application licenses that require 3 OSes, but will be within the capacity of a single system). In that case you want to have the app in the right spot and/or VM. You then choose benchmarks that represent your application.

        The intent of the benchmarking should be made clear, but without it you can't declare a benchmark invalid.

        With the intent you can call into question two items
        - the analysis and consequent construction of the testing.
        - the analysis and conclusions of the testing.

        Regards,

        Matthew

        Comment


        • Originally posted by kraftman View Post
          I was according to this:

          http://people.freebsd.org/~kris/scaling/os-mysql.png

          If 2.6.22 was tested here and there (also in benchmark I posted) I wonder what they base on saying some scaling problems were fixed in 2.6.22?

          Ok, now I know:

          I did some tests to compare Linux vs FreeBSD using mysql and sysbench. This graph shows fedora core 6 vs freebsd bleeding edge in cvs 7.0. The test is sysbench with the following parameters: sysbench --test=oltp --num-threads=${i} --mysql-user=root --max-time=60 --max-requests=0 --oltp-read-only=on…


          There's gnu's malloc used and that's why 2.6.21* drops down.

          And with Google's malloc:

          http://ozlabs.org/~anton/linux/sysbench/

          It could be like this. However, there's/was known problem with GNU's malloc library and I wonder what library was used (it seems it was used). There's also 2.6.22 test which I'm according to.
          That's why I'd really like to see a new benchmark. Unfortunately I simply don't have the HW necessary for it at my disposal to do it myself.

          Comment


          • Originally posted by mtippett View Post
            Without context, you can't make that decision. There may be any number of scenarios that would dictate that sort of configuration. (For example, 3 server application licenses that require 3 OSes, but will be within the capacity of a single system). In that case you want to have the app in the right spot and/or VM. You then choose benchmarks that represent your application.

            The intent of the benchmarking should be made clear, but without it you can't declare a benchmark invalid.

            With the intent you can call into question two items
            - the analysis and consequent construction of the testing.
            - the analysis and conclusions of the testing.

            Regards,

            Matthew
            I do agree with you, but I thought the context is rather clear, this being a thread for discussions about the results of the Phoronix's benchmarks.
            Besides, if you want to compare the performance of 2 or more OSes, my guess is that you'd want to have the least possible interferences with the running OS.

            Comment


            • No one here willing/able to help me? If not, I'll have to abandon running a new series of tests.

              I've got the hardware and can run the test on the bare metal, I've got the time to do the OS installs and the optimizations I'm able to do (FreeBSD), but I don't have the time to learn how to run the phoronix-test-suite.

              While I am responsible for a number of Linux systems (mostly RHEL and Gentoo), I wouldn't consider myself an expert on optimizing Linux.

              If no one has the time or desire to help, that is fine. I understand. If you do have it, please help with these:
              1) Install and Optimize Ubuntu on the hardware.
              2) Run the Phoronix Test Suite with the same options as the original test.

              I don't know how Michael Larabel ran it originally (what options.) I also can't get it to run on FreeBSD. It installs (with install-sh) but then errors out immediately with this:

              To run the Phoronix Test Suite locally you must first change directories to phoronix-test-suite/
              or install the program using the install-sh script. For support visit: http://www.phoronix-test-suite.com/

              I'm running it from the directory and /usr/bin/ is where the suite binary is located.

              If I can't get help soon, I'll need to put the new box into production middle of next week. If in production, it will have 8 production VM's on it which would make it worthless for any meaningful test.

              Comment


              • Originally posted by risner View Post
                No one here willing/able to help me? If not, I'll have to abandon running a new series of tests.

                I've got the hardware and can run the test on the bare metal, I've got the time to do the OS installs and the optimizations I'm able to do (FreeBSD), but I don't have the time to learn how to run the phoronix-test-suite.

                While I am responsible for a number of Linux systems (mostly RHEL and Gentoo), I wouldn't consider myself an expert on optimizing Linux.

                If no one has the time or desire to help, that is fine. I understand. If you do have it, please help with these:
                1) Install and Optimize Ubuntu on the hardware.
                2) Run the Phoronix Test Suite with the same options as the original test.

                I don't know how Michael Larabel ran it originally (what options.) I also can't get it to run on FreeBSD. It installs (with install-sh) but then errors out immediately with this:

                To run the Phoronix Test Suite locally you must first change directories to phoronix-test-suite/
                or install the program using the install-sh script. For support visit: http://www.phoronix-test-suite.com/

                I'm running it from the directory and /usr/bin/ is where the suite binary is located.

                If I can't get help soon, I'll need to put the new box into production middle of next week. If in production, it will have 8 production VM's on it which would make it worthless for any meaningful test.
                I do not have enough knowledge to help you. For Ubuntu I would just turn off debugging, chose 250Hz option, disable some services, because sometimes they can 'eat' up to 100% cpu (afaik some package management related) and if you plan to optimize FreeBSD as much as possible I would also use writeback mode and Ext4 file system in Ubuntu. The best option would be probably to ask at lkml about how to optimize it

                turning on writeback:



                I did not run PTS on FreeBSD, but on some Linux distros if PTS is already installed (if not I just cd to its directory and do ./phoronix-test-suite [...]):

                phoronix-test-suite list-tests - to show available tests,
                phoronix-test-suite install [test_name]
                phoronix-test-suite benchmark [test_name] no special options are required afaik.

                To make it running in Arch I had to change some option in php like described in this thread:




                However, it seems there's nothing wrong with the results:

                Last edited by kraftman; 02 October 2009, 07:58 AM.

                Comment


                • Originally posted by kraftman View Post
                  I do not have enough knowledge to help you. For Ubuntu I would just turn off debugging, chose 250Hz option, disable some services, because sometimes they can 'eat' up to 100% cpu (afaik some package management related)...
                  Why 250 Hz? To be more server oriented? If yes then he should enable and the "no forced preemption" option in the proccesor type and features and also to choose the correct type of processor.

                  Comment


                  • Originally posted by Apopas View Post
                    Why 250 Hz? To be more server oriented? If yes then he should enable and the "no forced preemption" option in the proccesor type and features and also to choose the correct type of processor.
                    Yep, it's noticeable in the test I gave a link. :>

                    http://unix.derkeiler.com/Mailing-Li.../msg00073.html - it's buried there somewhere

                    Comment


                    • Well, I'm about to give up.

                      I've just spend 8 hours trying to get Phoronix running in FreeBSD.

                      Running the install-sh program puts everything in /usr/bin and /usr/share/phoro* but doesn't make a pts / pts-core directory, so I need to cp those from the install directory to /usr/share/phor*/ before the program will think it is installed.

                      Once that is past, if I try to run an install, it says the checksum fails, but I can download the file and the checksum matches.

                      The only way I think I could manage to do these tests is to do them by hand and not use Phoronix-test-suite. Which I really don't have the time to do all these tests by hand. I've already spent close to 16 hours installing/prepping this.

                      Errors that I can't debug:
                      test# tar -xf phoronix-test-suite-2.0.0.tar
                      test# cd phoronix-test-suite
                      test# ./install-sh

                      Phoronix Test Suite Installation Completed

                      Executable File: /usr/bin/phoronix-test-suite
                      Documentation: /usr/share/doc/phoronix-test-suite/
                      Phoronix Test Suite Files: /usr/share/phoronix-test-suite/

                      test# phoronix-test-suite install compress-7zip

                      To run the Phoronix Test Suite locally you must first change directories to phoronix-test-suite/
                      or install the program using the install-sh script. For support visit: http://www.phoronix-test-suite.com/

                      test# cp -r pts pts-core /usr/share/phoronix-test-suite/
                      test# phoronix-test-suite install compress-7zip
                      dcop: not found

                      ====================================
                      PHORONIX TEST SUITE - WELCOME
                      ====================================

                      The Phoronix Test Suite is the most comprehensive testing and
                      ...
                      Do you agree to these terms and wish to proceed (Y/n)? y


                      Checking For Needed External Dependencies.

                      ====================================
                      Downloading Files For: compress-7zip
                      Estimated Download Size: 2.38 MB
                      ====================================



                      Downloading File: p7zip_4.65_src_all.tar.bz2


                      The MD5 check-sum of the downloaded file is incorrect.
                      Failed URL: http://voxel.dl.sourceforge.net/sour...rc_all.tar.bz2
                      Attempting to re-download from another mirror.


                      Downloading File: p7zip_4.65_src_all.tar.bz2


                      The MD5 check-sum of the downloaded file is incorrect.
                      Failed URL: http://internode.dl.sourceforge.net/...rc_all.tar.bz2
                      Attempting to re-download from another mirror.
                      ...
                      Download of Needed Test Dependencies Failed! Exiting.


                      Installation of compress-7zip test failed.

                      test# fetch http://ovh.dl.sourceforge.net/source...rc_all.tar.bz2
                      p7zip_4.65_src_all.tar.bz2 100% of 2442 kB 957 kBps
                      test# md5 p7zip_4.65_src_all.tar.bz2
                      MD5 (p7zip_4.65_src_all.tar.bz2) = f78ed232436dc8f8be25a2f95d4e5f9a
                      test# ls -l p7zip_4.65_src_all.tar.bz2
                      -rw-r--r-- 1 root 1000 2501418 Feb 14 2009 p7zip_4.65_src_all.tar.bz2
                      test# cat /usr/share/phoronix-test-suite/test-resources/compress-7zip/downloads.xml
                      <PhoronixTestSuite>
                      <Downloads>
                      <Package>
                      <URL>http://internode.dl.sourceforge.net/...rc_all.tar.bz2, http://ovh.dl.sourceforge.net/source...rc_all.tar.bz2, http://voxel.dl.sourceforge.net/sour...rc_all.tar.bz2, http://superb-east.dl.sourceforge.net/sourceforge/p7zip/p7zip_4.65_src_all.tar.bz2</URL>
                      <MD5>f78ed232436dc8f8be25a2f95d4e5f9a</MD5>
                      <FileSize>2501418</FileSize>
                      </Package>
                      </Downloads>
                      </PhoronixTestSuite>
                      test# cat /tmp/w.php
                      <?php
                      $md5file = md5_file("p7zip_4.65_src_all.tar.bz2");
                      file_put_contents("md5file.txt",$md5file);
                      ?>
                      test# php /tmp/w.php

                      test# cat md5file.txt
                      f78ed232436dc8f8be25a2f95d4e5f9atest#
                      test#

                      Comment

                      Working...
                      X