Announcement

Collapse
No announcement yet.

Fedora, Debian, FreeBSD, OpenBSD, OpenSolaris Benchmarks

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • #61
    BSD and Gcrypt

    It's only a guess, but maybe the difference in performance in Gcrypt between BSDs and other UNIXes is caused by different ways of generating the random numbers (for encryption).

    Comment


    • #62
      Could not help myself from commenting about those SAP benchmarks even though that might be a boring discussion right now.

      Kebabbert, you claim that HP must have done everything that they can to prove that their hardware was the fastest and that this benchmark thus proves that Linux is not as scalable as Solaris 10.

      Well there is a problem with that theory and that is that you do not really know if this is what HP was after.

      I did some quick "configure your server" on both sun.com and hp.com with the hardware used on each test and the price difference is more than 100%

      The HP costs $57254 while the Sunfire costs $125000, so it might not be that far fetched to assume that HP and Sun are targeting different markets with these benchmarks.

      Not only is the amount of RAM different (and yes this matters of course since it affects caching), they also use different DBs (and yes this matters since the SAP software has to talk with the DB otherwise it wouldn't be used in the benchmark now would it, and if the DB in one case is answering slower than the other there would most definitely be a difference in CPU utilization on the main server since it would be stuck waiting for answers, aka I/O bound).

      Also they have chosen to run with a different amount of benchmark users, 10000 on Sunfire and 8022 on HP.

      Considering that the Sunfire is more than twice the price of the DL785 there may be something with the hardware that makes is scale better even though it has lower GHz.

      All that we can tell from these benchmarks is that SAP with Oracle 10 runs better on the Sunfire than SAP with MaxDB on a DL785, and that your performance need will determine wheter the double costs is worth it or not.

      Comment


      • #63
        Originally posted by sustmi View Post
        It's only a guess, but maybe the difference in performance in Gcrypt between BSDs and other UNIXes is caused by different ways of generating the random numbers (for encryption).
        I don't think so, I downloaded the libgcrypt code and when benchmarking symmetric chipers it uses a preset key that is generated like this:
        Code:
        for (i=0; i < keylen; i++)
            key[i] = i + (clock () & 0xff);
        And this is done outside the iterations so it should only by run once per each chiper tested.

        It must be something else, perhaps the hand coded asm is disabled on BSD?

        Comment


        • #64
          Kraftman,
          Jesus. Just because SAP and Microsoft has a partnership, it doesnt mean that Solaris gets favoured over Linux in SAP benchmarks. It is like: My dog likes sausages, and therefore Solaris is favoured over Linux. Your logic is epic fail.

          I suggest you look at these sites. Linux companies are partners with SAP. Your theory says that SAP favours partners, right? SAP should favour Linux companies and give Linux the victory. It doesnt happen. Fail theory.

          See all of Red Hat’s open source products, including platform, cloud, application services, app development, automation, and data services products.


          Regarding SAP doesnt like Open Source, well, Sun is the company that has contributed most to OpenSource. You havent heard about OpenSolaris? SAP should hate Sun, if your theory is correct. And should punish Sun in the benchmarks. But Sun wins. Fail theory again.

          Your links have nothing to do with Solaris and Linux. Why dont you show us links about scuba diving instead? You havent proven anything that supports your theory.

          I feel it is a big waste of time discussing with someone who does not the basics of logic, nor how to supply valid arguments. Now you are just posting jibberish. I feel like I am discussing with a fourth grader. Why should I spend time explaining to a fourth grader, who calls me "idiot", pointing out all his errors and try to teach him basic logic? This "discussion" does not give me anything back. I have asked you to provide links to support your claims many times, but your links are just plain weird and doesnt support you at all. I want to have an interesting academic discussion with someone - who at the very least - finished the fourth grade. I have double M Sc, I work in a world famous, large finance company, I work with brilliant people, among the best in their field. I am not used to fourth graders. I will continue discussing with you when you have something sensible to say. My spare time is too precious to waste it on jibberish.



          Just some quick examples on strange reasoning from you:
          1) You prove that Linux is faster than Solaris by posting links to a Linux FUDer. I read that link and discovered the Linux FUDer compares 800MHz SPARC to 2.4GHz Intel Core Duo. I objected and said it was unfair. You didnt care. But now, at last you admit:
          "Didn't I agree this wasn't a fair comparison?"
          If you think that link was unfair, why did you post it in the first place? Why did you not agree it is was unfair when I first said so? But now you agree it is unfair.


          2) You claim that SAP has partnership with MS, and therefore Solaris is favoured over Linux. Well, SAP has partnership with RedHat and SuSE too. Wrong again.


          3) You claim that I FUD. But everything I claim, is backed up by relevant links and certified benchmarks and white papers in an academic way - then it is not FUD. On the other hand, YOU, have not provided any relevant backup for your claims. And you claim that I lie, but do not/can not point out my lies. You post links that Linus is faster than Solaris, on 800MHz SPARC vs 2.4GHz Dual Core - that is FUD and unfair. If I had not read your link, no one have noticed your FUD. I claim that you are the FUDer, not me.


          4) You post some strange links about MySQL having a slow Sum() routine and therefore the whole Solaris is buggy and slower than Linux. Well, that is just plain wrong. If Solaris is faster than Linux, it doesnt mean it is faster in every little single thing. Solaris may be faster as a whole, but slower on some small parts. Wrong of you, again.

          You post a link, where a guy says MySQL query slow on Solaris. The Solaris T2000 machine he uses is slow as a 1.4GHz Pentium 3. The T2000 has many small cores that are slow as a 1.4GHz P3. If you start a big job with lots of work, then all cores will work simultaneously very effective. If you do a single threaded thing, that Niagara will only use one core, and each core is slow. You have googled lots about my posts, about Niagara vs POWER6. You know that the Niagara is fast on multi threaded work, but is slow on single threaded work. You linked to such posts, where I discuss these things. You know this, why do you FUD again? Why do you try to fool people that Solaris is slow, by showing links to MySQL query slow on a T2000? 1.4GHz P3? FUD again.


          5) You say that a company can provide criticism without you consider it being FUD, only if the company is not competitors/enemies. Well, that is wrong. Even competitors/enemies can criticise. IBM says that Niagara CPU is slow on single threaded work, and that is true. I have never objected to that. I do not consider that as FUD, it is true. But IBM also says things about Niagara which is FUD and lies. I have discussed these things, you have linked to them. Just because a company is competitor it doesnt mean that everything is FUD. Maybe some things are true.

          As you write:
          "I agree, I also was FUDing sometimes (according to wikipedia)"


          6) You say that the reason there are several more Linux links showing Linux having problems, than Solaris - is because Linux has more users. That may be true and this is almost the only sane thing you have said. I have no objection to this, and you win this argument by providing sound correct logic. (MS says Windows is more attacked by virus, not because Windows is unsafe or bad - but because it is more used than Linux/Unix. That may be true, too.)


          7) I quote links to lots of Linux kernel developers, everyone says Linux has declining code quality, Linux kernel is bloated, Linux kernel is going to pieces, etc - and you explain to me that they did not mean what they said. I can not be much clearer than this Linus T: (something like) "Linux is bloated.. no question about it". And you interpret it very strangely. I bet in a few posts, you are ready to agree that they mean exactly what they say. I find your explanation very strained and not probable.


          8) I have posted lots of links where Linux gets unstable under high load and crumbles. You say it is not true, and your proof is one link where IBM explains that Linux is stable. That is wrong thing of you to do. If you want to show that Linux is stable under high load, you can not counter this by showing one IBM link. You must instead disprove my links, by showing that each link is wrong. You havent done that. You have not disproved anything.


          9) You call me names, such as "idiot", "there is something wrong with your head", etc.


          10) Regarding that Harvard professor and FUD. If he is paid to write a paper - then the professor can not scew science too much. He can not claim obviously false things.

          If someone already proved he is wrong, then his paper will be rejected, it will not be accepted. Each paper is examined by experts in the field, to see if the paper is true and reasonable. If he is paid to write something strange, then the experts will object, and the professor has to withdraw his paper and change it. If he is paid to write something true, then the experts will accept his paper and it will be published as a contribution to science.



          etc etc etc. There are a lot more to say, but I stop here.

          Sorry, Kraftman, but you dont provide any interesting sane arguments nor links. You havent proven anything, nor disproven anything. If you have some interesting links or find interesting arguments I may comment on them, but they better be relevant. Not links or arguments about scuba diving, when you talk about Solaris. I have more important things to do, than to continue a totally meaningless discussion, where you contradict yourself. I will stop here.

          Comment


          • #65
            Originally posted by F.Ultra View Post
            Could not help myself from commenting about those SAP benchmarks even though that might be a boring discussion right now.
            Agreed. I dropped that boring "discussion" now. Nothing interesting nor concrete was said.


            Originally posted by F.Ultra View Post
            Kebabbert, you claim that HP must have done everything that they can to prove that their hardware was the fastest and that this benchmark thus proves that Linux is not as scalable as Solaris 10.

            Well there is a problem with that theory and that is that you do not really know if this is what HP was after.
            HP has submitted results to a BENCHMARK that measures performance. It did not measure reliability, it did not say anything about "uptime" or something similar. If we talk about FPS in a game, then it is reasonable that it is a FPS benchmark, and not about picture quality? If we talk about peoples weight, then is it reasonable to believe that the salary was the thing we measured?

            IBM submitted results to TPC-C benchmarks earlier, you do not expect IBM to tailor their $35 million Unix server for reliability then? These benchmarks are very prestigious.


            Originally posted by F.Ultra View Post
            I did some quick "configure your server" on both sun.com and hp.com with the hardware used on each test and the price difference is more than 100%

            The HP costs $57254 while the Sunfire costs $125000, so it might not be that far fetched to assume that HP and Sun are targeting different markets with these benchmarks.
            I dont agree with this. IBM benchmarks three P570 servers that costs 1,000,000 USD against one Sun T5440 which costs 76,000 USD. IBM thinks it is fair.

            We are not measuring price/performance here. Only the highest performance. I agree that it seems that HP has a cheaper server (if you configured exactly the same models as in the white papers, e.g. did you use the fastest HP RAM memory sticks?). But we are discussing who has the world SAP record right now. Not the best price/performance ratio.


            Originally posted by F.Ultra View Post
            Not only is the amount of RAM different (and yes this matters of course since it affects caching), they also use different DBs (and yes this matters since the SAP software has to talk with the DB otherwise it wouldn't be used in the benchmark now would it, and if the DB in one case is answering slower than the other there would most definitely be a difference in CPU utilization on the main server since it would be stuck waiting for answers, aka I/O bound).
            I dont think that HP Enterprise benchmarking team would have used less RAM if it yielded lower score. For these prestigous benchmarks, every company uses lots of resources to try to be the best on the market. They try everything they can to get higher scores. Look what IBM did in their former TPC-C record, they shortstroked several thousands of drives! (short stroke is to only use a tiny partition, to decrease search time). These benchmarks are prestigious and they try everything to win it.


            Originally posted by F.Ultra View Post
            Also they have chosen to run with a different amount of benchmark users, 10000 on Sunfire and 8022 on HP.
            Sun used a heavier benchmark. The more users you have, the more difficult it gets. Any server can handle 10 users adequately. If you try to handle 100,000 users, then most servers will choke. The more users, the higher the difficulty.


            Originally posted by F.Ultra View Post
            Considering that the Sunfire is more than twice the price of the DL785 there may be something with the hardware that makes is scale better even though it has lower GHz.
            Maybe. But we also know that server Enterprise gear costs more as they have better quality. Sun has hard drives that cost 1000 USD. These are built to last longer, but they are not faster. These drives are selected from the drive manufacturers. Mac Pro is more expensive than Dell computers, but Mac Pro uses server stuff which are more reliable. A dual socket server mobo costs 500 USD as the cheapest? But I wonder if the Mac Pro is faster than a corresponding Dell? I doubt that.

            IBM stuff costs 5-10x more than the corresponding Sun server. And as I have proved in other discussions, the IBM servers are not that faster than Sun gear. On SIEBEL v8 benchmarks, you need six IBM P570 ($413,000/each) servers to match one Sun T5440 ($76,000).


            Originally posted by F.Ultra View Post
            All that we can tell from these benchmarks is that SAP with Oracle 10 runs better on the Sunfire than SAP with MaxDB on a DL785, and that your performance need will determine wheter the double costs is worth it or not.
            And we also can tell that Solaris utilized the CPUs much more efficient, and got higher scores on slower hardware. And we know that Solaris has an old reputation to scale well. That is the reason Solaris won, on slower hardware.

            Comment


            • #66
              And we also can tell that Solaris utilized the CPUs much more efficient, and got higher scores on slower hardware. And we know that Solaris has an old reputation to scale well. That is the reason Solaris won, on slower hardware.
              No we don't. We do not know why the Sunfire machine has higher CPU utilization. It could be because Solaris scales better, it could also be because it has a higher load (higher number of test users), it could be because the Oracle database is faster than the MaxDB. It could also be because SAP is not designed to scale well on Linux.

              It could be all these things and we cannot know which it is. I can tell you this though, I create realtime systems for the financial markets for a living, and I don't have any problems creating software that scales to 16 core systems on Linux with 99% CPU utilization. But that is because I know how to make things scale well under Linux, running the same code under Solaris might make it fail miserably performance wise, I do not know since I do not run Solaris.
              I dont agree with this. IBM benchmarks three P570 servers that costs 1,000,000 USD against one Sun T5440 which costs 76,000 USD. IBM thinks it is fair.

              We are not measuring price/performance here. Only the highest performance. I agree that it seems that HP has a cheaper server (if you configured exactly the same models as in the white papers, e.g. did you use the fastest HP RAM memory sticks?). But we are discussing who has the world SAP record right now. Not the best price/performance ratio.
              We are not talking about what's fair, and yes I configured the servers exactly as they where in the benchmarks. You have to ask HP about what they want their benchmark to prove, you cannot simply guess why they entered with the setup that they did and then create arguments based on that guess.

              That IBM brings an even more expensive machine with lower performance to the table is no evidence on HP's behavior, it only tells that IBM chose to enter with this particular setup.

              Comment


              • #67
                Ok you may disagree that Solaris scales better on SAP, but we both agree that Solaris had better cpu utilization and had better performance on slower hardware - in this benchmark, right? So for this SAP setup, Solaris is the best. Right?



                If you create realtime systems for the financial markets for a living, that is good for you. But it does not prove anything. You can not exptrapolate that. We all know that Linux scales well to 4CPUs with quad core = 16 cores. That is no question about it. The question is how Linux scales to many cores, like 48 cores or more. I claim (based on what I have read on the internet) that scaling to more cores is more difficult. 4 CPUs is easy. But 100s of CPUs/cores is difficult. We saw that Linux has severe problems with 48cores in SAP bench even today - we have not agreed on what the problem is, but we see there is a problem with Linux scaling on SAP. We see Linux has low CPU utilization.

                Also, regarding your programs, maybe you have extremely parallell problems, making it easy to scale to 16 cores? Then I expect your code to scale well to many 100s of cpus. Some problems are inherently parallell. Such problems are not interesting though, as it is no match to make such code scale well. So, therefore, that you can scale to 16 cores on Linux is expected becuase Linux handles small numbers of cores/cpus well. Also, your problems may be parallell, so it says nothing.



                Regarding pricing, if you have configured the machines similarly, then we can agree that Sun is more expensive. But that is expected, as I showed that other Unix vendors as IBM is many times more expensive than Sun gear. Ok, so we agree that Sun charges more for slower hardware. Sun also has a reputation of building high quality server gear. I have heard that 1 small SAS disk drive can cost $1000 or more, from Unix vendors. I doubt that disk drive is faster, but it is built to be powered on 24/7, which consumer grade disks are not built to do.

                Comment


                • #68
                  Ive read something interesting on SAP benchmarks:

                  "You should just think about the point that there is no significant storage at a saps benchmark and that the influence of the database is rather miniscule. So there is no really I/O heavy lifting for example."

                  So, I think we could rule out the database here. I could email SAP to get this confirmed, if you wish. And also I could email SAP to ask about RAM requirements to do a SAP benchmark. Maybe 32GB RAM is enough for SAP benchmark?

                  If so, then we can rule out RAM size (but RAM speed is important) and we can rule out the DB for SAP benchmarks.

                  Comment


                  • #69
                    Ok you may disagree that Solaris scales better on SAP, but we both agree that Solaris had better cpu utilization and had better performance on slower hardware - in this benchmark, right? So for this SAP setup, Solaris is the best. Right?
                    No I don't disagree at all, what I am trying to tell you is exactly what you wrote yourself in the text that I have quoted. The only thing that your posted benchmarks tells is that SAP runs better on Solaris on a Sunfire than Linux on a HP DL785.
                    Also, regarding your programs, maybe you have extremely parallell problems, making it easy to scale to 16 cores? Then I expect your code to scale well to many 100s of cpus. Some problems are inherently parallell. Such problems are not interesting though, as it is no match to make such code scale well. So, therefore, that you can scale to 16 cores on Linux is expected becuase Linux handles small numbers of cores/cpus well. Also, your problems may be parallell, so it says nothing
                    If you know what you are doing, most things are easy to parallelize, and even if they are not it's not often the kernel that stands in your way. I wrote about 16 cores because that is the biggest hardware that I have needed so far (only have to process a few million TPS), so I had first hand experience with that number.
                    Regarding pricing, if you have configured the machines similarly, then we can agree that Sun is more expensive.
                    This is completely besides the point, what I wanted you to notice was that HP chose to enter the benchmark with a much cheaper solution (they have hardware matching and surpassing the Sunfire when it comes to price), and the question is why. HP is the only ones that knows this.
                    "You should just think about the point that there is no significant storage at a saps benchmark and that the influence of the database is rather miniscule. So there is no really I/O heavy lifting for example."
                    Even if the DB is not used heavily it can still have an impact on the numbers, especially since we are talking about close to a million TPS. Trust me, I write this kind of software.

                    Comment


                    • #70
                      Originally posted by F.Ultra View Post
                      No I don't disagree at all, what I am trying to tell you is exactly what you wrote yourself in the text that I have quoted. The only thing that your posted benchmarks tells is that SAP runs better on Solaris on a Sunfire than Linux on a HP DL785.
                      Ok, glad we can agree on something at last!


                      Originally posted by F.Ultra View Post
                      If you know what you are doing, most things are easy to parallelize,
                      I do not agree on this. I suspect you have not studied complexity theory for parallell computers. There are many problems that are P-complete.


                      Originally posted by F.Ultra View Post
                      and even if they are not it's not often the kernel that stands in your way. I wrote about 16 cores because that is the biggest hardware that I have needed so far (only have to process a few million TPS), so I had first hand experience with that number.
                      I think it is wrong to extrapolate from 4 cpus up to hundreds of CPUs. Microsoft has tried to scale for decades, and still dont succeed.


                      Originally posted by F.Ultra View Post
                      This is completely besides the point, what I wanted you to notice was that HP chose to enter the benchmark with a much cheaper solution (they have hardware matching and surpassing the Sunfire when it comes to price), and the question is why. HP is the only ones that knows this.
                      To me, the price is not important. IBM has very expensive gear. Any modern x86 CPU is 5-10x faster than the fastest Mainframe CPU. You need 16 Nehalem-EX cpus to match the biggest IBM Mainframe with 64 CPUs. Both give 400Mips/cpu, the difference is that Nehalem-EX gives that under software emulation which is a factor 5-10x slower. The biggest mainframe costs maybe 20 million USD? Read about this on wikipedia on article "Herkules Mainframe emulator"

                      I have Armani clothes that cost much more than ordinary clothes. Price doesnt tell you everything. I think we should study the components, are they equivalent?

                      HP and Sun entered this benchmark with almost identical hardware. HP used slightly faster components, yes.


                      Originally posted by F.Ultra View Post
                      Even if the DB is not used heavily it can still have an impact on the numbers, especially since we are talking about close to a million TPS. Trust me, I write this kind of software.
                      For the benchmark, I suspect it is designed to not be DataBase bound. That would be a bad benchmark. Most probably the speed of the RAM is important, not the DB.

                      I have no reason not to trust you, but I can email SAP and ask about the benchmark if you wish. OTOH, maybe you should trust me, as I have double master; one in Comp Sci (specialization: algo theory and discrete math) and another Master in Math (specialization: pure math/financial math). The Math Master includes a B Sc in Math. The Comp Sci Master includes a B Sc in Comp Sci: two B Sc and two Masters. I work at one of the most famous well known financial companies, with extreme demands on our largest systems. Maybe you are using it.

                      Comment

                      Working...
                      X