Announcement

Collapse
No announcement yet.

Linux vs Solaris - scalability, etc

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Linux vs Solaris - scalability, etc

    Creator of ZFS, Jeff Bonwick, said that Linux scales bad. Many Unix people (including Kebabbert) say that Linux scales bad.

    Linux supporters say that Linux scales excellent, they say Linux scales to 1.000s of cores. So what is the deal, does Linux scale bad or what?

    The thing is, Linux scales excellent on HPC servers (a big cluster, a bunch of PCs sitting on a fast network). Everybody say this, including Unix people. No one has denied that Linux scales excellent on a cluster. It is well known that most of super computers run Linux. And those large super computers with 1000s of cores, are always a big cluster. Google runs Linux on a cluster of 900.000 servers.




    The famous Linux SGI Altix server with 1000s of cores is such a HPC clustered server running a single Linux kernel image. This SGI Altix server has many thousands of cores:

    Support for up to 16TB of global shared memory in a single system image enables Altix UV to remain highly efficient at scale for applications ranging from in-memory databases to a diverse set of data and compute-intensive HPC applications.

    CESCA, the Catalonia Supercomputing Centre, has chosen the SGI Altix UV 1000 for its HPC system



    Here we have another Linux HPC server. It has 4.096 up to 8.192 cores and runs a single Linux image, just as the SGI Altix server:

    The vSMP hypervisor that glues systems together is not for every workload, but on workloads where there is a lot of message passing between server nodes ? financial modeling, supercomputing, data analytics, and similar parallel workloads. Shai Fultheim, the company's founder and chief executive officer, says ScaleMP has over 300 customers now. "We focused on HPC as the low-hanging fruit,"
    A programmer writes about this server:
    I tried running a nicely parallel shared memory workload (75% efficiency on 24 cores in a 4 socket opteron box) on a 64 core ScaleMP box with 8 2-socket boards linked by infiniband. Result: horrible. It might look like a shared memory, but access to off-board bits has huge latency.
    Thus, we see that these servers are just HPC servers. And we if look at the workload benchmarks for instance, the SGI Altix server they are all parallell benchmarks. Clustered workloads.




    However, let us talk about SMP servers. SMP servers are basically, a single big fat server that has up to 64 cpus, weighs 1.000kg and costs many millions of USD. For instance, the IBM Mainframe z10 has 64 cpus. The newest IBM z196 Mainframe has 24 cpus. The biggest IBM Unix server, the P795 has 32 cpus. The earlier IBM P595 for the TPC-C world record costed 35 million USD list price. The biggest HP-UX server Superdome has 32 cpus or so. The biggest Solaris server has today 64 cpus.

    Why dont IBM just insert another 64 cpus and reclaim all world records from Oracle? Or insert another 128 cpus? Or even 512 cpus? Or insert 1024 cpus? No, it is not possible. There are big scalability problems on as many as 64 cpus when you use a SMP server.

    On these kind of SMP servers, Linux scales bad. The biggest Linux SMP servers today have 8 cpus, which are the normal x86 servers that for instance Oracle/HP sells. On a SMP server, Linux have severe problems to scale. The reason is explained by Ext4 creator Ted Tso:
    thunk.org/tytso/blog/2010/11/01/i-have-the-money-shot-for-my-lca-presentation/
    ...Ext4 was always designed for the ?common case Linux workloads/hardware?, and for a long time, 48 cores/CPU?s and large RAID arrays were in the category of ?exotic, expensive hardware?, and indeed, for much of the ext2/3 development time, most of the ext2/3 developers didn?t even have access to such hardware. One of the main reasons why I am working on scalability to 32-64 nodes is because such 32 cores/socket will become available Real Soon Now...



    We also see the SAP benchmarks. Linux used slightly faster CPUs, and slightly faster DIMM memory sticks, and still Solaris got 23% higher score. The Linux cpu utilization was 87% which is quite bad on 48 cores. Solaris had 99% cpu utilization on 48 cores, because Solaris is targeted for big SMP servers. Solaris uses the cores better, and that is the reason Solaris got higher benchmarks.

    Linux, 48 core, AMD Opteron 2.8 GHz
    download.sap.com/download.epd?context=40E2D9D5E00EEF7CCDB0588464276 DE2F0B2EC7F6C1CB666ECFCA652F4AD1B4C

    Solaris, 48 core, AMD Opteron 2.6GHz


    Solaris used 256 GB RAM, and Linux used 128 GB RAM. The reason is because if Linux HP server wants to use 256 GB RAM, then Linux must use slower DRAM memory sticks. So HP chose fast DRAM memory sticks which had lower storage capacity. But SAP benchmarks require only 48GB RAM or so, so it is not important if a server uses 128GB or 256GB or 512GB RAM.




    There dont exist big SMP Linux servers on the market for sale. Thus, the Linux kernel developers have no hardware to develop on, just as Ted Tso explained.

    Thus, Linux scales excellent on clusters HPC servers. But Linux dont scale too well on SMP servers. Thus, Bonwick is right when he says that Linux scales bad on SMP servers. There are no big SMP Linux servers for sale on the market, there are no 16 cpu Linux servers, nor 24 cpu servers.

    There are Linux SMP servers: 4, 6, 8 cpu servers and then there are Linux HPC servers: 4096 core servers or more. But where are the 32 cpu Linux SMP servers? No one sells them. Very difficult to make. Big scalability problems. You need to rewrite the OS, and build specialized hardware that costs millions of USD.

  • #2
    Ok, answer to my comments here.

    Comment


    • #3
      Originally posted by kraftman View Post
      Ok, answer to my comments here.
      Let the wars begin!

      Comment


      • #4
        Kebabbert, you appear to be quite confused when it comes to terminology, and hence your posts on multiprocessor scalability tends to read like complete nonsense. You always state that you "just want to learn and be correct", so let me try to clear things up for you before we even get into the discussion of performance or scaling numbers.
        First, you seem to think that HPC and SMP are comparable and opposing terms. They are not - SMP (Symmetric Multi-Processing) describes a specific architecture for multiprocessor systems. HPC (High Performance Computing) on the other hand is a _usage scenario_ for large multiprocessor systems. HPC can be done on SMP systems, on distributed clusters or recently via GPGPU- it all depends on the specific HPC task.

        Also, "SMP" in and of itself is today a somewhat "fuzzy" term, since there are barely any modern server system on the market nowadays that really fit the hard "SMP" criteria. The large "SMP" systems are almost all NUMA designs.

        In any case, the distinction you're trying to make when you mistakenly contrast "HPC" vs. "SMP" is really between _shared memory multiprocessor systems_ versus _distributed memory multiprocessor systems_.

        The sytems that you like to refer to as "large SMP" are shared memory systems. The "cluster of small systems on a fast network" is a distributed memory system. And then, thanks to the wonders of hypervisors and hardware virtualization, we now have "hybrid" systems like ScaleMP, which essentially creates a virtual shared memory system on top of a distributed memory cluster. In these systems, the "Single OS Image" is really a virtual machine, and the "shared memory" is "faked" via a software layer in the hypervisor. These virtual shared memory systems would theoretically be able to run applications written for shared memory systems, but as shown in the link in your post won't perform well since essentially they will be like a NUMA machine with an absolutely ATROCIOUS R:L NUMA factor. On top of that, cache coherency will have to be done by _software_. I don't even like to think about the performance implications of that...

        Now, once we have these distinctions cleared up, let's just lay down some basic facts:
        1. The SGI Altix UV and Altix 4000/3000 systems are NOT distributed memory systems, they are shared memory systems. They are the same "class" of multiprocessor servers as what you (Kebabbert) would refer to as "large enterprise SMP systems", such as the HP SuperDome, Oracle T & M-series enterprise servers, etc.
        2. The fact that the Altix UV systems add processor nodes in blade format doesn't change fact 1 since what the blades plug into is not a network, but a NUMA interconnect - the equivalent to this in the current Oracle servers would be the Jupiter interconnect used for the crossbars. I've seen some people argue that the M9000 isn't NUMA, since it's a crossbar architecture, but as you can see straight from the
        3. The relatively recent Altix ICE systems ARE clusters. These did not exist until 2010 - every large multiprocessor system sold by SGI prior to Altix ICE has been a shared memory NUMA system.
        4. Even a single AMD Opteron 6100-series CPU contains two NUMA nodes, which really underlines just how clouded the line between SMP and NUMA systems is these days.


        Now some notes on the above - I've seen it argued in a few similar discussions to this that the M9000 is not a NUMA system, and Sun/Oracle likes to present it as such, and this is not as dishonest as it might initially seem. In the vast majority of NUMA systems, each CPU or CPU node has its own local memory with it's own local memory controller, and CPUs from other nodes will need to "traverse" that CPU's memory controller to access the remote memory. The M9000 uses a crossbar architecture that differs from this layout, and which would make at least each CPU board "non-numa" since each CPU has an "equal" distance to memory. BUT - the boards then connect to a system crossbar, and while very very quick, there IS a NUMA factor for cross-CPU-board memory access, making the system NUMA after all. However the NUMA factor is apparently so low that NUMA optimizations are not always beneficial;ref - http://kevinclosson.wordpress.com/20...-gives-part-ii ). Perhaps they should have called this beast FloraServer and marketed it with the "I Can't Believe It's Not SMP!" slogan


        Anyway, now that that's out of the way, let's at least touch on some of the other of your misconceptions that seems to have grown out of this lack of basic understanding of different multiprocessor architectures.

        1. "No large Linux SMP systems" - maybe technically true since there aren't any large "true" SMP systems today at all. Even the "big" commodity boxes from Dell, HP and IBM are NUMA systems, since both the Opteron and the Westmere EX are NUMA designs in multi-socket configurations.

        2. But obviously, the biggest shared memory multiprocessor systems sold today are, in fact, "Linux servers".

        3. Item 2. has been true since 2006, when SGI's Origin 3000 series of NUMA machines were discontinued, and the Altix systems became officially supported in 1024 CPU configurations.

        4. SGI has always been WAY ahead of SUN when it comes to massive shared memory multiprocessor systems:
        First SGI SMP server was launched in 1988 and supported up to 8 CPUs. Sun did not launch its first SMP server until 1991, and only did 4-way SMP at the time.
        In 1993, SGI launched a 36-way SMP server. SUN was at 20-way max.
        In 1996, SGI introduced their first NUMA systems, the Origin 2000, available in up to 128-way configurations that year. Sun introduced the E10000, a 64-way server.
        In 2000 when the Origin 2000 was superseded by the Origin 3000, the max configuration was 512 CPUs. SUN's biggest box was still the E10k.
        In 2001 the Origin 3000 scaled to 512 CPUs (it did so from the get-go), and Sun introduced the R15k - maxing out at 106 CPUs, still less than SGI's initial Origin 2000 offering in 1996.
        In 2003 SGI announced its first Linux-based large scale Itanium Based NUMA systems, the Altix 3000 series. The inital max supported configuration was "Only" 64 CPUs, and SUN had the R25k, with 72 CPUs and 144 "threads". While SGI was way ahead of SUN on hardware scaling, there's also no question that Solaris was ahead of Linux at this point. Nonetheless - already back in 2003, there was a large scale multiprocessor server with 64 CPUs being sold. And...
        In 2006 when the Origin 3000 series was discontinued, the Altix 4000 had already been introduced and scaled to 512 CPUs. SUN's biggest was the M9000 which at the time topped out at 128 real cores, or the 25k with it's 72 cores and 144 threads.

        So as you can see, SGI had plenty of experience at making large scale shared memory multiprocessing and SMP systems longer than SUN, and SGI had all of this knowledge already in-house when they went to work at making Linux scale well on huge number of CPUs.

        I'll get into the whole "but how WELL does Linux vs. Solaris scale on large shared memory multiprocessor systems" in a later post - I think this is TL;DRish enough.

        Comment


        • #5
          Originally posted by kebabbert View Post
          The thing is, Linux scales excellent on HPC servers (a big cluster, a bunch of PCs sitting on a fast network). Everybody say this, including Unix people. No one has denied that Linux scales excellent on a cluster. It is well known that most of super computers run Linux. And those large super computers with 1000s of cores, are always a big cluster. Google runs Linux on a cluster of 900.000 servers.
          Needless to say, from a kernel standpoint a cluster is very different from a single system image machine.

          The famous Linux SGI Altix server with 1000s of cores is such a HPC clustered server running a single Linux kernel image. This SGI Altix server has many thousands of cores:



          Arguably, by virtue of having a single kernel image and hardware provided cache coherency, it's quite a stretch to call it a cluster. Of course, like all big machines, the NUMA factor is by necessity quite big.

          Here we have another Linux HPC server. It has 4.096 up to 8.192 cores and runs a single Linux image, just as the SGI Altix server:



          A programmer writes about this server:
          I tried running a nicely parallel shared memory workload (75% efficiency on 24 cores in a 4 socket opteron box) on a 64 core ScaleMP box with 8 2-socket boards linked by infiniband. Result: horrible. It might look like a shared memory, but access to off-board bits has huge latency.
          So a workload that gets only 75% efficiency on 24 cores/4S is slow when running on a system with cache coherency provided by a virtualization layer rather than hw and connected with IB is somehow NOT bleeting obvious? Sheesh

          Thus, we see that these servers are just HPC servers. And we if look at the workload benchmarks for instance, the SGI Altix server they are all parallell benchmarks. Clustered workloads.
          As opposed to Solaris which can magically parallelize non-parallel workloads? Duh..

          Of course, in order to utilize parallel HW to any non-trivial extent, a workload must itself have parallelism. The OS kernel can, at best, avoid imposing further scalability bottlenecks.

          However, let us talk about SMP servers.
          Lets. If you want to play pedantic word games, SMP as in SYMMETRIC multi-processing, died a long time ago due to limits imposed by physics. "SMP" machines today are essentially all CC-NUMA. Just like the SGI Altix. Everybody realized that the alternative to CC-NUMA isn't UMA (=SMP), but rather USMA (uniformly SLOW memory access); hence nobody sane does it anymore.

          Thus, Linux scales excellent on clusters HPC servers. But Linux dont scale too well on SMP servers.
          No, this is wrong. It's not about "SMP servers" vs. "HPC servers". It's more about which kind of workloads the OS has been tuned for. The Linux VM subsystem has seen extensive scalability work in order to run (mostly) HPC style workloads on gigantic CC-NUMA machines. OTOH, database workloads are very different, and there hasn't been as much interest in scalability work that would benefit these kinds of workloads (although not too long ago a big bunch of "VFS scalability" work was integrated into the mainline kernel, so this situation is changing). For various reasons, the proprietary Unix vendors have had different priorities.

          Comment


          • #6
            Originally posted by devsk View Post
            Let the wars begin!
            Damn right, but as far I can't see a Kebbabert's response. I hope you will reply in this thread to my last comment, Kebb. If you replied somewhere else then it will be great if you just copy and past your answer here.

            Comment


            • #7
              Originally posted by Qaridarium
              again and again and again you don't get the point.
              you just don't get it.. its not the solaris kernel vs the linux kernel its only ZFS vs ext4.
              and then you show us a benchmark with a file-system bottle neck "SAP"
              but you don't get it. any benchmark with a file-system bottle neck are invalid in testing the speed of a kernel!

              drive your solaris on a fat32 file-system and you will cry!
              These benchmarks are not bottle necked by filesystem. One of the reasons they have lot of RAM is to cache a lot. Have you heard about Disk Cache?



              You talk about "ZFS being the fastest filesystem", that is not true. ZFS is the most _advanced and secure_ filesystem, but ZFS is actually quite slow. The reason is that ZFS protects your data, and does checksum calculations all the time, and burn lot of CPU to do that. If you do a MD5 or SHA-256 on every block, then everything will be slow, yes? Have you done a MD5 checksum on a file? It takes time, yes?



              Ext4 and other filesystems do not protect your data, and skips calculations and are very fast on few disks. As Ext4 creator Ted Tso explains:

              Originally posted by Ted Tso
              In the case of reiserfs, Chris Mason submitted a patch 4 years ago to turn on barriers by default, but Hans Reiser vetoed it. Apparently, to Hans, winning the benchmark demolition derby was more important than his user's data. (It's a sad fact that sometimes the desire to win benchmark competition will cause developers to cheat, sometimes at the expense of their users.)
              ...
              In the case of ext3, it's actually an interesting story. Both Red Hat and SuSE turn on barriers by default in their Enterprise kernels. SuSE, to its credit, did this earlier than Red Hat. We tried to get the default changed in ext3, but it was overruled by Andrew Morton, on the grounds that it would represent a big performance loss, and he didn't think the corruption happened all that often --- despite the fact that Chris Mason had developed a python program that would reliably corrupt an ext3 file system


              Here is another cheat that Linux developers are doing, to win benchmarks:
              Thirty years ago, Linus Torvalds was a 21 year old student at the University of Helsinki when he first released the Linux Kernel. His announcement started, “I’m doing a (free) operating system (just a hobby, won't be big and professional…)”. Three decades later, the top 500 supercomputers are all running Linux, as are over 70% of all smartphones. Linux is clearly both big and professional.

              "This is really scary. I wonder how many developers knew about it especially when coding for Linux when data safety was paramount. Sometimes it feels that some Linux developers are coding to win benchmarks and do not necessarily care about data safety, correctness and standards like POSIX."



              One person said that when he does a XFS fsck of 16TB data, it takes 20 minutes or so. That is extremely quick. ZFS does a check on 16TB for many hours. Thus, XFS is much faster. But, if XFS does a fsck check of 16TB in 20 minutes, it means XFS checks 13.000MB/sec. That is not possible on a 16TB small array with 8 disks, because one typical disk give 100MB/sec. The only conclusion is that XFS does not check all the data, only some of the data. XFS fsck skips lot of controls of data, that is the only way that XFS can reach 13GB/Sec on 8 disks. Thus, XFS is fast, but not safe.



              A Linux user writes:

              ZFS uses ...loads of CPU: If you have more than one CPU, expect to dedicate 1 CPU to ZFS, when your programs do IO on ZFS! If you have a single CPU, expect 95% CPU spent for ZFS when copying. (This disadvantage is due to internal compression, bookkeeping, checksum and error correction of the ZFS code. It cannot be evaded
              Now ZFS have been improved, so it does not burn as many cpu cycles for a simple copy, but still ZFS burns lot of cpu cycles because of data protection calculations.



              There are data protection calculations all the time, on every block. The cpu burns lot of cycles to check that all data is correct. If you do a SHA-256 checksum on every block that is read, it does burn CPU cycles, I hope you know how slow checksum calculations are?

              ZFS trades CPU cycles for peace of mind. Every block is checksummed after it arrives from the disk (or network or whatever).

              One thing Jeff Bonwick's post doesn't go into, however, is the actual cost in CPU cycles of all this checksumming....is it well suited to more traditional desktop CPUs with one or two cores? How much CPU overhead is acceptable for a desktop file system driver?

              My only regret is that I have but one CPU core to give for my data...


              Sure, on many disks ZFS is faster, because it scales better on 128 disks and more. Solaris has no problem of using many cpus, and 128 disks burn lot of cpu, but that load is distributed to many cpus. Thus, you need many disks and much cpu, for ZFS to be fastest.

              On single disk/few disks I expect most filesystems to be faster. Because other filesystems dont burn lot of CPU cycles protecting your data. And if we look at the benchmarks, ZFS is never the fastest on single disk.

              The point of using ZFS, is it protects your data. What do you prefer, a slow but reliable filesystem, or a fast but unsafe filesystem that might corrupt your data? All the fancy functionality of ZFS is secondary to me. I mean it. If ZFS lacked all those functions, I would still use ZFS because I want to protect my data. That is the most important to me. I dont mind if it is a bit slower, as long as it is safe.

              Because of all this cpu cycles that ZFS burn as soon as there is a read of any block, Solaris is punished. Thus, Solaris benchmarks should actually be higher.

              Comment


              • #8
                Originally posted by kraftman View Post
                Damn right, but as far I can't see a Kebbabert's response. I hope you will reply in this thread to my last comment, Kebb. If you replied somewhere else then it will be great if you just copy and past your answer here.
                I have a job to do. It takes time to write these long posts






                Originally posted by kraftman View Post
                When Kebbaberts compares different systems it's good, but when others do this it's bad?
                Again, I compare hardware that is roughly the same in performance. There is not a big difference in performance between the Solaris and Linux servers. When Phoronix does the benchmarks, Phoronix uses the same hardware. When TomsHardware does GPU benchmarks, the rest of the hardware stays the same: same cpu, same mobo, etc. What would people say if TomsHardware compared latest Nvidia + latest x86 cpu, vs an old ATI + Pentium 4? Would this be fair? No. But why do you keep doing this every time?

                You have several times compared a 3x faster Linux server to an old Solaris server. Is this fair?

                I have seen benchmarks of 32 cpu Solaris servers, but I would never compare such a Solaris server to a dual cpu Linux server and say "this proves Solaris is faster". That would be fanboyish of me to do. Biased. I want to learn and see which system is best. Therefore I want to have a objective comparison to learn. I do not want to see propaganda, I do not want to see 64 solaris servers vs a 2cpu Linux server - that would say nothing.

                If we discuss Linux vs Solaris, then we need to have roughly similar hardware. I think it is strange that you dont agree on this? Do you think the latest NVidia vs an old ATI 4850 would be fair? Strange.




                You compare more powerful hardware to much less expensive one with less amount of RAM, slower memory sticks and slower database. Is this fair? Even slowlaris that wastes CPU cycles, because of its bloat can win benchmark when it's running on much better hardware. That CPU utilization and much lower score just means slowlaris is bloated, don't you think? Do you have something to backup your claims regarding Westmere-EX giving Linux huge advantage on 40core machine compared to 48core one? As a single CPU comparison it mayb be three times faster in some things, but do you have something to backup your claims that it will still have such advantage on 40core machine?
                It is well known that different Vendors charge different prices. For instance, IBM charges 100s of millions USD for their Mainframes. And the IBM Mainframes are really slow cpu wise. An 8-socket x86 server is almost as fast as the biggest IBM Mainframe with 24 cpus.

                The price is not important, neither the brand. What is important is that the hardware is similar. If both servers are using 6-core AMD Opteron 8435 cpus then that is good. You are comparing the worlds fastest x86 cpu vs an old AMD Opteron. The Intel cpu is 3x faster. Is this fair?

                Regarding my claims that 10-core Westmere-EX gives a huge advantage, yes, I have already showed you the SAP benchmarks where Westmere-EX was 52% faster than an old 12-core AMD cpu. The older AMD Opteron 8435 has 6 cores, so it has less than half the performance compared to this 12 core AMD cpu. Thus, the Westmere-EX is 3x faster than the old AMD Opteron 8435 that Solaris used.

                Do you really think it is fair to use 3x faster cpus in a comparison? The Westmere-EX was released this year, the server is brand new with new DRAM technique, etc. I really dont understand your concept of "fairness"? Why do I have to explain these things, as to a small child?

                Regarding if Solaris is bloated or not, it does not matter, as long as Solaris are winning benchmarks with similar hardware. Regarding if Solaris is bloated, it is only in your mind, you have never showed any links to prove that. I have asked you many times. Regarding if Linux bloated, several Linux kernel developers have said that including Linus Torvalds. Unless you call Linus Torvalds a liar, he speaks true and Linux is bloated. But maybe Linus Torvalds and all other kernel devs are lying?

                1) You have numerous times said that Solaris is bloated, but have never showed any links nor proof. I have many times asked for more information on this. So I would like you to prove this now. If you never have read this or heard this, why are you saying this? Are you FUDing? Earlier, you confessed you FUD. Is this just more of your FUD?

                2) You have said many times that Oracle is killing Solaris, but you have never showed any links nor proof. So I would like you to prove this now, I have many times asked you. Go ahead. Or is this is also FUD?

                3) You have said that Solaris is slow. You have compared 3x Linux faster servers to Solaris, that does not prove Solaris being slow. That only proves that your sense of fairness is strange. Go ahead and prove that Solaris is slow, on similar hardware. I have showed you links where Solaris holds several world records, being the fastest in the world on 4 cpu servers, and similar.




                Being HPC doesn't mean it's not SMP. Try again, because it's SMP system.
                Que? HPC and SMP is not the same thing. If they were the same thing, they would not have different names. Here is more info on SMP servers, and HPC servers:



                HPC servers are not general purpose servers, they are specialized and do only one thing well: calculations. Everything is ripped out from the Kernel.
                SMP servers are general, and can be used for everything, including calculations.
                HPC servers are similar to a Graphics Card, they are very fast on calculations but very weak on general purpose. CPUs are general purpose and can be used for calculations, but Graphics card are faster for calculations.
                Thus, CPU can do everything, including calculations.
                Thus, GPU can only do calculations and nothing else.

                SMP can do everything, including HPC work.
                HPC can only do calculations and nothing else.




                http://www.pcworld.com/businesscente...ll_street.html
                The wallstreet migration to Linux proves only Linux is fast enough and that's why they abandoned solaris. It's nothing amazing in this decision since Linux isn't bloated the way solaris is. It seems having 30% slower binaries is too much for critical workloads and that's why they replaced solaris. Damn shame.
                Now you say again that Solaris is bloated. Again, prove this. You can not just make up negative things without any evidence, because the definition of FUD says:

                FUD is ... negative and dubious/false information ... to undermine the credibility
                4) In short, if you are just saying negative and false information to undermine Solaris, then you are FUDing. Let me ask you, are you FUDing now? Or are you telling the truth? If you are telling the truth, then you can show us links that prove you are speaking true.




                Just lies. You saw 64CPUs Big Tux.
                I gave it you few times. Just google for Big Tux. It was 64CPUs (not cores) machine and afaik it has been upgraded lately.
                Kebbabert's lying as usual. Like I said, Big Tux.
                I spoke earlier of SMP servers, for instance IBMs large Unix servers with 32 cpus. And I mentioned Solaris with 64 cpus. And I mentioned HP Unix Superdome servers with 32-64 Itanium cpus.

                This Linux Big Tux server you speak of, is the HP Unix Superdome server just as I mentioned. HP have just recompiled Linux onto the HP Unix server. This Superdome server uses nPar, that is, partitioned into virtual servers consisting of 4 cpus and RAM, or 6 cpus, etc. Thus, you can carve up several small servers and run different OSes on this Superdome server. For instance, you can run Windows in one nPar, and at the same time run Linux in another nPar. Now, this Big Tux server you talk about, is Linux running in one nPar. This Linux nPar can at most have 16 cpus if you cluster two nPars 8 cpus.

                Thus, you can not run Linux on this server and use 64 cpus. The biggest server you can run Linux on, is in nPar with a clustered 8-cpu nPar. The Linux installation on this Big Tux server is at most 16 cpus (using a cluster).




                It is the same document, I think. At the bottom of page 5 and top of page 6, it says
                "64 Processors / 128 Cores
                Maximum nPars 16 (if you use two 8-nPar clusters)"

                So again, there are no big SMP Linux servers on the market. Sure, you can take Linux and recompile it on a big Unix SMP server from Solaris, IBM or HP. But there are no big SMP Linux servers. And, frankly, I suspect that Linux running on 8 cpu nPar have problems using all 8 cpus well. Because this is a SMP server, and general purpose. As we have seen, Linux excels on doing only one simple task: calculations. But as soon as there are more complex work loads, then Linux has problems.

                Thus Jeff Bonwick was right; Linux scales bad on SMP servers. But everyone agrees that Linux scales excellent on HPC servers.




                You FUD and lie like it was proven.
                You have many times said that I FUD. Prove it, or it is you that FUD about me.





                I don't reject this test, because ZFS used many disk, but because it was tested against unstable btrfs.
                You reject this test. In this test 16 SSD disks are used, and ZFS is faster than BTRFS. You say it is not relevant because BTRFS is unstable. You ARE rejecting this test, dont you understand?
                "I dont reject this test, but this test is not relevant because BTRFS is unstable" - this is a rejection. Dont you understand what you are saying?
                You also accepted test of unstable OpenSolaris vs Linux, in Phoronix benchmarks.

                When Linux is unstable, you reject the tests. When Solaris is unstable, the test is good and fair. What is the matter with you Kraftman. Are you serious, or are you joking? You also compared several times 3x faster Linux servers to old Solaris servers? Kraftman? What is going on? Have you been joking all the time?




                I won't stop replying to your FUD. Will I get a price, too? Kebb, what do you think? Is there anyone so generous as sun was? ;>
                If I FUD, then quote me. Cite a post where I FUD and disprove my post by a link. If you can not, you are just
                FUD is ... negative and dubious/false information ... to undermine the credibility
                about me, dont you agree?

                But I will tell you Kraftman, you have chance to prove you are not FUDing now. Prove every negative claim you said about Solaris and about me, and I will agree you are not FUDing. I have always showed links to every criticism on Linux I have cited from Linux kernel devs. I have not made up anything, I have not written "false" information - everything is true. Linus Torvalds did say Linux is bloated, I am not lying on this. Everything I said, I can show links to. Thus, I am not FUDing. So, go ahead Kraftman, prove that you are correct, and prove that I have been lying and FUDing - disprove me. Go ahead, show links.

                Regarding a fine price, I dont know. If you call a dinner a fine price, then go ahead. But you have 4 questions to answer now. Question 1) 2) 3) and 4). I am waiting for your answers, Kraftman.

                Comment


                • #9
                  Originally posted by TheOrqwithVagrant View Post
                  Lots of interesting stuff.

                  1. The SGI Altix UV and Altix 4000/3000 systems are NOT distributed memory systems, they are shared memory systems. They are the same "class" of multiprocessor servers as what you (Kebabbert) would refer to as "large enterprise SMP systems", such as the HP SuperDome, Oracle T & M-series enterprise servers, etc.
                  Ok, this is interesting. Forgive me for expressing this, but I am just used to debate to Kraftman, and as you have seen, those "debates" tend to be quite... strange. I am not used to debate with sane Linux supporters. I have some questions to your interesting and informative post:

                  1) You talk about the SGI Altix systems, and call them SMP servers. Why dont people run typical SMP work loads on them, in that case, such as databases? They mostly run HPC workloads.

                  2) Why does lot of links refer to the Altix systems as HPC servers?

                  3) I read a link on this SGI Altix servers, and the engineers on SGI said "it was easier to construct the hardware than to convince Linus Torvalds to allow Linux to scale to Altix 2048 cores". If it is that easy to construct a true SMP server - is this not strange? I know it is easy to construct a cluster. But to construct a SMP server in less time it takes to convince Linus Torvalds?!

                  4) I suspect the Altix systems being cheaper than a IBM P795 AIX server. Why dont people buy a cheap 2048 core Altix server to run SMP workloads, instead of bying an 32cpu IBM P795 server to run SMP workloads? The old 32 cpu IBM P595 server for the old TPC-C record costed 35 million USD, list price. What does the Altix servers cost?

                  5) Why dont IBM, Oracle and HP just insert 16.384 cpus into their SMP servers, if it is so easy as you claim to build large SMP servers? Why are all of them still stuck at 32-64 cpus? SGI has gone into 1000s of cores, but everyone else with their Mature Enterprise Unix systems are stuck at 64 cpus. Why is that?
                  Last edited by kebabbert; 15 November 2011, 08:49 AM.

                  Comment


                  • #10
                    Holy Cow! ANOTHER sane Linux supporter. I did not even knew they existed! I dont know if you have seen the usual Linux supporters here, but the debates with them.... are quite strange.

                    You are the fourth person I debate with, so this might take time as I have work to do. I will use Round Robin, or I should just drop the FUDers and talk only with the sane Linux persons.



                    Originally posted by jabl View Post
                    No, this is wrong. It's not about "SMP servers" vs. "HPC servers". It's more about which kind of workloads the OS has been tuned for. The Linux VM subsystem has seen extensive scalability work in order to run (mostly) HPC style workloads on gigantic CC-NUMA machines. OTOH, database workloads are very different, and there hasn't been as much interest in scalability work that would benefit these kinds of workloads (although not too long ago a big bunch of "VFS scalability" work was integrated into the mainline kernel, so this situation is changing). For various reasons, the proprietary Unix vendors have had different priorities.
                    This is strange. As Ted Tso said, Linux kernel devs had not access to big SMP servers. It could be interpreted as the reason as Linux is not targeting SMP servers.

                    You say that there are not much interest in SMP workloads such as databases. That is strange. RedHat, Suse, Novell etc all of them wants to make money. The big money is in Enterprise workloads, such as Databases. We all know that Oracle made a fortune in databases.

                    Thus, I am not convinced in your explanation "there have not been much interest". My interpretation is that Linux kernel devs can not handle that kind of work load, they dont have the servers nor experience of building such servers. Oracle, IBM and HP has.

                    To build a cluster is easy and Linux kernel devs can develop for them. But not for SMP servers costing many millions of USD.

                    Comment

                    Working...
                    X