Announcement

Collapse
No announcement yet.

Is The Linux Kernel Scheduler Worse Than People Realize?

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • #61
    Just as a side note, we run Oracle on about 1000 linux servers, all using RAC, and the *MINIMUM* RAM size is 128 GB. We know if you page, you are dead.

    Comment


    • #62
      Originally posted by dweigert View Post
      Just as a side note, we run Oracle on about 1000 linux servers, all using RAC, and the *MINIMUM* RAM size is 128 GB. We know if you page, you are dead.
      How do you mean? Can you elaborate? Are you confirming that Linux is unstable under high loads?

      Comment


      • #63
        Originally posted by dweigert View Post
        Just as a side note, we run Oracle on about 1000 linux servers, all using RAC, and the *MINIMUM* RAM size is 128 GB. We know if you page, you are dead.
        I don't have anything on the scale you do, but personally I've come to prefer a kernel panic over paging, recovery time is better. imo

        Comment


        • #64
          Originally posted by duby229 View Post

          That may be true in absolute terms, but 99.9% of people will never notice the difference between a preemptable kernel and a stock kernel as long as there is plenty of free RAM. A lot of people believe that if you have free RAM you should do something to fill it up, but really that's bullshit. Having available RAM is very, very important for highly responsive systems. (Only people with very bloated low RAM systems will notice it)

          The attitude because it's there it should be full is total nonsense.
          That's a strong statement and I think that you should back it up. What speed difference do you see between allocating from free memory and allocating from file cache memory? AFAIK file cache can be freed and reassigned just as fast as free memory. The only slowdown is memory zeroing. And memory can be zeroed at many gigabytes per second, so not exactly a bottleneck.

          If you want responsive then you *really want* your applications, data files, fonts, icons, etc, all in file cache and not sitting out there on disk.

          Comment


          • #65
            Originally posted by kebabbert View Post
            Also, among Unix and Mainframe sysadmins, Linux have always had a bad reputation of being unstable. During light loads, Linux is stable. Even Windows is stable if the server idles. But under very high load, Linux becomes very jerky and stuttery and some threads finish fast, other threads takes a very long time. Add in the "Ram overcommit syndrome" where Linux randomly kills processes when all of RAM is filled up (imagine Linux killing of the Oracle database process!!!) and it is understandable why Unix sysadmins would never let Linux into their high end Enterprise server halls. It is the same reason that Enterprise companies who use Linux, ALWAYS make sure Linux servers are lightly to medium loaded. They know that if load increases much, there is a high probability that Linux becomes unstable.
            Since the place I work has been running PostgreSQL on Linux for lots of years, and often run multi-hour queries that use gigabytes of RAM and 800% CPU, I know a bit about it.

            Linux has never failed us.

            And if those "Unix and Mainframe sysadmins" went out and read a book about Linux administration, then they'd know about all of the tuneables for disk IO, virtual memory, swap, etc. And they'd know that if they would like to disable the OOM and run with strict memory commit, the knobs to do that are right in front of them.

            Running a database server like PostgreSQL or Oracle is not something you just apt-get and go. You read the documentation, best practices, etc. You tune the database configuration to the system, and you set everything it needs in the kernel to the right values.

            If the system falls over because it can't grab the right number of huge pages or shared memory or whatever, that's the sysadmin's fault for not configuring it properly.

            Comment


            • #66
              Originally posted by Zan Lynx View Post

              That's a strong statement and I think that you should back it up. What speed difference do you see between allocating from free memory and allocating from file cache memory? AFAIK file cache can be freed and reassigned just as fast as free memory. The only slowdown is memory zeroing. And memory can be zeroed at many gigabytes per second, so not exactly a bottleneck.

              If you want responsive then you *really want* your applications, data files, fonts, icons, etc, all in file cache and not sitting out there on disk.
              The file cache is completely transparent, I agree with you. But as you can obviously read, we were talking about loading files from disk.

              Comment


              • #67
                Originally posted by Zan Lynx View Post

                Since the place I work has been running PostgreSQL on Linux for lots of years, and often run multi-hour queries that use gigabytes of RAM and 800% CPU, I know a bit about it.

                Linux has never failed us.

                And if those "Unix and Mainframe sysadmins" went out and read a book about Linux administration, then they'd know about all of the tuneables for disk IO, virtual memory, swap, etc. And they'd know that if they would like to disable the OOM and run with strict memory commit, the knobs to do that are right in front of them.

                Running a database server like PostgreSQL or Oracle is not something you just apt-get and go. You read the documentation, best practices, etc. You tune the database configuration to the system, and you set everything it needs in the kernel to the right values.

                If the system falls over because it can't grab the right number of huge pages or shared memory or whatever, that's the sysadmin's fault for not configuring it properly.
                Well, as I tried to explain earlier, those Unix/Mainframe sysadmins are using LARGE enterprise servers. With HUGE loads. Linux and Windows people fail to realize what a huge workload is. They sit with low to medium workloads and believes Linux is stable because they dont see any problems on their tiny 4-socket or 8-socket servers with small workloads. The largest Linux business server until a couple of months ago was 8-socket servers with a few 100 GB of RAM. That is low - medium end and not high end.

                High end business servers are exclusively running Unix or z/OS. They have 16 or 32 sockets, and several Terabytes of RAM. And they serve 1000s of clients, 10.000 clients. And int that area, Linux does not do. First of all, the largest Linux server (until recently) is 8-sockets. So no one has ever run a large database or a large SAP install on a Linux server, as Linux scales up to 8-sockets and not above (until recently). And everything below 8-sockets are low-medium not high end.

                I tried to explain this, but apparently failed. For LIGHT workloads, Linux is fine. But not for high end and huge workloads - then Linux becomes unstable and can not even run huge workloads because the hardware does not exist. Just recently a 16-socket Linux server has been released, it is a HP Kraken (a redesigned Unix server) to use x86 cpus. I expect it to be immature and have lot of problems.

                Again: Just look at the SAP benchmarks. You will see no Linux nor Windows at the top. The top spot are all Unix, because Unix has scaled to 32 or 64 sockets for decades and is mature. Linux has low SAP scores because it only scales to 8 sockets. To reach the highest SAP scores, you need go to 16 or 32 sockets. Fujitsu have a 64-socket Unix server, called M10-4S. It is the largest server on the market targeted to business scale-up workloads. The SGI UV2000 is a cluster, and can only run scale-out clustered workloads.

                Linux has problems tackling workloads running on 8-sockets and above. The scheduler starves some threads, and other threads get all attention. Also, RAM usage is not optimal. Linux on 1 socket runs fine. But many threads becomes a problem. For instance, there are Solaris servers with 4.192 threads. Linux would suffer bad on such a beast.
                Last edited by kebabbert; 04-22-2016, 01:26 PM.

                Comment


                • #68
                  Originally posted by kebabbert View Post
                  High end business servers are exclusively running Unix or z/OS.
                  99.99999% of statistics are made up on the spot. If you're going to make such a claim, please, provide us with some citation.

                  Comment


                  • #69
                    Originally posted by F1esDgSdUTYpm0iy View Post
                    99.99999% of statistics are made up on the spot. If you're going to make such a claim, please, provide us with some citation.
                    Again, ALL high end business servers are exclusively running Unix or z/OS. I talk about 16 or 32 socket servers with Terabytes of RAM. There did not exist any such large Linux servers until just a couple of months ago - HP Kraken is a redesigned 16-socket version of their Unix 64-socket server. But these Enterprise users, take very long time to evaluate an Enterprise server. Should they really migrate from their very expensive 32- or 64-socket proven Unix server to a cheap 16-socket Linux HP Kraken server? Put their entire database that drives their entire business at risk? Their business juggles billions of dollars. Is it worth saving some money to migrate to a cheap Linux server? What do you think? Do you think they will do it in a couple of weeks? If they really want to migrate away from Unix or z/OS mainframes, the companies need to evaluate the new HP Kraken for at least a year, before risking their business of billions of dollars. Do performance tests, stability tests, etc. And Linux is not even stable under high load when going above 8-sockets. Why would they?

                    I invite you to post any link to a customer that have migrated away from a Unix server to a brand new Linux server with 16-sockets. You will not find any such links. Why? The first 16-sockets Linux servers just hit the market some months ago, and for Enterprise usage, it takes years before they consider migrating to cheap 16-socket Linux servers. Except the HP Kraken and SGI UV300H and Bullion - I dont know of any other Linux servers with 16-sockets. And if you look at the benchmarks for these servers, they lag far behind Unix. Linux does not scale well to 16-sockets. For instance, HP tried earlier to compile Linux to their 64-socket Unix server - called Big Tux. And the Big Tux had 40% cpu utilization under full load. Under maximum load, every other cpu were idle. Linux could not handle 64-socket back then, and can not today. Because the Linux kernel developers do not have access to large servers. Why? They did not exist until recently. Linux are for desktop loads, and light server loads with 1-2 sockets or so. Because that is what the Linux kernel developers have experience of, and what they target.

                    Comment


                    • #70
                      Originally posted by kebabbert View Post
                      Again, ALL high end business servers are exclusively running Unix or z/OS...<incoherent rant>...I invite you to post any link
                      The burden of proof is on you, sir. First, re-read what I wrote. Did I discount your claim? No, I did not. I just asked for citation, nothing more. For proof of your claim.

                      So, the next time, before you go on a (repeated) rant-binge, please observe the very vital skill of comprehensive reading. Do not merely read the words one by one. Read the message, understand it. What does it say? What did I say? Did I say -- "No, you're wrong!". No, I did not. Did I imply it? Not the point. I asked for proof, nothing more. You made a claim, the burden of proof most decidedly is on you.

                      Comment

                      Working...
                      X