Apple Is Looking For Linux Kernel Developers

[Guest]

Can you link to any such Linux high core count server you talk about? Because the last time I checked, no high core count servers running Linux was offered on the entire market. No single vendor sold such systems. Maybe it has changed in the recent year, but until 1-2 years ago, there were no Linux servers with high core count. All high core count servers are running RISC and Unix (Solaris/AIX) or Mainframes. Linux servers are exclusively low core count, with 8 sockets or less. Can you show a 32-socket or 64-socket linux server? Unix and Mainframes have run those large socket configurations for decades. Linux has not. It takes decades to grow a kernel that scales to 8 sockets or more. It does not take one year.

Ted Tso, the famous Linux kernel developer wrote on his blog post that until recently, 24-32 cores (not cpus) where considered as exotic hardware that no Linux dev had access to. So, how can Linux scale well on 24 cores, when those high core count servers does not exist? Only a naive desktop user would consider 32 cores as high core count. SPARC M6 is a high core count servers with 384 cores with 32TB RAM.

Kebbabert troll is back. However, he was proven wrong many times. What's worth to note slowlaris is dead and unix is dead as well when comes to big irons. Furthermore, scale up is only for lower performance, scalability demands. Its very limited expansion, scalability and vendor lock in makes it a stone age solution. It will never achieve scalability and performance of scale out servers. Nobody sane would invest in scale up today. It's one of the reasons why aged slowlaris is dead. Oh, and scale out is much harder from operating system stand point, so your bullshit about slowlaris being more mature in scalability is nothing, but wishful thinking.

[Guest]

In a scale UP environment, administrators can scale just capacity. The downside here is that it becomes more likely that the existing processing and network connectivity of the head end could be overwhelmed as more and more is done with the array. In the scale out environment, capacity, compute and network connectivity are scaled equally, so performance should remain linear even as more units are added.

In many cases, the scale up environment is suitable for SMB and low end mid market and often carries a lower total price tag. Scale out, on the other hand, is a bit more expensive and carries with it some additional complexity as the solution needs to be able to continue to scale as a single whole, even as additional devices with their own compute stacks are added.

So, kebbabert why are you lying?

[unixfan2001]

Wait! So pavlerson isn't actually Guest who got their account name changed?
Do you have a stalker, Pawlerson? Lol

[Guest]

Wait! So pavlerson isn't actually Guest who got their account name changed?
Do you have a stalker, Pawlerson? Lol

PaVlerson is actually known troll and slowlaris fanboy kebabbert.

[pavlerson]

Wait! So pavlerson isn't actually Guest who got their account name changed?
Do you have a stalker, Pawlerson? Lol

Yes, I am a big admirer of Pawlerson! He is so tech savvy and knows how to spell to Linux with one hand only. And he knows how to type B-A-S-I-C with the other hand at the same time. He is the besto.

[Guest]

Yes, I am a big admirer of Pawlerson! He is so tech savvy and knows how to spell to Linux with one hand only. And he knows how to type B-A-S-I-C with the other hand at the same time. He is the besto.

You can now eat your shit kebbabert:

You	Well, there will be quite a bit of scalability work needed. For example, you will receive a scalability bug report involving a 512-CPU shared-mmeory system.
Me	Hmmm... It took Sequent from 1985 to 1997 to get from 30 to 64 CPUs, so that is doubling every 12 years, so I am guessing that I received this bug report somewhere near the year 2019. So what did I do in the meantime?
You	No, you will receive this bug report in 2004.
Me	512-CPU system in 2004??? Well, suspending disbelief, this must be why I will start maintaining RCU in 2005.
You	No, a quick fix will be supplied by a guy named Manfred Spraul, who writes concurrent Linux-kernel code as a hobby. So you didn't do the scalability work until 2008.
Me	Concurrent Linux-kernel coding as a hobby? That sounds unlikely. But never mind. So what did I do between 2005 and 2008? Surely it didn't take me three years to create a highly scalable RCU implementation!
You	You will work with a large group of people adding real-time capabilities to the Linux kernel. You will create an RCU implementation that allowed readers to be preempted.
Me	That makes absolutely no sense! A context switch is a quiescent state, so preempting an RCU read-side critical section would result in a too-short grace period. That most certainly isn't going to help anything, given that a crashed kernel isn't going to offer much in the way of real-time response!
You	I don't know the details, but you will make it work. And this work will be absolutely necessary for the Linux kernel to achieve 20-microsecod interrupt and scheduling latencies.
Me	Given that this is a general-purpose OS, you obviously meant 20 milliseconds!!! But what could RCU possibly be doing that would contribute significantly to a 20-millisecond interrupt/scheduling delay???
You	No, I really did mean sub-20-microsecond latencies. By 2010 or so, even vanilla non-realtime Linux kernel will easily meet 20-millisecond latencies, assuming the hardware and software is properly configured.
Me	Ah, got it! CPU core clock rates should be somewhere around 50GHz by 2010, which might well make those sorts of latencies achievable.
You	No, power-consumption and heat-dissipation constraints will cap CPU core clock frequencies at about 5GHz in 2003. Most systems will run in the 1-3GHz range even as late as in 2017.
Me	Then I don't see how a general-purpose OS could possibly achieve sub-20-microsecond latencies, even on a single-CPU system, which wouldn't have all that much use for RCU.
You	No, this will be on SMP systemss. In fact, in 2012, you will receive a bug report complaining of excessively long 200-microsecond latencies on a system running 4096 CPUs.
Me	Come on! I believe that Amdahl's Law has something to say about lock contention on such large systems, which would rule out reasonable latencies, let alone 200-microsecond latencies! And there would be horrible reliability problems with that many CPUs! You wouldn't be able to keep the system running long enough to measure the latency!!!
You	Hey, I am just telling you what will happen.
Me	OK, so after I get RCU to handle insane scalability and real-time response, there cannot be anything left to do, right?

https://paulmck.livejournal.com/46439.html

[name99]

They won't drop Mach. The entire dynamic messaging system is at the core of their frameworks, even with Swift.

What's true is they won't drop the capabilities Mach offers them over traditional UNIX, precisely because those capabilities are used throughout macOS and iOS.
But they may well switch to a more modern OS that provides the same capabilities and more. The question is whether more modern OSs (which are even more mach-like in terms of a small micro-kernel and building everything else in terms of communication primitives) are performant enough compared to mach. I don't think Apple would care about losing 10% on some micro-benchmarks for the sake of something that scales a lot better, has a much smaller attack surface, and is much more maintainable, but 50% loss might be unacceptable.

Remember Apple has familiarity with L4 (seL4 runs on the Secure Enclave), so that's one contender. Another contender would be something like Barrelfish which was designed to take advantage of manycore even though targeting personal machines.

[pavlerson]

https://paulmck.livejournal.com/46439.html

Hahaha! That man is deluded. He talks about a large server which uses RCU having 200 microsecond latency. 200 microsec latency is the same thing as a 0.005 MHz cpu. So he basically says that with RCU you get insane performance - what kind of insane performance? Well, a 0.005 MHz cpu! Hahaha! He is lost. He will never ever see a 0.005MHz server on the top SAP benchmark list, I promise you that.

[Guest]

Hahaha! That man is deluded. He talks about a large server which uses RCU having 200 microsecond latency. 200 microsec latency is the same thing as a 0.005 MHz cpu. So he basically says that with RCU you get insane performance - what kind of insane performance? Well, a 0.005 MHz cpu! Hahaha! He is lost. He will never ever see a 0.005MHz server on the top SAP benchmark list, I promise you that.

Kebabbert, it seems you're ate your own crap already. He was probably talking about scheduling and interrupt latency. Bellow is type of server he was talking about (which uses Xeon X5560 4C 2.8GHz):

https://www.top500.org/system/176326

You won't see this in SAP, because there are only low end systems in comparison to above one.

Ps. Do you really think one of the main RCU inventors doesn't know what he's talking about?

I have worked with real-time operating-system kernel synchronization mechanisms (e.g., realtime RCU in Linux), SMP/NUMA scalability and performance in Linux and UNIX operating-system kernels, performance analysis, routing and congestion control in networking, embedded realtime applications. I am the maintainer of RCU and of the rcutorture test module in the Linux kernel. I have more than 200 publications and hold more than 100 patents. I am editor of "Is Parallel Programming Hard, And, If So, What Can You Do About It?"

Specialties: Performance programming for multi-threaded and multicore systems, including NUMA architectures. Free/open-source software projects.

One more thing: why don't you tell us SAP results are meaningless?

[pavlerson]

Kebabbert, it seems you're ate your own crap already. He was probably talking about scheduling and interrupt latency. Bellow is type of server he was talking about (which uses Xeon X5560 4C 2.8GHz):

https://www.top500.org/system/176326

Ps. Do you really think one of the main RCU inventors doesn't know what he's talking about?

No, he does not know what he talks about. Linux developers said that "Linux scales better than Solaris because Linux scales to 4096 cores and the largest Solaris server goes to 32-cpus". This 4096 core server is the same machine you linked to, as this deluded guy worked on. Linux developers typically use desktop PCs with 2-4 cpus and know nothing about the difference between scale-out and scale-up. A 4096 server is scale-out cluster and cannot be used to run scale-up workloads.

If you try to use a 200 microsecond latency server, the performance equals 0.005 MHz. But that is not a problem when you run clustered workloads, because the nodes run one small for loop over and over again on a small data set. Everything fits into the cpu cache. You never reschedule nor need to go out to RAM. So the 200 microsecond latency is never triggered for scale-out workloads. But scale-up workloads always go out to RAM and communicate with other nodes so 200 microsecond latency would never work for SAP or databases or other business workloads. Because SAP runs at 0.005 MHz on a cluster such as the Altix server you linked to.

Again, that guy have only worked on HPC clusters where 200 microsecond is not a problem. He have never touched Big Iron and knows nothing about scale-up workloads which maxes out at 32-cpus. That is clear because he talks about 200 microsecond latencies, which is the same thing as 0.005 MHz cpu.

You won't see this in SAP, because there are only low end systems in comparison to above one.
One more thing: why don't you tell us SAP results are meaningless?

Well, one single scale-up business server that run database workloads, with as many as 32-cpus costed $35 million. No typo. IBM sold their P595 for $35 million for database work. One SAP installation can cost several $100s millions. So business servers are highly profitable and everybody wants to go into business server market which is exclusively POWER, SPARC and Mainframes - all are much more expensive than clusters.

Clusters with 256 nodes are basically 256 PCs sitting on a fast switch. 256 PCs are much cheaper than $35 million. A cluster with 100s of cpus are dirt cheap and can never cost $35 million. Why would SGI choose to sell 256 cpus for small money, if they easily could sell 32-cpus for $35 million? The reason SAP is dominated by 32-cpu servers and no large clusters with 10.000s of cores is not because cluster vendors dont want to easily earn millions, but because clusters running SAP gives 0.005MHz at best. That is why no clusters dominate SAP benchmarks.

But this getting tiresome, I have explained this to you many times. And you dont know anything about comp sci nor programming, so why "discuss" with you? You also earlier confessed you are trolling. I dont want to waste time on Trolls that dont know anything about comp sci. If you think that 0.005 MHz servers are fantastic, so good for you. Be happy with your large clusters that have bad latency so you get 0.005MHz server and can never run SAP nor databases nor any other business workloads. There is a reason there are no large clusters in no business benchmarks such as SAP, TPC-C, etc. Bye Troll.