The Linux Kernel's Scheduler Apparently Causing Issues For Google Stadia Game Developers

[Almindor]

Well, don't want to sound like a broken record, but Shadow of the Tomb Raider port by Feral heavily utilizes multiple threads which are dependent on each other.
When I force low resolution and GPU effects to be entirely CPU bound in an NPC crowd (i.e. CPU never waits for GPU), I get 90% total CPU usage reported on the 6700k (four cores + HTT, so eight threads almost fully utilized).
Windows DX12: ~102fps
Linux Vulkan port: ~94fps
Also frametime consistency is absolutely comparable, no noticeable spikes/stalls (the game has some mostly minor stalling issue when loading certain new areas, happens on Windows and Linux the same).

So, how bad can the issue actually be if the developer knows what he/she's doing?

Linux literally "locks up" during heavy thread activity. It's not just spinlocks it's crappy scheduler. It used to be the other way around btw. windows was unusable under load. Even old Solaris was better than today's linux is under load it's pretty terribad.

I'm glad some people are finally looking into the finer details with proper testing that can't be handwaived away.

[aufkrawall]

What is "heavy load"? When I run x264 on all threads, Windows runs pretty sucky too. I'd say even worse.

[Linuxxx]

Michael
Any ETA on integrating the aforementioned kernel benchmark posted by the blog post author into the Phoronix Test Suite?

[Michael]

Michael
Any ETA on integrating the aforementioned kernel benchmark posted by the blog post author into the Phoronix Test Suite?

Already should be available since the other day via `phoronix-test-suite benchmark mutex`

I added like 10 of the different sub-tests for now, there are dozens so tried to just add some of the most relevant ones but any others can be trivially added.

https://openbenchmarking.org/test/pts/mutex

[polarathene]

I figure robotics applications would benefit from a more optimized scheduler.

Robotics tend to run an RTOS(Real-time OS) which just focuses on it's purpose, rather than a GPOS(General Purpose OS), which we use to run various desktop programs/GUIs.

Or at a lower level, robotics can be a series of microcontrollers, which may be co-ordinated through a central CPU running linux, but their actual logic is deterministic and hard real-time beyond that for what they do on the microcontrollers(nothing else can fight for resources as it's more tailored to a specific purpose(s).

[Volta]

Linux literally "locks up" during heavy thread activity. It's not just spinlocks it's crappy scheduler. It used to be the other way around btw. windows was unusable under load. Even old Solaris was better than today's linux is under load it's pretty terribad.

I'm glad some people are finally looking into the finer details with proper testing that can't be handwaived away.

Maybe sometimes under I/O load, but It's something else. Did they make a bug report and contacted Kernel developers? It seems someone is whinning without clue on a blog.

[Volta]

Quote from a Gnome developer:

Just for completeness: do you mind sharing which exact distribution and desktop environment you are running? (Gnome Shell dev here, and we know that we have some issues on this front as well that might contribute to bad results)

Now I want to see this under KDE.

[PuckPoltergeist]

Linux literally "locks up" during heavy thread activity. It's not just spinlocks it's crappy scheduler. It used to be the other way around btw. windows was unusable under load. Even old Solaris was better than today's linux is under load it's pretty terribad.

I'm glad some people are finally looking into the finer details with proper testing that can't be handwaived away.

What's "locks up" and "heavy thread activity"? I'm running a compile job with make -j20, listening to music and writing this answer at moment. Everything works smooth. So how is the scheduler crappy here?

[MadCatX]

Quote from a Gnome developer:



Now I want to see this under KDE.

The remark about stuttering video playback raised my eyebrows too. I thought this problem has been solved years ago with cgroups and I haven't seen it myself for a very long time. I can easily a watch videos and listen to music while compiling code on all CPU cores. There must be something else going on here.

[indepe]

I'm not sure about his concept of measuring idle time. The only reason a thread wouldn't lock a spinlock or spinlock_amd that is available, is if that thread is not actually running. It's not weird, it just isn't running. And among the 16 threads, at least one should be running.

So the "idle time" he is measuring would seem to be something very different: the thread already holding the lock getting preempted at the point in time between taking or releasing the lock, and reading the TSC time value.

And except for that, I'm not sure if there is anything significant different between his Windows and Linux measurements.