SysVinit 3.11 Released With An "Important Feature" At Long Last

[wertigon]

I give you a hint: Altix systems, 2004/2005, kernel 2.4 or 2.6, not x86 ...

Wait, so your gripe is that systemd does not support 20 year old legacy systems?

First, you do know "HPC" stands for "High Performance Computing" right? The i3 14100 based system I built for my dad is not only running circles around those ancient systems, it is doing it while cartwheeling and juggling fire. Those ancient systems were good for their time but today they are basically e-waste.

These days 16 cores / 32 threads is a bare minimum threshold for HPC. Even the last gen Raspberry Pi 4 have 8 threads to play around with.

There was a time when systemd critique was justified, but these days it just works and excels at most tasks. And that is why it is slowly becoming as embedded into the Linux world as it is. Because it is just working.

If you want to run a Linux system on an Arduino or FPGA system with < 64 MB flash storage, then yes, sure, systemd is not a good fit for that (although, one could argue, neither is Linux - go check out FreeRTOS). For all other use cases, it is just amazing compared to the competition.

[access]

I give you a hint: Altix systems, 2004/2005, kernel 2.4 or 2.6, not x86 ...

Sir, this is 2024. We have better tools now.

[tobias]

Your process supervisor does not need to run as root. Obviously, if it is supervising something that runs as root and that thing does not use capabilities(7) then it needs to be ran as root and the same security issues that apply to PID1 apply to the supervisor too, but in the common case you can run the supervisor unprivileged.

A service supervisor can by definition manage services on the system. That makes it a security critical task, independent of what user it runs as.

[ahrs]

And you said before that PID2 may not crash and it obviously also needs to be secure as it also runs as root. So that is the same requirements you want for PID1.

Your process supervisor does not need to run as root. Obviously, if it is supervising something that runs as root and that thing does not use capabilities(7) then it needs to be ran as root and the same security issues that apply to PID1 apply to the supervisor too, but in the common case you can run the supervisor unprivileged.

[Akiko]

I hope you are aware of the PREEMPT_RT, nice, cpulimit and cgroup tools/technology that, combined, nullifies like 99.99999% of your issues?

Even then, in HPC you have like, what? 32 threads or more to work with? Is it so bad lending one of those threads once every 100th hour for system maintenance work? If you need a consistent throughput, why not simply lock system work to thread #0 and #1 always and use the other 30 threads for workloads?

I give you a hint: Altix systems, 2004/2005, kernel 2.4 or 2.6, not x86 ...

It smells like this is a skill issue or a badly configured system issue, not a problem with the technology itself. I have had PREEMPT_RT systems in soft realtime work for months without a single missed deadline, and systemd was part of that setup, so...

To me it looks like a reading comprehension/"search engine usage" issue...

[tobias]

It doesn't complicate anything. The two are functionally identical. As long as both result in the process getting restarted after failure then does it really matter?

You have one more process, (a bit of) code that runs in that process, the build system to build that extra binary, the installer that copies the extra binary, ... . Why have any of that when -- as you say -- the functionality is identical?

On stability and security, you could argue that taking process supervision out of PID1 simplifies things

Thats my point: How is it simpler to have a process over *not* having that process?

and makes it easier to assert that the system is secure (a flaw in the process supervisor can't result in the highly privileged PID1 - which is running as root with the keys to your entire system - being compromised).

It makes it easier to assert that PID1 is secure. Its always easy to argue over the security of a process that does nothing productive for the system.

It does help in no way to argue that PID2 is secure and probably makes arguing over the security of PID1 + PID2 a tiny bit harder. And you said before that PID2 may not crash and it obviously also needs to be secure as it also runs as root. So that is the same requirements you want for PID1.

[wertigon]

Okay, so let me tell some of my experience as a developer who worked at one of the biggest Linux distributors building professional distributions for (at that time) unique systems. In my office back then I was surrounded by a AlphaSever DS20 and DS20E, HP C3750 and about 2 years ago I killed my HP C8000. I was surrounded by AMD Sledgehammer engineering samples and UltraSparce systems. I had remote access to SGI Altix systems with 512 and 1024 CPUs (Itanium) to investigate and fix bugs. We had a bug where the customer reported this: SGI Altix 1024 CPUs, 1024 GiB RAM, runs HPC application over weeks, highly tuned to run with exactly 1024 threads, but sometimes a CPU runs two threads and another runs nothing, the HPC runtime increased about 25% because of this ... please fix this scheduling issue, because CPU time is expensive.

To make it short: On HPC system doing serious work, running for weeks or months, you really want a predictable runtime behavior. Having daemons running on the system, which start to do some work like deleting or swapping out journals and eating into your IO or CPU time can become a nightmare. This is the reason why running systemd on a HPC system may be a bad idea. And yes, I know this is an extreme example. But I want to demonstrate you, that I do not throw bullshit around. I tell this, because I encountered these issues. I really try to see the good and the bad.

I hope you are aware of the PREEMPT_RT, nice, cpulimit and cgroup tools/technology that, combined, nullifies like 99.99999% of your issues?

Even then, in HPC you have like, what? 32 threads or more to work with? Is it so bad lending one of those threads once every 100th hour for system maintenance work? If you need a consistent throughput, why not simply lock system work to thread #0 and #1 always and use the other 30 threads for workloads?

It smells like this is a skill issue or a badly configured system issue, not a problem with the technology itself. I have had PREEMPT_RT systems in soft realtime work for months without a single missed deadline, and systemd was part of that setup, so...

[Akiko]

The only thing you _really_ require is udev — and if you're in the "removing udev" territory, then it's just not a target use-case for systemd. Roll your own minimal init as you've done before and be done with it.

You don't really read the thread, don't you? I already explained because of systemd becoming a standard that other software dropped init scripts (you now have to rewrite) and even introduced a dependency to systemd (getting this removed is even harder). You need to do more customization.

If you're such a big boy doing serious work on serious HPCs,

Oh, so going for the persona now? (I will do this, too, once.) You did not really read my post. I clearly stated that I had a customer who did serious HPC work. Why do you twist my words?

then you really ought to know how to stop unneeded dæmons, isolate your CPUs and pin your threads. And you probably should know that having a bunch of processes that mostly do nothing is not supposed to impact scheduling of CPU hogs in any way — or if it does, then you're in trouble anyway, because guess what, kthreads exist and most of them are unbound.

Yeah, I see your problem. You got the "I have a hammer and everything looks like a nail" problem, well, in that case "I'm an admin and now I can fix everything by configuration/administration". See, I gave you an example and you should have looked up what an Altix system is, when it was used, what kernels where used at that time by professional distributions (we talk about certifications here which take months and a change of single software would nullify the certification), and then look up what was possible in these kernel versions. I know that today you have a lot more features you can use.

I'm a shitty hobbyist admin

Oh boy, and it shows ... (see, once, like promised up there, and did you notice how I twisted your words? You like to do that a lot. You seem to pick what perfectly helps your argument and ignore everything else)

(BTW, most of the times, "unpredictable behavior" is shorthand for "I have no clue how to predict behavior". This is one of them times.)

Do you want to fight and get personal or do you actually want to learn something? If the later, just dig into the rabbit-hole "what the introduction of CPU caches, out of order execution and speculative execution means for predictable behavior". And I will ignore you now until you show some decent human behavior.

[ahrs]

No, because you have one process whose *only* job it is to another process once that has all functionality and very similar requirements on stability and security as the first. That first job just complicates the pstree as well as the overall system.

It doesn't complicate anything. The two are functionally identical. As long as both result in the process getting restarted after failure then does it really matter?

On stability and security, you could argue that taking process supervision out of PID1 simplifies things and makes it easier to assert that the system is secure (a flaw in the process supervisor can't result in the highly privileged PID1 - which is running as root with the keys to your entire system - being compromised).

[tobias]

It is a design decision, yes. Why does it make no sense today? Because Systemd does it that way?

No, because you have one process whose *only* job it is to run another process once that has all functionality and very similar requirements on stability and security as the first. That first job just complicates the pstree as well as the overall system.