Announcement

Collapse
No announcement yet.

BUS1 Working On "r-linux" - A Rust Capability-Based Linux Runtime

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • oiaohm
    replied
    Originally posted by sinepgib View Post
    Again with assuming I am drawing a comparison with systemd. Answer these two questions:
    1. Did sysvinit get faster after getting parallelism in?
    2003 is when sysvinit gets parallelism management interfaces. First example of sysvinit starting services up in parallel is it first version. Posix shell supports threading. So the trap here is sysvinit always had parallellism there was a major usability issue. Early sysvinit far user-unfriendly at parallelism before 2003 that people incorrect presume it was a new feature because distributions were not using the feature. So did parallelism make sysvinit faster no because it could not because you are talking about a day 1 feature in the open source thing called sysvinit and the platform sysv init it design was based on. Yes the platform sysv init parallelism configuration nightmare as well to the point operators of that platform did not use parallelism unless they absolutely had it.

    Originally posted by sinepgib View Post
    2. Did sysvinit get faster after replacing Bash with Dash?
    This is what happens when you only look at a point in history on one distribution not all of it. Bash progressively got slower over the years as it added features. sysvinit is dash is in fact slower than sysvinit with a early less feature full version of Bash. Also not every distribution used sysvinit with bash or dash.

    So this is really not as black and white as it first appears. Could you cripple sysvinit performance by using over feature filled version of bash yes you could. Do note that /bin/sh when used with bash disables a huge stack of bash features from being used. So problem here bash is loading a stack of code into memory and worse setting up for use that never going to be used in the init process.

    Remember sysvinit your shell script is meant to be limited to only posix standard define features. This limitation comes from the where the Linux sysvinit design was copied from being the sysv platform. Being on the sysv platform limiting the init scripts to posix standard features was not a problem.

    Originally posted by sinepgib View Post
    Regarding its implementation, it worked so badly due to dependency management and parallelism itself being afterthoughts that nobody would enable it by default and most would discourage its use.
    This is another area where people go down non factual rabbit hole. Few things you need to come aware of the sysv platform the origin point of the sysvinit design lacks a few problems. PID numbers on sysv platform can only be used once per boot when system runs out of new PID numbers the system crashes so this historic system does not suffer from PID recycling or PID tree breaking.. Using PID value in file for detecting if a dependency was started or not was valid on the sysv platform.

    The reality here is sysvinit on Linux design is really a round peg in a square hole problem. Yes how to do dependency management and parallelism is in the sysvinit design from day one. Problem here is the design presumes that you PID don't recycle and your PID tree cannot break it interconnects.

    There is a fundamental difference between the origin of the sysvinit design and where it ends up being used. That difference effectively renders many parts of the sysv init design not functionally usable.

    Fundamental difference between the origin of a init solution design and Linux resulting in implementation of that init system design on Linux not being correctly functional is not unique to sysvinit.

    Also be aware by end 2003 debian was enabling parallelism in sysvinit by default along with many other distributions. Yes 2003 is when we see the LSB work around to .pid files not being dependable due to contained PID value not being dependable.

    Originally posted by sinepgib View Post
    But rest assured, I am not under any confusion here, you are answering claims I did never make about projects I did never mention. I am aware of all those things, the reliability problems of sysvinit that are the actual reason of its replacement. I could also add that cross platform support was always rather moot, as opposed to the option of easily using alternatives on Linux, since only a few chimeras used sysvinit without a Linux kernel (proper BSDs use their own init, only Debian/kFreeBSD used sysvinit and I have yet to meet somebody who doesn't notice the contradiction of using that).
    Yes like this cross platform is moot claim you see it a lot. This complete ignores that reason why sysvinit does not work is itself is a cross platform design that was not designed to fit the Linux kernel. Turns out sysvinit implement on kernel that matchs what it designed for is a lot more feature complete than people assume.

    Sysvinit has a reliability problem on Linux and OS with PID recycling because the design was not designed for that behavour.

    FreeBSD own init uses their equal to PIDFD and FreeBSD has PID recycling..... Debian/kFreeBSD using sysvinit should be straight up alarm bells here is a design that is sysvinit that is not designed for a kernel that does PID recycling and now you are using sysvinit with a kernel with PID recycling. So a little more than a contradiction. Yes its a little more than a contradiction to be using sysvinit with the Linux kernel.

    The problem with sysvinit is that it was a decade old design that a person read out of a old book and brought to Linux and in the decade before sysvinit appeared on Linux OS operation changed a lot. Yes the source of the sysvinit design is 1983 UNIX System V yes that based off older UNIX System III 1982 so its at least a decade old design by the time Linux starts implementing the design.

    Interesting point freebsd own init system(and most other BSD init systems) is a direct descendant from Unix System V init system. Please note descendant not 1 to 1 copy but connected by source fragments. The BSD developers were willing to change and improve init system to keep up with current kernels this includes ignoring and extend posix standard where it made sense.

    Yes you could say having the open source sysvinit on freebsd is very much like allowing the bind and bat cousin of your great great great great grandfather who only knows how to driver a horse and wagon correctly attempting to drive your modern day car. Yes those 4 greats before grandfather are based on the Unix time line from sysv init origin point to current day freebsd init system with all the major alterations to make it compatible with modern day kernels.

    Us using linux with sysvinit was using a very badly out of date and incompatible solution. But that design solution put on system that design is for is very competent. Yes that great great great great grandfather given a old school horse and cart and a person who is use to modern only with only horse and cart is the one going to be in trouble right. Remember these examples the people are basically immortal because software is.

    This is what created the sysvinit problem the person saw sysv init on sysv compatible OS and the thing was brilliant with every feature you could want in a init system. Then it lands on Linux and 90% of those features no longer work all due to Linux kernel being a PID recycling OS then another 7% don't work due to PID tree breaking. Yes so left with only 3% of the starting features. There is over 300 features that the old sysv init had in usage.

    sinepgib the reality you have most likely never seen how good sysvinit should have been. This results in you thinking it did not have X feature. When in reality most of the time you will find sysvinit has X feature just that X feature is not usable because it either depends on PID not recycling or PID tree not breaking.

    Also we need to learn this lesson because we don't want to be 2 decades into the future and find that systemd... have in fact stagnated and drifted into being incompatible with modern day kernels.

    Leave a comment:


  • sinepgib
    replied
    Again with assuming I am drawing a comparison with systemd. Answer these two questions:
    1. Did sysvinit get faster after getting parallelism in?
    2. Did sysvinit get faster after replacing Bash with Dash?

    Regarding its implementation, it worked so badly due to dependency management and parallelism itself being afterthoughts that nobody would enable it by default and most would discourage its use.

    But rest assured, I am not under any confusion here, you are answering claims I did never make about projects I did never mention. I am aware of all those things, the reliability problems of sysvinit that are the actual reason of its replacement. I could also add that cross platform support was always rather moot, as opposed to the option of easily using alternatives on Linux, since only a few chimeras used sysvinit without a Linux kernel (proper BSDs use their own init, only Debian/kFreeBSD used sysvinit and I have yet to meet somebody who doesn't notice the contradiction of using that).

    Leave a comment:


  • oiaohm
    replied
    Originally posted by sinepgib View Post
    What I implied is a fact: parallelism is better for starting many independent services (or with clusters of dependencies) performance wise than sequential startup, and shells add massive overhead, specially if the actual service is quick to start.
    Sysvinit was not sequential by the time of systemd or upstart.
    https://manpages.debian.org/wheezy/s...tpar.8.en.html

    Note the date on this man page of 2003 yes upstart is 2006 and is 2010. Yes it 2003 when sysvinit with shell scripts got parallelism.

    "Starting many independent services (or with clusters of dependencies)" predates systemd. You have basically missed that Parallelism startup with shell existed.

    https://wiki.debian.org/LSBInitScripts
    The Linux standard base extra meta data being added to sysvinit scripts for services is also from 2003. So sysvinit 2003 also have clusters and dependencies. Of course not very clear to users when something was badly wrong.

    So its "Parallelism startup with shell" vs upstart and systemd that comes latter.

    I am starting to see the cause of your mistake.. Turns out shells overhead was not that massive compared what you need so everything can work correctly. Sinepgib systemd using pidfd and cgroups has overhead. Lot more performance overhead than one would otherwise presume.

    Systemd will spend more time with threads waiting for the kernel to deal with locking be this cgroups because there is locking there to record the cgroup information to deal with PID tree breaking issues or be it using PIDFD again there is need form of locking to prevent problems caused by PID recycling. This waiting has a performance cost. Note I said that sysvinit with shell could be faster but its consumer more power this CPU processing more instructions not being stuck waiting. Time at wall sysvinit is fast power usage sysvinit is worse.

    So it depends how you measure performance. If you are measuring performance by power usage systemd is better if you are measuring by startup time sysvinit is better.

    Parallelism with sysvinit is problem because you are depending on optional comments in the shell to be correct to have the configuration scripts process everything and configure everything so it starts up correctly. Advantage of systemd service files you are not getting anywhere if you don't fill in the basic information for parallelism this is advantage over upstart as well. Lot of people got upset with systemd sysvinit compadiblity layer as well because if you did not have your LSB extra comments it would just skip your service until you added them.

    sinepgib you were making a mistake that the system prior to systemd and upstart was still sequential startup. Sequential startup started not being used in 2003.

    Systemd selling points over sysvinit should read as.
    1) slightly better power effectiveness.
    2) Resistant to miss configuration of the parallelism startup(it still possible but less so).
    3) Proper service management due to not having the PID issues from the usage of cgroups and pidfd.
    Sysvinit selling points over systemd.
    1) Faster start up by insanely small margin if configured correctly.

    Yes if sysvinit was starting up sequential mode it would have been a pure walkover for systemd or upstart in performance. But that was not the historic case that by the time systemd and upstart came along. Distribution sysvinit usage was parallelism a few years before upstart and 7 years before systemd.

    The reality here kind of horrible. If posix shell script had been improved like having pidfd equal required by the posix standard then you init scripts using that this would have improved the service management a lot. Of course to fix all Linux problems you need cgroup support as well but there are a lot of Unix and Unix like systems that don't have the PID tree breakage problem in their kernels.

    sinepgib interesting right when you dig into it having a shell based init/service management system does not have to be poor performing. The reality is when you dig into it the shell cannot be 100 posix conforming because you are needing extensions for items like pidfd. This is where the complete problem of demanding that the shell be posix conforming falls in on it face.

    sinepgib you are basically using rose colour glasses to make the history look how you expect. You need to take those glasses of and notice that parallelism was earlier event. Shell based service management is part of the Linux world parallelism in start-up history as a operational solution. Please note I said operational solution not as functionally good solution.

    Leave a comment:


  • sinepgib
    replied
    Besides, did you even read the answers in the stack exchange question you linked to? What do they say?

    Leave a comment:


  • sinepgib
    replied
    Originally posted by oiaohm View Post

    The horrible part here is early systemd was slower in parallelism than modified sysvinit with dash. Yet debian still goes systemd at the time.

    https://superuser.com/questions/6026...this-even-work
    Welcome to a funny reality. Systemd was more power effective with early debian than sysvinit even that it was slightly slower.

    So sorry sinepgib you just stated a commonly believed myth that systemd and upstart was about improvement boot speed by improving parallelism. The reality here is systemd and upstart in fact has less parallelism than the old modified for parallelism sysvinit solution but it also has less CPU time wasted spinning on stuff.

    Yes there was a performance gain using dash over bash interest enough that less than the overhead early systemd overhead trying to use cgroups to cover for what is now done by PIDFD and the overheads of using cgroupv1. Yes there are some quite horrible overheads dealing with cgroupv1 defects.

    Early Systemd is slightly slower boot but a more power effective boot and closer proper service management that works with the way the Linux kernel treats PID values and PID trees.. Of course systemd overhead drops over time as the kernel side improves with cgroupv2 and pidfd.

    Its a surprise to some people that systemd does in fact give up some boot performance so that service management can work correctly. Defective service management even defective service management writing in shell script can be faster than systemd at starting the system up. Please note reporting that services are up correctly and being able to correctly stop/restart services all the time goes out the window with defective service management so making system issue diagnostic in production more time costly..

    sinepgib good question what one is more important. Boot speed/Parallelism vs Production service management providing correct information on service status and correctly stopping/starting and restarting services?

    Something I do find really annoying there is not command line tools using pidfd functions to perform tasks from shell scripts. Think about it a pidfdkill program that you are able to pidfdkill PIDnumber cgroup,,,,(list of other details to confirm target). This would allow you from script to exactly kill a target under Linux instead risking some random passing process that happened to take the PID number.

    https://pubs.opengroup.org/onlinepub...ties/kill.html

    Notice the kill command is in the posix standard for shell usage and it does not work correctly on systems that can recycle PID numbers. Yes the ps command again posix by the time you get the information and use the information it can be out of date. What the objective of pidfd is to make sure that if you are acting on out of date information to kill a PID nothing get killed because it knows it out of date information and that the PID died/crashed before you got around to attempting to send it the kill command. Note some other process can have taken over that PID number by that point..

    sinepgib the issue here is once you truly look maybe you don't want to start your services using bash/dash any more. Or at least you should be after some replacements to the common usage of kill and ps in you service management scripts because they don't work correctly with the Linux kernel and don't in fact work correctly with freebsd and many other kernels. Yes lot of Unix and Unix like operating systems have provide their own alternatives like pidfd because of the same fundamental problem. Yet people keep on writing service start up scripts with these fundamental problems.
    You're wrong again. For someone who expects people to read walls of text you do read too little of the short comments I posted. Did I mention systemd? Did I oppose systemd? Did I say shells are a good idea for init? Nope. Never. I corrected one claim about the reasons some people fret over shell scripts being POSIX. No more, no less. I said not that this is why systemd exists, as I am fully aware sysvinit was broken. What I implied is a fact: parallelism is better for starting many independent services (or with clusters of dependencies) performance wise than sequential startup, and shells add massive overhead, specially if the actual service is quick to start. Whether initial versions of systemd beat a patched up sysvinit is entirely irrelevant to the claim.
    Don't look for subtexts where there are none, that's a paranoid trait my friend.
    Last edited by sinepgib; 12 August 2022, 09:44 PM.

    Leave a comment:


  • coder
    replied
    Originally posted by oiaohm View Post
    ...
    What do you think you gain by this? Nobody is reading it. It's functionally equivalent to spam.

    Leave a comment:


  • oiaohm
    replied
    Originally posted by coder View Post
    You're starting to be abusive. Don't cross the line or your posts might start getting flagged.
    How with the defects of the Linux PID system has Gnome/KDE/XFCE/X11 DE session management worked correctly prior to systemd?

    Next question who was the party that first made solution that made service management or session management that could in fact work correctly with the Limitations of the Linux kernels PID system?

    What is the name of the project where the first functional correct service management/session management solution was made?

    Final question how many solutions exist today that are fully implemented to do session management/service management under Linux correctly?

    The last one is important the answer is very horrible. Coder what would you do if that last question turns out to be only 1? What if the problem appearance reducing diversity and the changes is because there was no real functional correct solution diversity for Linux in the first place?

    Extra question coder who is the standard body to submit system service management solutions to?


    Answer the questions Coder. Once you answer the questions you will work out that Redhat had no choice. Ubuntu starting upstart was because of the PID and the PID tree issues of Linux kernel yes canonical also had no choice. Keeping solution that causing CVE after CVE after CVE cannot go on forever.

    All major open source standards are developed after a functional implementation already exists

    sinepgib and others had the incorrect idea that systemd was about bootspeed. Yes people selling system sold boot speed but the real tests of systemd done by core debian and ubuntu maintainer in their compare of init before changing to systemd in fact found systemd is slightly slower other distributions found the same thing. That brings a question if systemd is not faster why did so many distributions decide to change to systemd for real. The answer like it or not is what I am point at coder.

    Why is Linux at the moment coming so systemd monolithic. There are a stack of areas that were also being effected by the same problem. CVE for PID recycling caused issues.

    https://saelo.github.io/presentation...st_the_pid.pdf The PID recycling issue is not restricted to Linux. This a presentation for OS X. I could find presentations for all kinds of posix compatible solutions. Every time the correct solution to fix PID recycling issue is use kernel particular code cease being a generic code base that works every where.

    Coder don't try moving the goal posts to another topic. The reality to take the debate forwards you need to know the answers to the questions I am asking. Also you need to accept there is a fundamental problem that comes from POSIX around process handling. Yes every Unix and Unix like OS with PID recycling support is effected by this problem. Every Unix and Unix like OS with PID recycling has not used a generic solution to solve this problem. Unique to kernel design solution in every case.

    Yes the fundamental problem is why KDE and Gnome had to find different session management solution to what they were using.

    Coder the Linux environment userspace coming in short term more monolithic is simple side effect of finally parties stopping sweeping the fundamental problems of POSIX under rug and ignoring them.

    Coder big thing here major problem effecting all POSIX based OS with PID recycling that the POSIX standard group is doing nothing about. There is a true bad organisation here that has been driving fragmentation and defective solutions.

    Remember POSIX is made after the fact. Yet people mandate you make scripts conforming to POSIX standard limitations without any consideration if the POSIX standard is correct. Getting POSIX standard changed to fix these issues would take effort. Remember you have a hurding cats problem here where all the different posix systems have already gone different directions.

    POSIX is key part of problem that caused Linux userspace excessively fragmented. Once one party decided to go hey we will make a solution that not limited by posix limitations and try to fix the problem is what leads to systemd and once that is created area after area that people had attempted over and again to make work while being cross platform parties have gone why in hell are we doing this.

    Coder you need to take off your rose colour glasses look back and see how a fundamental problem leads to Init, service management and session management fragmentation because end users where hitting problems. Yes coding to scratch itch problem was these people were coding without knowing exactly what the itch was causing them problems or knew what the itch was and found a worse problem with the method they tried to use to fix it. Systemd comes along and developer here finally solves it for the Linux kernel. Problem that puts people noise out of joint is that this is a kernel unique solution and lots of code needs to be updated no longer to depend on PID values alone.

    Coder this is another question how do you fix the PID recycling problem without breaking compatibility with service management solutions that are still using PID as their only ID system? The answer is a horrible you cannot.

    Redhat in reality is trying to remove the PID recycling problem from Linux. Redhat has the funding and resources to-do this. This is exactly like the old saying you cannot make a omelette without breaking a few eggs. Yes the PID recycling and PID tree problems have to be solved to stop future CVE from happening.

    The reality here we need a higher min standard of what is classed as acceptable in service management and session management. Mind you we most likely would not require this higher min standard if POSIX standard was not defective.

    Leave a comment:


  • coder
    replied
    Originally posted by oiaohm View Post
    No the party that upvoted you need to read back and notice you moved the goal post and to answer these questions themselves.
    You're starting to be abusive. Don't cross the line or your posts might start getting flagged.

    Leave a comment:


  • oiaohm
    replied
    Originally posted by coder View Post
    I think I've made it abundantly clear that I have no intention of playing your game. At this point, you're just spamming the forum.

    On the plus side, it makes your agenda that much more transparent.
    Next question who was the party that first made solution that made service management or session management that could in fact work correctly with the Limitations of the Linux kernels PID system?

    What is the name of the project where the first functional correct service management/session management solution was made?

    Come on now coder answer those questions.


    Final question how many solutions exist today that are fully implemented to do session management/service management under Linux correctly?

    The last one is important the answer is very horrible. Coder what would you do if that last question turns out to be only 1? What if the problem appearance reducing diversity and the changes is because there was no real functional correct solution diversity for Linux in the first place?


    No the party that upvoted you need to read back and notice you moved the goal post and to answer these questions themselves.

    What I'd rather see is open standards and open implementations, like we had with X11 and POSIX. Or, at the very least, maybe trying to establish solutions more by creating an ecosystem, rather than developing their own internal projects which they subsequently entrench via their dominant market position.
    Do note you X11 and POSIX standard was created after the fact and you used that as excuse as well why POSIX is bad. Then you use W3C and. Khronos group.

    Please note every one of the open standards vendor implements first make standard latter after vendor has worked out how it should work. Mozilla, Microsoft Google to W3C do their only implementations first before submitting to standard with W3C. Khronos group all your GPU vendors. Posix same thing all your Unix vendors implement their side first then submit to POSIX standard. X11 Protocol implement first then submit to X11 Protocol for approval.

    Yet you were here in that past post asking redhat to do the the reverse to normal coder.

    Extra question coder who is the standard body to submit system service management solutions to? Guess did exist in the Linux Standard Base that died because people would create init scripts not conforming to Linux Standard Base standards so what was point of having a standard for this.

    Coder the reality you don't want to get into this arguement because none of your points were ever right. Linux is no more monolithic or less standardised than it ever was. Maybe this is area were we need todo better. The real state is more in face now that redhat has taken many projects they had 100 percent control over anyhow since 1995 and put them under 1 project called systemd. Its no different to 50+ shell companies displaying themselves as a single company.

    Coder the history here totally horrible. Ways people claim it has changed it has not. Init and service management systems have almost always been in the Linux world chosen by survival of the most common. The most common has mostly been decided by the commercial distributions since 1995.

    Leave a comment:


  • oiaohm
    replied
    Originally posted by sinepgib View Post
    Not gonna read it all, the first phrase is enough to see you think I am defending the use of boot scripts when I am not. I said one thing and only one, and that is iff you go for scripts, the choice of POSIX is about picking your shell. It's specially important for Debian that needed to migrare from Bash to Dash due to slow boot times. Of course, the real solution to slow boots is parallelism and avoiding the shell altogether.
    The horrible part here is early systemd was slower in parallelism than modified sysvinit with dash. Yet debian still goes systemd at the time.

    https://superuser.com/questions/6026...this-even-work
    Welcome to a funny reality. Systemd was more power effective with early debian than sysvinit even that it was slightly slower.

    So sorry sinepgib you just stated a commonly believed myth that systemd and upstart was about improvement boot speed by improving parallelism. The reality here is systemd and upstart in fact has less parallelism than the old modified for parallelism sysvinit solution but it also has less CPU time wasted spinning on stuff.

    Yes there was a performance gain using dash over bash interest enough that less than the overhead early systemd overhead trying to use cgroups to cover for what is now done by PIDFD and the overheads of using cgroupv1. Yes there are some quite horrible overheads dealing with cgroupv1 defects.

    Early Systemd is slightly slower boot but a more power effective boot and closer proper service management that works with the way the Linux kernel treats PID values and PID trees.. Of course systemd overhead drops over time as the kernel side improves with cgroupv2 and pidfd.

    Its a surprise to some people that systemd does in fact give up some boot performance so that service management can work correctly. Defective service management even defective service management writing in shell script can be faster than systemd at starting the system up. Please note reporting that services are up correctly and being able to correctly stop/restart services all the time goes out the window with defective service management so making system issue diagnostic in production more time costly..

    sinepgib good question what one is more important. Boot speed/Parallelism vs Production service management providing correct information on service status and correctly stopping/starting and restarting services?

    Something I do find really annoying there is not command line tools using pidfd functions to perform tasks from shell scripts. Think about it a pidfdkill program that you are able to pidfdkill PIDnumber cgroup,,,,(list of other details to confirm target). This would allow you from script to exactly kill a target under Linux instead risking some random passing process that happened to take the PID number.

    https://pubs.opengroup.org/onlinepub...ties/kill.html

    Notice the kill command is in the posix standard for shell usage and it does not work correctly on systems that can recycle PID numbers. Yes the ps command again posix by the time you get the information and use the information it can be out of date. What the objective of pidfd is to make sure that if you are acting on out of date information to kill a PID nothing get killed because it knows it out of date information and that the PID died/crashed before you got around to attempting to send it the kill command. Note some other process can have taken over that PID number by that point..

    sinepgib the issue here is once you truly look maybe you don't want to start your services using bash/dash any more. Or at least you should be after some replacements to the common usage of kill and ps in you service management scripts because they don't work correctly with the Linux kernel and don't in fact work correctly with freebsd and many other kernels. Yes lot of Unix and Unix like operating systems have provide their own alternatives like pidfd because of the same fundamental problem. Yet people keep on writing service start up scripts with these fundamental problems.
    Last edited by oiaohm; 12 August 2022, 05:02 AM.

    Leave a comment:

Working...
X