Announcement

Collapse
No announcement yet.

Wine Developers Are Working On A New Linux Kernel Sync API To Succeed ESYNC/FSYNC

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • #91
    Originally posted by indepe View Post
    Why would it set the event state to 'signaled' if then, in the same operation, it resets it? Unless it calls a lower level function like set-event which makes the wake up calls, and this lower level function has a side-effect outside the locked region. In other words, the whole operation is not an atomic whole anymore, the kind of atomic whole that a lock should provide.
    As I said we are talking pre atomic locking. It has non atomic behaviours that come very hard to emulate with atomic instructions that lead to horrible overheads and race condition issues as you attempt to performance optimise while using atomic locks.

    Its really simple when you are not attempting atomic and you just use old syscall form of lock for this when the function is wake up as many as you can waiting in the kernel que for the lock and telling them that the event was triggered and setting the state back to nothing if it set or leaves it alone if it not set. This is not using atomic shared memory its using per process memory remember we are talking old school locking before atomic memory operations in this design. So its not needing to call a lower level function like set-event in a pulseevent in windows because its not in fact atomic. Pulseevent just has to find the process that should be started set the that they have the lock in their process memory and pass them to the scheduler. Per process state of lock is something your atomic locks was not designed to-do same with the idea that multi items take the same lock at the same time.

    There are old class of lock used for process management/thread queing not data protection these are not atomic class locks guess what Windows Event is. Windows Event is a horrible hybrid some of it use cases is a 1 to 1 active thread lock like a normal atomic others use cases of Windows Event is a 1 lock to many active thread lock at the same time that is absolutely not atomic. This beast Windows Event is not atomic lock but its still a lock.

    Since this stuff is pre atomic locking it doing operations that atomic locks should not do in ways you would not with atomic locks without a lot of work. Like you are not going to duplicate that you have the lock/event in per process memory with atomic because that undermines total userspace solving of locking but pre atomic where you have the kernel setting the lock state in the per process memory and the lack of atomic memory protection and the poor early MMU/embedded MMU you will do this(yes there are MMU without page tables so you cannot share memory safely between process in the pre atomic world so memory duplication is a thing). Once you have per process lock state of the pre atomic locks you can now perform horrible actions that don't suit atomic locking at all.

    The idea that pulseevent had to call a set-event outside it self is because you are thinking this has to fit into atomic style of lock but event is a pre atomic lock with per process lock state in the process memory not shared and the kernel can keep track of how many of those it started.

    Fun right pre atomic locks have the concept of selective signalling and this is easy todo when the lock state is unique per process and is insanely hard with atomic lock where the lock state is in shared memory between processes.

    Windows Event is a feature rich pre atomic style lock these are design that the lock will be kernel managed and its not in fact using shared memory between processes for lock state and due to this it can do stack of horrible things like permission guided locking and multi threads holding the same lock at the same time.... Basically stack of world breaking things to atomic lock concepts due to being world breaking to atomic lock concepts comes really hard to emulate with atomic locking and next to impossible to-do in a performance way using atomic locks.

    This is purely a different class of locking. Problem here you are keeping on thinking everything has to be atomic when the lock you are looking at is a feature rich pre atomic style of lock is its really not atomic but in limited use cases of it can look atomic. Other problem is courses teaching locking don't cover these pre atomic locks.

    indepe really with atomic you are being like a person with a hammer so everything looks like a nail including screws because they don't know screws exist.

    Comment


    • #92
      Originally posted by oiaohm View Post
      Pulseevent just has to find the process that should be started set the that they have the lock in their process memory and pass them to the scheduler. Per process state of lock is something your atomic locks was not designed to-do same with the idea that multi items take the same lock at the same time.
      As far as I can tell, this appears to be the core of your argument, and it appears to be a misunderstanding.

      It is very possible to implement per-thread/per-process wait/wake control in userspace, if you want to or need to. And you don't need scheduler support for that. It is actually relatively easy to do. And that is not substituting the scheduler's functionality. It is always application logic which decides which thread is runnable and which thread needs to be in a wait state. The scheduler's function is just to assign threads to available CPUs.

      I don't know why you think what you think, since you are not really explaining that. What are your sources? Are just referring to the proposal's email, for example this part:

      According to my reasoning, in order to really have PulseEvent work right, we need explicit scheduler support. That does bring up the option of having a "scheduler", i.e. managing wait-queues, in user space.
      Managing wait-queues, as such, does not require scheduler support, and is not problematic. And it is always application logic that decides which thread needs to wait. If there really is a need for scheduler support, then the real reason has not been named.

      EDIT: Actually the OS can also be given the option to select one of several threads to be runnable, but if that is a problem, managing wait-queues solves that as well. I don't yet see why that specific detail would be really the problem here, though.
      Last edited by indepe; 01 February 2021, 07:45 PM.

      Comment


      • #93
        Originally posted by Weasel View Post
        indepe Dude I told you, he just spams technobabble every time without understanding what he's reading and just links random stuff. You can keep replying to him of course but keep it in mind, so you don't feel like you've wasted your time. I'm not talking about this thread in particular, he always does it.
        Thanks for the tip, so to speak. Currently I'm curious how far this goes...

        Comment


        • #94
          Originally posted by indepe View Post
          As far as I can tell, this appears to be the core of your argument, and it appears to be a misunderstanding.

          It is very possible to implement per-thread/per-process wait/wake control in userspace, if you want to or need to. And you don't need scheduler support for that. It is actually relatively easy to do. And that is not substituting the scheduler's functionality. It is always application logic which decides which thread is runnable and which thread needs to be in a wait state. The scheduler's function is just to assign threads to available CPUs.
          Except this is where you are going wrong. Windows Event with PulseEvent is to multi threads. We are not talking about 1 thread being runnable and other threads waiting.

          Instead this is where you have like 4 threads avalible you can wake up 4 threads and there are 16 waiting.

          Remember the threads that have been woken up by the PulseEvent think the Event is set but everything else is to think the Event is unset. Any thread that starts after PulseEvent that was not started by the PulseEvent think the event is not set. This is a real Schrödinger's Cat where the lock can be both in unlocked and locked at the same time in different parts of the program and that is 100 percent valid.

          Its is the fact that with Window Events the scheduler is meant to remain starting other threads is why emulation of Windows Events get so hard. So its not only do you need to device what thread is runnable its what thread should know the Event happened/triggered. If lock that allowed you code to run was triggered by X event you need to know that but if you thread was not triggered by X Event you should not see X Event as set. This causes the dual state problem.

          Do take a good read on wait on multiable
          The following example uses the CreateEvent function to create two event objects and the CreateThread function to create a thread.


          "First event was signaled.\n" These bits in particular. Where you are meant to know what event out the waiting on multiple list in fact triggered the wake up or if all of them did. Of course if the event did not trigger your process wake up you are to know that even it the event is triggered while you code is running.

          Also do note if your thread is woken up by PulseEvent on windows it is only meant to wake up if the scheduler can straight away schedule time to run that thread. So deciding if a thread is runnable no longer is the thread in a wait state alone either. PulseEvent logic is thread waiting on Event and is there CPU time to run it decides if it should be moved into wait state to be woken up by the Windows scheduler so PulseEvent is a priority inversion class item where stuff should be pushed to the front of the CPU scheduler que for it to behave right.

          indepe your idea of Application logic does not apply. This is a pre atomic design from VMS originally its was not build at first around wait states. Instead was based around events happen so respond to them. Waits states is Unix way.

          Comment


          • #95
            Originally posted by oiaohm View Post

            Except this is where you are going wrong. Windows Event with PulseEvent is to multi threads. We are not talking about 1 thread being runnable and other threads waiting.

            Instead this is where you have like 4 threads avalible you can wake up 4 threads and there are 16 waiting.
            Your descriptions are really hard to understand (if at all), and I'm afraid that it is because you hardly understand what you are talking about.

            I talked about 1 thread runnable in the "EDIT" part of my post, but that was a side note and has nothing to do with the main situation. It was a quite irrelevant detail. But I can't really tell if that is what you are referring to. However see the following.

            The logic for each waiting thread in PulseEvent is separate and independent, except in so far as an auto-reset event is used. I am referring to this:

            https://docs.microsoft.com/en-us/win...ase-pulseevent

            So it doesn't matter how many threads are this or that. if it is auto-reset, then at most one thread is satisfied, otherwise as many threads as can be. That is according to what else the threads are waiting for, if they have called WaitForMultipleObjects with the for-all option.

            Originally posted by oiaohm View Post
            Remember the threads that have been woken up by the PulseEvent think the Event is set but everything else is to think the Event is unset.
            This is too imprecise. They don't think it *is* set, what they think is that it *was* set (or 'pulsed').
            (And the event state can always change while the thread is runnable or running.)

            Originally posted by oiaohm View Post
            Any thread that starts after PulseEvent that was not started by the PulseEvent think the event is not set. This is a real Schrödinger's Cat where the lock can be both in unlocked and locked at the same time in different parts of the program and that is 100 percent valid.

            Its is the fact that with Window Events the scheduler is meant to remain starting other threads is why emulation of Windows Events get so hard. So its not only do you need to device what thread is runnable its what thread should know the Event happened/triggered. If lock that allowed you code to run was triggered by X event you need to know that but if you thread was not triggered by X Event you should not see X Event as set. This causes the dual state problem.
            This appears to be nonsense. After PulseEvent, before any other modification, all threads see and think the event *is* not-set now. Yet those threads that have been waiting for PulseEvent know that it *was* set for an infinitesimal short time (only conceptually). But they also know that it may have been reset, because PulseEvent is documented to reset the event. And a reset-event may have been called anyway. There is no conflict if the threads understand that this information relates to the past.

            Besides, if this was a problem, it would be a problem with PulseEvent itself, not a problem for WINE. WINE doesn't have to solve any problem PulseEvent might have.

            Originally posted by oiaohm View Post
            Do take a good read on wait on multiable
            https://docs.microsoft.com/en-us/win...ltiple-objects

            "First event was signaled.\n" These bits in particular. Where you are meant to know what event out the waiting on multiple list in fact triggered the wake up or if all of them did. Of course if the event did not trigger your process wake up you are to know that even it the event is triggered while you code is running.
            What's the problem here? I don't see any.

            Originally posted by oiaohm View Post
            Also do note if your thread is woken up by PulseEvent on windows it is only meant to wake up if the scheduler can straight away schedule time to run that thread. So deciding if a thread is runnable no longer is the thread in a wait state alone either.
            This would be nonsense, it is your own misunderstanding. Even if the thread can be scheduled right away, there is no guarantee for how much code gets executed before the next time slice de-schedule. Also that is not in the documentation, but it would have to be. It is a ridiculous assumption.

            What it does depend on is: the other wait conditions if the threads is waiting for multiple with the for-all option.

            Originally posted by oiaohm View Post
            PulseEvent logic is thread waiting on Event and is there CPU time to run it decides if it should be moved into wait state to be woken up by the Windows scheduler so PulseEvent is a priority inversion class item where stuff should be pushed to the front of the CPU scheduler que for it to behave right.
            That wouldn't make anything behave right.

            Originally posted by oiaohm View Post
            indepe your idea of Application logic does not apply. This is a pre atomic design from VMS originally its was not build at first around wait states. Instead was based around events happen so respond to them. Waits states is Unix way.
            You don't understand the matter well enough to make such judgements. Far from it.
            Last edited by indepe; 02 February 2021, 09:59 PM.

            Comment


            • #96
              Originally posted by indepe View Post
              This appears to be nonsense. After PulseEvent, before any other modification, all threads see and think the event *is* not-set now. Yet those threads that have been waiting for PulseEvent know that it *was* set for an infinitesimal short time (only conceptually). But they also know that it may have been reset, because PulseEvent is documented to reset the event. And a reset-event may have been called anyway. There is no conflict if the threads understand that this information relates to the past.
              The wine testsuite around this function showed the behavour. Take a closer look at the wait multiable to check what triggered the wait multiable was check the event data in the process. So no thread waiting on a pulse event that have woken up by a pulse event see the event as set not reset. The processes that start after see the event as reset to unset.

              Now this gets dangerous to say the lease what happens if you perform a windows pulse event on a event that is already set you get to see the reset process that the processes that time slice at the time pulse event was called other than the one pulse event run in keep on thinking the event is set until time slice end and context switch happens.

              Non atomic locking you are not seen the current state of the lock in lots of cases instead you are seeing the past. Modern atomic locking using memory methods you are seeing the current state of the lock.

              The idea that your thread knows that a lock has been reset here is wrong this you thinking atomic again. Thread woken up by a PulseEvent for a full time slice under windows thinks the event is set at least because the event was set when the timeslice started and only comes reset when context switch happen.

              Originally posted by indepe View Post
              This would be nonsense, it is your own misunderstanding. Even if the thread can be scheduled right away, there is no guarantee for how much code gets executed before the next time slice de-schedule. Also that is not in the documentation, but it would have to be. It is a ridiculous assumption.
              No the documentation on functions is incomplete. This is kind of on the right track this is another part of the problem that the amount of time slice given to a process on a pulse event has to be suitable long or you will risk a race condition where something that should be set in the application code is unset before it should be.

              Why is a windows context switch so slow compared to a Linux one? Part of the answer to this is how Windows Events is processed. Different pre atomic designed thing in windows that state is updated on context switch. Yes common in pre atomic locking that state is not updated without context switch happening this leads to very different behaviours.

              Comment


              • #97
                Originally posted by oiaohm View Post
                The wine testsuite around this function showed the behavour. Take a closer look at the wait multiable to check what triggered the wait multiable was check the event data in the process. So no thread waiting on a pulse event that have woken up by a pulse event see the event as set not reset. The processes that start after see the event as reset to unset.
                It doesn't really matter for the questions we are discussing, but I am curious: What does the testsuite test? The Windows 10 implementation, or WINE's implementation? The version using evenfd?

                If an implementation of PulseEvent leaves the event state on SET for a non-zero amount of time before it goes to RESET, I would consider that problematic, but not necessarily wrong. Problematic because it can cause the confusion that we now have to deal with.

                Originally posted by oiaohm View Post
                Now this gets dangerous to say the lease what happens if you perform a windows pulse event on a event that is already set you get to see the reset process that the processes that time slice at the time pulse event was called other than the one pulse event run in keep on thinking the event is set until time slice end and context switch happens.

                Non atomic locking you are not seen the current state of the lock in lots of cases instead you are seeing the past. Modern atomic locking using memory methods you are seeing the current state of the lock.
                Any non-atomic behavior here is just a problem, not a feature that needs to be reproduced. A correct application will not rely on seeing the event SET in this context when calling READ_EVENT afterwards. It will just use the return code from the Wait call. And an incorrect one may not see it SET anyway when it uses READ_EVENT.

                Originally posted by oiaohm View Post
                The idea that your thread knows that a lock has been reset here is wrong this you thinking atomic again. Thread woken up by a PulseEvent for a full time slice under windows thinks the event is set at least because the event was set when the timeslice started and only comes reset when context switch happen.
                It doesn't know for sure that it has *already* been reset (because that is implementation and potentially timing dependent), but it knows that if not, it will soon be. Note that I used the word "may" in my previous post.

                Originally posted by oiaohm View Post
                No the documentation on functions is incomplete. This is kind of on the right track this is another part of the problem that the amount of time slice given to a process on a pulse event has to be suitable long or you will risk a race condition where something that should be set in the application code is unset before it should be.
                "Suitable long"? That is a race condition in itself. There is no such thing in any sane world. We are talking about buggy applications, here, if they try to rely on that. Rest assured, the kernel engineers will declare you insane before even thinking of modifying the Linux kernel scheduler in such a way. I think this is just a gross misunderstanding on your part (and anyone else thinking like that.) Or maybe you are making stuff up as you go along.

                The documentation does not guarantee that the event will remain SET for any amount of time or for the next READ_EVENT call. The application needs to use the return code of the wait call, and that won't be a problem. Or the application itself needs to make additional guarantees about the behavior of specific events.

                Originally posted by oiaohm View Post
                Why is a windows context switch so slow compared to a Linux one? Part of the answer to this is how Windows Events is processed. Different pre atomic designed thing in windows that state is updated on context switch. Yes common in pre atomic locking that state is not updated without context switch happening this leads to very different behaviours.
                Not without context switching? Are you maybe talking about so-called cooperative multitasking that switches context only on yield calls? Holy cow!
                Last edited by indepe; 03 February 2021, 12:35 AM.

                Comment


                • #98
                  Are you perhaps saying WINE should support applications that were written for cooperative multitasking?

                  Comment


                  • #99
                    Adding to the post before: I see that in the email proposal for "NtPulseEvent" it also says:
                    In both cases the event state is *not* changed to signaled.
                    Which sounds like your 'interpretation' of that point is not shared there either, as there is no mention of setting it to signaled at first. So I wonder: where did you get that idea?

                    Comment


                    • Originally posted by indepe View Post
                      Are you perhaps saying WINE should support applications that were written for cooperative multitasking?

                      When people say windows is a hybrid kernel is true in more than one way. Lot of people think of the hybrid between monolithic and microkernel designs but it does not stop there.

                      That the fun part of windows. Sections of Windows are still cooperative multitasking. Sections are preemptive multitasking others are old cooperative multitasking style enhanced with preemptive multitasking.


                      Fun part here Windows NT when it starts was meant to be multi platform supporting including platforms that could not perform preemptive multitasking. So really core things like the Event system you find its not designed for pure preemptive multitasking world instead its in fact designed for a cooperative multitasking world because of the idea that could work on any CPU/Hardware type. So on Windows you have these nice hybrid between cooperative multitasking and preemptive multitasking applications in games that depend on both behaviours in different areas of their code base.

                      Yes the pre atomic style locks I am talking about you would commonly see in your old cooperative multitasking systems.

                      Really fun its not that wine should support pure cooperative multitasking applications because those basically don't exist on modern windows. But has to support a hybrid that is somewhere in the middle between cooperative multitasking and preemptive multitasking. Yes its kind important to be aware what kind of area you are in with windows. The really old stuff Windows NT design are pre atomic locking ideas with cooperative multitasking feature mixed in for good measure.

                      Now trying to emulate the old areas of the Windows NT design with pure more modern design parts not going to be fun.

                      Windows internally is a hybrid mess this makes emulation in particular areas insanely hard.

                      Originally posted by indepe View Post
                      Which sounds like your 'interpretation' of that point is not shared there either, as there is no mention of setting it to signaled at first. So I wonder: where did you get that idea?


                      When you read the reverse engined studies on it most of them with different errors.

                      Function sets event to signaled state, releases all (or one - dependly of EVENT_TYPE) waiting threads, and resets event to non-signaled state. If they're no waiting threads, NtPulseEvent just clear event state.
                      Step one set the event to signaled state to release the waiting threads and since they are released when it signeled to them them the event is triggered. Fun here is that releases all or one is not complete. The releases is in fact all, one or as many that can be allocated time slices waiting threads based on event_type nothing like being nice bit I do different things.
                      You have just pointed straight at one of the goofs why wine implementation is not working right with some applications because when the thread triggered by event fires up the event is meant to be set not being set causes different applications to fail.

                      Notice the fun here that NTPulseEvent triggers threads in 3 different amounts based on the EVENT_TYPE the event past to it has. This is another trap that can really get developers using this function. Think of Event with max number waiting with event_type that that when you do pulse event all trigger is a nice way to stall out windows. Remember the developer could have only been handling Event types that start only 1 thread from NTPulseEvent so gets really caught out.

                      Please note this documentation here does not cover the started threads what they look like. The documentation on NTPulseEvent triggered threads them shows the event remaining in signalled state from their prospective even after NTPulseEvent is complete as the Event is cleared for everything else. This area of Windows really does not look like a preemptive multitasking OS or modern Locking. This area really shows NT Windows design age as really really old.

                      Comment

                      Working...
                      X