Announcement

Collapse
No announcement yet.

Wine Developers Are Working On A New Linux Kernel Sync API To Succeed ESYNC/FSYNC

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • #61
    Originally posted by Weasel View Post
    WTF is a pre atomic method? Give actual examples.
    I really should not have to be doing this you are meant to be a Windows programmer apparently you don't know your stuff.

    The following example uses the CreateEvent function to create two event objects and the CreateThread function to create a thread.


    Notice here wait for multiple is not Mutex but instead event. Happens that the event in windows is not atomic design.

    The Windows security model enables you to control access to event, mutex, semaphore, and waitable timer objects. Timer queues, interlocked variables, and critical section objects are not securable. For more information, see Access-Control Model.


    The give away is right on this page. EVENT_MODIFY_STATE is the give away. The fact state is meant to be able to change while running.

    Modify state access, which is required for the SetEvent, ResetEvent and PulseEvent functions.
    I do like now that the ResetEvent goes to 404. If you cannot work out these pre atomic lock designs can in places be here be dragons. Reset event the wrong way of course is going to result in the data protection you thought the event was going to give you not being there as well.

    Do notice on this page that Mutex there is a space left in the API/ABI for a Mutex to be a muggable lock like a EVENT is with the but that does not mean its not muggable.

    There is also another bit of fun on the security page.
    SYNCHRONIZE (0x00100000L) The right to use the object for synchronization. This enables a thread to wait until the object is in the signaled state.
    Notice if you can wait on any of the windows built in locking structures you have to have a privilege. Do note its legal to change a thread from being allowed with SYNCHRONIZE to not allowed while its holding the lock be it Windows kernel controlled Mutex so is meant to result in the lock being reclaimed from the process by kill it for perform a now unprivileged action.

    Your general atomic locking is not your old school privilege/kernel controlled/pre atomic locking. The big things here is the old pre atomic locking include methods to take back the lock by force that requires extra information on who has the lock to be tracked by privilege party being the kernel. Yes I know its a idea to say we will avoid syscalls to take out a lock but these old pre atomic are designed to use syscalls and the difference in privilege between kernel and user-mode and have security privileges on the locking to be processed as part of the locking solution. These beasts are different.

    Yes saying this stuff can be implemented with atomic locking in userspace means you don't understand the problem at all. These are locking structures with privileges designed to be kernel processed that were not designed to use atomic to avoid syscalls. Attempting to avoid syscall makes implementing them hell.

    Comment


    • #62
      Originally posted by oiaohm View Post
      I really should not have to be doing this you are meant to be a Windows programmer apparently you don't know your stuff.

      https://docs.microsoft.com/en-us/win...ltiple-objects

      Notice here wait for multiple is not Mutex but instead event. Happens that the event in windows is not atomic design.

      https://docs.microsoft.com/en-us/win...-access-rights

      The give away is right on this page. EVENT_MODIFY_STATE is the give away. The fact state is meant to be able to change while running.
      Events are absolutely atomic on Windows. Obviously all events have a state else they wouldn't be events (you'd only have PulseEvent). Actually all synchronization objects have state. And yes you absolutely can modify this state atomically, especially with the kernel syscalls (which is what it is under Windows). EVENT_MODIFY_STATE is just an access right, it has nothing to do with atomic or not, just "permissions".

      ResetEvent is a basic function that simply sets the state of an event to non-signaled. It's absolutely atomic. To be fair, you can do it even in userspace atomically, by using the "xchg" instruction, which would ALSO give you the event's former state, atomically. It might be implemented that way, who knows?

      Regardless, the whole point of any synchronization API is to synchronize things, and being atomic is a hard requirement. Otherwise, the API is deeply flawed (like PulseEvent is).

      Comment


      • #63
        Originally posted by oiaohm View Post
        Your general atomic locking is not your old school privilege/kernel controlled/pre atomic locking. The big things here is the old pre atomic locking include methods to take back the lock by force that requires extra information on who has the lock to be tracked by privilege party being the kernel. Yes I know its a idea to say we will avoid syscalls to take out a lock but these old pre atomic are designed to use syscalls and the difference in privilege between kernel and user-mode and have security privileges on the locking to be processed as part of the locking solution. These beasts are different.

        Yes saying this stuff can be implemented with atomic locking in userspace means you don't understand the problem at all. These are locking structures with privileges designed to be kernel processed that were not designed to use atomic to avoid syscalls. Attempting to avoid syscall makes implementing them hell.
        When the locking happens inside the Windows kernel, it is still atomic locking, it just happens hidden by the Windows kernel, under the Windows kernel's control.

        But in WINE the emulation of a Windows kernel call happens in Linux user space (except if that userspace function in turn calls the Linux kernel).

        So you have 3 layers: 1) Windows application, 2) WINE emulation layer, 3) Linux kernel.
        The WINE emulation layer is inside the Windows kernel (so to speak), but in LINUX user space.

        So in this discussion when we talk about userspace, we mean the WINE layer which is inside the emulated Windows kernel.

        That means you cannot use userspace atomic operations in the Windows application for events, but you can use LINUX userspace atomic operations in the WINE layer emulating the inside of the WIndows kernel.

        Hope that helps.
        Last edited by indepe; 22 January 2021, 09:54 PM.

        Comment


        • #64
          Originally posted by Weasel View Post
          Events are absolutely atomic on Windows. Obviously all events have a state else they wouldn't be events (you'd only have PulseEvent). Actually all synchronization objects have state. And yes you absolutely can modify this state atomically, especially with the kernel syscalls (which is what it is under Windows). EVENT_MODIFY_STATE is just an access right, it has nothing to do with atomic or not, just "permissions".
          just "permissions".< This is the big catch permissions have to be processed somewhere this is what you are missing. Pre atomic cpu instructions perform permission checks on locking functions is acceptable behaviour. Post cpu atomic instruction designs avoid this as it brings a overhead.

          Originally posted by Weasel View Post
          ResetEvent is a basic function that simply sets the state of an event to non-signaled. It's absolutely atomic. To be fair, you can do it even in userspace atomically, by using the "xchg" instruction, which would ALSO give you the event's former state, atomically. It might be implemented that way, who knows?
          The stuff design pre atomic instructions in cpu use syscall to kernel space for the atomic operations. These are not done by the xchg instruction in userspace. There are kernel space. ResetEvent and PulseEvent in the syscall are checking if you still have the permission to perform them in Windows.

          Weasel they design is before the xchg instruction existed because the code starts on alpha not x86 in Event area. xchg is a more modern instruction than that section of the Windows code base.

          Originally posted by Weasel View Post
          Regardless, the whole point of any synchronization API is to synchronize things, and being atomic is a hard requirement. Otherwise, the API is deeply flawed (like PulseEvent is).
          PulseEvent flawed logic is part of wait multi objects under windows. Yes the Event based locking under windows is hell for the oddities it allows with a lot of odd ball quirks and its used all over the place in Win32 api.

          Yes the whole point of any synchronisation API should be to synchronise things but this is windows we are talking about with its Event system and other old synchronise API/ABI they are broken in places applications are designed to function with the way the windows synchronise systems fail to-do what you expect.

          Comment


          • #65
            Originally posted by indepe View Post

            When the locking happens inside the Windows kernel, it is still atomic locking, it just happens hidden by the Windows kernel, under the Windows kernel's control.

            But in WINE the emulation of a Windows kernel call happens in Linux user space (except if that userspace function in turn calls the Linux kernel).

            So you have 3 layers: 1) Windows application, 2) WINE emulation layer, 3) Linux kernel.
            The WINE emulation layer is inside the Windows kernel (so to speak), but in LINUX user space.

            So in this discussion when we talk about userspace, we mean the WINE layer which is inside the emulated Windows kernel.

            That means you cannot use userspace atomic operations in the Windows application for events, but you can use LINUX userspace atomic operations in the WINE layer emulating the inside of the WIndows kernel.
            This is you being wrong. Wine is a Compatibility Layer. So wine tries to avoid emulation of the Windows kernel where ever possible, Its is required to avoid emulating Windows kernel where possible to avoid the performance over head.

            Wine developers are wanting the compatible locking in the Linux kernel so it does not have to be emulated instead just straight up substituted.

            Wine is not emulation is old name of Wine because instead of emulating windows kernel where possible its substitution with what the host OS kernel provides.

            Comment


            • #66
              Originally posted by oiaohm View Post

              This is you being wrong. Wine is a Compatibility Layer. So wine tries to avoid emulation of the Windows kernel where ever possible, Its is required to avoid emulating Windows kernel where possible to avoid the performance over head.

              Wine developers are wanting the compatible locking in the Linux kernel so it does not have to be emulated instead just straight up substituted.

              Wine is not emulation is old name of Wine because instead of emulating windows kernel where possible its substitution with what the host OS kernel provides.
              You are going to get a lot of objections if you try to turn the LINUX kernel into a Windows kernel directly. That's going to be a disaster.

              A longer time ago I have taken a look at the esync source code, and it is doing the same kind of things that I am talking about. Except I am proposing other (existing) technologies instead of using eventfd.

              Comment


              • #67
                Originally posted by indepe View Post

                You are going to get a lot of objections if you try to turn the LINUX kernel into a Windows kernel directly. That's going to be a disaster.

                A longer time ago I have taken a look at the esync source code, and it is doing the same kind of things that I am talking about. Except I am proposing other (existing) technologies instead of using eventfd.
                Something to remember esync code does not work that games and other things needing windows behaviour does work. That was covered in the post asking for this functionality.

                Comment


                • #68
                  Originally posted by oiaohm View Post

                  Something to remember esync code does not work that games and other things needing windows behaviour does work. That was covered in the post asking for this functionality.
                  That much was clear from the beginning. (More or less.)

                  Comment


                  • #69
                    Originally posted by oiaohm View Post
                    just "permissions".< This is the big catch permissions have to be processed somewhere this is what you are missing. Pre atomic cpu instructions perform permission checks on locking functions is acceptable behaviour. Post cpu atomic instruction designs avoid this as it brings a overhead.
                    ???? Permission checks are simple if conditions (cmp instructions probably, or test), they don't even have to be atomic because they do NOT change or do anything in regards to the object's state. They happen before any lock is taken. They do not change during the lifetime of the object anyway; they're set when you create or open the sync object.

                    It's not different than permission checks in any other part of the Windows API, such as for files, or literally any other API. If a permission check fails on a file the file doesn't even get opened.

                    Originally posted by oiaohm View Post
                    Weasel they design is before the xchg instruction existed because the code starts on alpha not x86 in Event area. xchg is a more modern instruction than that section of the Windows code base.
                    xchg existed since 8086, the original x86 CPU.

                    It wasn't "special" like now, though, because every instruction was atomic, all memory operations "locked" the memory bus, because there was no caching or multi-threading. In practice, you could have used it for locks just like now, so nothing changed, if multi-threading existed.

                    You're just grasping at straws, you make me cringe with the nonsense you post.

                    Comment


                    • #70
                      Originally posted by Weasel View Post
                      Ok, I understand what you're trying to say now. You want to implement the whole thing in userspace, just with locks to protect the code from races.
                      On thinking about it, I want to add something here: The reasons to use locking here, even for functions like SET_EVENT and PULSE_EVENT, even during normal operation, are very related to the reasons why I think this is not an API suited for general use on Linux (or to be added to the kernel where they would still need to address contention, and be an additional larger burden even if not used). Also these APIs would lead programmers to structure their application code in an very inefficient way. Linux should have better and more efficient APIs, if any new ones. Those, however, would (likely) not be directly suited to emulate Windows API.

                      Therefore I think that Windows emulation is a use case that should be handled differently than everything else, and as far away from the Linux kernel as possible, so to speak.
                      Last edited by indepe; 23 January 2021, 03:57 PM.

                      Comment

                      Working...
                      X