Windows NT Sync Driver Proposed For The Linux Kernel - Better Wine Performance

[oiaohm]

For (3), it's a bug because they didn't expect the kernel to reach the limit and block itself. It was not accounted for, not intended.

It's a mistake that is hard to fix, but still, a mistake. A bug. It's not a design flaw, it's the kernel's "quirk" (imposing limits not in the API itself).

Yes as said from the start agreeing with you is the deadlocks wineserver suffers from are bugs normally in something else.

Not accounted for is correct. Yes not intended is also could be correct. There is a little miss. These issues with kernels having some quirk causing wineserver to deadlock get reported when reduced to unique cases about 10 times a year. Out of these cases less than 1 percent is a full kernel stop. This means that for 99%+ of these issues a watchdog process would be able to pick up that wineserver has stopped and step in prevent everything from going out of control. Kernel full stop the system already dead.

This is a mistake you can in fact design to mostly counter by adding a watchdog.

Car crashes are hard to account for yet we still build in seat belts and airbags.

Kernel blocking itself and scheduler then not giving thread any more CPU time happens more than we like and it lot more common of a issue.

https://stackoverflow.com/questions/25683795/batch-file-kill-a-stuck-processcpu-utilization-is-zero

Please note OS deadlocked process(type 3 problem) is not a Linux/BSD/Unix/Macos unique thing when you look into windows stuck processes you find that this happens on Windows as well as a rare event..

Thing is a wineserver deadlocked by any cause can equal the windows PE applications going into totally stupid code paths. Why this comes a major design issue is when Windows PE applications wine is running go out of control because the wineserver is deadlocked.

Minor design change adding a watchdog to detect when wineserver has stalled and to terminate all applications connected to that wineserver is the counter to 99+% of the reported cases of type 3 deadlocking.

Weasel any deadlock mitigation that does not have watchdog of some form is not designed to control deadlocks to the max possible. MS Windows does have a few watchdog against type 2 and 3 deadlocks but they are not perfect.

Wine aiming to be bug to bug compatible with windows it really should have some anti- type 2 and type 3 deadlock watchdog.

Weasel like it or not this is a design flaw. Yes a design flaw because you have failed to account for something in the design.

Getting deadlocked without way out due to OS quirk majority of cases this is a design flaw you forgot in your design to allow that OS kernel can have quirks that result in processes not keeping on getting CPU time. For servers stalled out this way without watchdog is particular bad.

[Weasel]

Not a internal-lock-too-big problem. A completely single-threaded server wouldn't even need internal locks (insofar as really all operations are carried out on that thread only).

That would come into play only if and insofar as some operations are carried out by clients or async callbacks or so (which might be an optimization in an event server of other design than the wineserver, unless those are using so-called lock-free programming.)

ntsync needs locks because those operations within the kernel are multi-threaded (and not using lock-free programming, which would most likely be very difficult in that design)

It's not about internal locks, it just doesn't account for arbitrary limits. The API doesn't specify this, because those are "management" limits so to speak, that can be changed by admins and so on. It's like cgroups limiting your RAM usage despite the max ram being reported higher in a way.

It should gracefully handle it though but it's hard.

[indepe]

It's not about internal locks, it just doesn't account for arbitrary limits. The API doesn't specify this, because those are "management" limits so to speak, that can be changed by admins and so on. It's like cgroups limiting your RAM usage despite the max ram being reported higher in a way.

It should gracefully handle it though but it's hard.

Yes, I know, however that is what the discussion was originally about (when you quoted someone else, but accidentally made it look like I wrote that), at least as I understood it. The wine-server can't deadlock itself with internal locks insofar as it isn't using any itself. It just manages locks on behalf of the Windows-apps. So that might cause the Windows-app threads to deadlock, or more likely, they might lock themselves, but not the wine-server thread.

However ntsync code can theoretically deadlock itself with internal locks (one of them being that "wait-all-lock" which was discussed) because they guard memory that is used by multiple threads inside the Linux kernel ('shared' memory that isn't declared as such because it is used inside the kernel only). When two Windows apps call ntsync functions at the same time, there are two threads inside the Linux kernel using ntsync code. So now the ntsync code could deadlock itself theoretically, using internal locks, if that code uses these internal locks incorrectly.

Probably that sounds more complicated than it is....

EDIT: The reason the same functionality can't deadlock by internal locks, is that the Wine-server's interface serializes the functionalty into a single thread, so it doesn't need locks anymore (at the expense of not being able to use more than one core at the same time, for this functionality).

[Weasel]

Yes, I know, however that is what the discussion was originally about (when you quoted someone else, but accidentally made it look like I wrote that), at least as I understood it. The wine-server can't deadlock itself with internal locks insofar as it isn't using any itself. It just manages locks on behalf of the Windows-apps. So that might cause the Windows-app threads to deadlock, or more likely, they might lock themselves, but not the wine-server thread.

However ntsync code can theoretically deadlock itself with internal locks (one of them being that "wait-all-lock" which was discussed) because they guard memory that is used by multiple threads inside the Linux kernel ('shared' memory that isn't declared as such because it is used inside the kernel only). When two Windows apps call ntsync functions at the same time, there are two threads inside the Linux kernel using ntsync code. So now the ntsync code could deadlock itself theoretically, using internal locks, if that code uses these internal locks incorrectly.

Probably that sounds more complicated than it is....

EDIT: The reason the same functionality can't deadlock by internal locks, is that the Wine-server's interface serializes the functionalty into a single thread, so it doesn't need locks anymore (at the expense of not being able to use more than one core at the same time, for this functionality).

That's totally fine though? ntsync is called by any thread directly that wants to use it, not the wineserver. It's no different than futex2 or any other sync syscall in the kernel in this respect.

[indepe]

That's totally fine though? ntsync is called by any thread directly that wants to use it, not the wineserver. It's no different than futex2 or any other sync syscall in the kernel in this respect.

Yes, I see it as just one of several considerations when comparing alternatives. Personally I like to use multiple (actually many) threads, yet separate them by functionality, so that each functionality can be implemented without using locks. It simplifies the code and it becomes easier to use optimizations. I have encountered situations where it became quite time consuming to avoid internal deadlocks during development, when using locks. However I'd see this as less of a problem for ntsync than for the Wine-server in general, as I would expect it has a large varied interacting complexity. Which is one reason I currently like the idea of a separate event server.