The FUTEX2 System Call Continues Working Its Way Towards Mainline In 2021

[F.Ultra]

Right, it's shared memory anyway. And according to your logic, shared in userspace.

Tried where, when and how? Except for "fsync", I only know of the attempt to use eventfd, which is something completely different. I believe I have read everything pointed out in previous phoronix articles, and some sources of "esync", and I don't know of any extensive attempt based on userspace and existing futex API. As far as I know, "fsync" is trying to use the kernel patches called "futex2". That's also how it appeared in the email proposal we discussed a few days ago: there, managing-a-waitlist appeared to be something that wasn't even seriously considered. Although that's perhaps a separate effort.

Why would it would it not be a viable solution? And how would it get more viable by doing it inside the kernel? I have implemented similar things in userspace, and they work just fine. Except for the wait-for-all-signals-at-the-same-time option, but with more functionality in other regards. And also another function with wait-for-all, but not necessarily at-the-same-time. So that's a very weird claim to me. Even doesn't require locking during normal operation, which in my case is the permanent subscription use case. To me, that's like denying the obvious. I'm like "what the heck is their problem, and why don't they say what it is?". In another discussion I heard a reason is that issuing multiple wake calls would be a performance problem. That can't be a big problem if the wake calls are only done for waiters actually waiting (which is an expensive operation anyway), and as I have written previously, that could be addressed with a simple and general purpose wake-multiple syscall which would be trivial to implement since it would basically just be a loop over existing functionality. In other words, I have a hard time thinking of anyone as knowing what they are talking about if/when they make such claims.

Yes the futex integer is shared in userspace, I'm not talking about the syscall but the actual integer. But before you get more upset, please remember that I only wrote that I would find a waitv() syscall useful, I never wrote that the sky would fall without it. Having pthread_create() is nice too but I could also live without it.

esync is WINE building a userspace broker using eventfd, fsync is WINE building the userspace broker using futex instead. The futex2 patches is how the WINE devs thinks that they can (among other things) enhance the performance of their fsync solution by moving the broker into the kernel.

Either you are underestimating the problem that the WINE devs are trying to solve or they have no clue what they are doing. That is not me being salty, that is just me stating where things stand, I have not enough information to make a valid judgement in either direction, all I know is that you claim that this is dead easy and they are claiming that it's infeasible.

[indepe]

Yes the futex integer is shared in userspace, I'm not talking about the syscall but the actual integer. But before you get more upset, please remember that I only wrote that I would find a waitv() syscall useful, I never wrote that the sky would fall without it. Having pthread_create() is nice too but I could also live without it.

Are you saying it would be useful at the application level to have a good library for such functionality? Or are you actually talking about the futex level, low level functionality?

esync is WINE building a userspace broker using eventfd, fsync is WINE building the userspace broker using futex instead. The futex2 patches is how the WINE devs thinks that they can (among other things) enhance the performance of their fsync solution by moving the broker into the kernel.

I believe you are mistaken. (But perhaps they have an internal test version, or so.)
Here a quote from a previous thread, which pretty much reflects my general impression:

There's no fsync kernel patchset, fsync is a wine thing. There's also no fsync2 anything.

The Linux kernel has long had a futex syscall, but it's not enough for what Windows API needs. So the "fsync" patchset on wine depends on a new syscall, called futex2, which extends it so it can better match Windows.

tl;dr: futex ≠ fsync

In any case, I would think that if such an implementation exists, using the existing futex API, that it is not (or wasn't) sufficiently optimized to be representative.

Either you are underestimating the problem that the WINE devs are trying to solve or they have no clue what they are doing. That is not me being salty, that is just me stating where things stand, I have not enough information to make a valid judgement in either direction, all I know is that you claim that this is dead easy and they are claiming that it's infeasible.

It is not that easy at all. However it is not easier if you do it inside the kernel. Especially if you attempt to replace the whole FUTEX system from scratch, while you are at it. Just for giggles, so to speak

[F.Ultra]

Are you saying it would be useful at the application level to have a good library for such functionality? Or are you actually talking about the futex level, low level functionality?

I was simply replying to a poster here asking if there could be a use case for futex2 outside the WINE one, and there is.

I believe you are mistaken. (But perhaps they have an internal test version, or so.)

Sorry about that, there have been so many back and forth with Proton and kernel patches, yes fsync requires a kernel patch but it's not the futex2 patch that this article is about, the patch it requires is very small and does not add a new syscall but a new FUTEX_WAIT_MULTIPLE flag https://lkml.org/lkml/2019/7/30/1399

Then later last year the same devs proposed a new syscall instead (sys_futex_waitv) and that is the futex2 patches https://lore.kernel.org/lkml/2020061...collabora.com/

It is not that easy at all. However it is not easier if you do it inside the kernel. Especially if you attempt to replace the whole FUTEX system from scratch, while you are at it. Just for giggles, so to speak

True but the WINE devs (or rather Proton devs) that have proposed to add futex2 to the kernel is also the very same devs that have written the patch so it's the one writing the actual code that claims that it works better in the kernel than in userspace.

My guess is that they want it in the kernel so that a producer can wake up several consumers with a single syscall instead of having a userspace broker having to make a futex() call for each and every thread that is waiting on the futex they are holding.

[pal666]

On that same note, does futex2 bring something to the table outside of Wine?

waiting on multiple mutexes, according to article. you can multiplex file descriptors, but you can't multiplex mutexes. i'd like to be able to mix and match them, i.e. to add mutex requests to epoll

[pal666]

There are many ways in which threads can wait for multiple things to happen, it doesn't require any syscall at all. Unless the thread has to actually wait, in which case the existing futex syscall (called directly or indirectly) does just fine. Why would you need to wait specifically for multiple futexes, as opposed to multiple events or multiple notifications?

because there are two ways you could wait - on fd and on condition variable. to wait for any of thousand fds you just add them to epoll. to wait for any of thousand condition variables you just spawn thousand threads each waiting on one. later is clearly suboptimal

[pal666]

Are you saying the queues make a FUTEX_WAKE call each time the state of the queue changes to non-empty? This would seem extremely inefficient and unnecessary to me.

as long as there's sleeping waiter. how in the hell you'll wake him up otherwise?

[pal666]

they are used to implement application level features. And that can be done with the existing futexes.

you can't wait for many futexes with existing syscall and one thread because existing syscall only accepts one futex and doesn't return until wait is over. you can wait by spawning many threads, but it's stupid

[pal666]

In other words, under the hood the Windows implementation could theoretically be more like a userspace implementation using the existing futex API, just as a low level API within the kernel. Theoretically.

with awful performance, but why would you ask for that?

[pal666]

If so, then they already need an interchange between waiter and sender in userspace. At which point they could also, instead of burdening the kernel with that, add the waiter to a list

now you have to use your brain to understand that kernel takes part in this "interchange" and you can't change it without changing kernel. exactly what this patch is doing

Done. No kernel patch necessary, you could run it on any kernel you want, yesterday.

in some alternative universe where sleeping and waking does not require kernel help

[pal666]

Tried where, when and how? Except for "fsync", I only know of the attempt to use eventfd, which is something completely different.

tried by wine upstream. they are inventing fsync, esync and so on in search for a replacement for current solution exactly because current solution sucks.

I believe I have read everything pointed out in previous phoronix articles, and some sources of "esync", and I don't know of any extensive attempt based on userspace and existing futex API.

what are you trying to say? current wine crashes when program tries to wait for multiple objects?

Why would it would it not be a viable solution?

because you wouldn't provide patch