Google's Gasket Driver Framework Landing For Linux 4.19

[oiaohm]

Where's this code? That would be interesting (it was one of my theoretical assumptions but idk how efficient it is to implement).

Sorry, I'm not going to look for code that may not exist. You claimed it does, so show me where it is (line of code or just link to a mirror, i don't care).

Or at least link to the article that taught you this, I'm sure it will have references to the code. I mean, where the heck did you get this information from about registering the futex? Just link it then?

Because it was not a article when you get to sit down with the developer who made that code and have them walk you though how it works you know quite a few things. To understand it you in fact need all the different bits of code. Bits from the kernel bits from the glibc.

These complex things are never just point to a single comment or single line of code. You have set in one location and picked up in another. The fact that the set can be set by syscall and by writing in memory it is kind of complex. This is in fact worse than getting you head around how RCU works. RCU I could point you to a nice detail artical how it works that was only written like 15 years after Linux kernel started using it. Futex is too young to have great documentation. When you are using futex and performance profiling you have to understand it or you start miss reading your profiling.

[Hi-Angel]

oiaohm just wanted to thank you for the comments (though granted, I didn't read the 5-th and 6-th pages). I didn't know how in-kernel mutices as well as futices are implemented, it was an interesting read.

Your comments are always an interesting read ☺

[Weasel]

oiaohm just wanted to thank you for the comments (though granted, I didn't read the 5-th and 6-th pages). I didn't know how in-kernel mutices as well as futices are implemented, it was an interesting read.

Your comments are always an interesting read ☺

In case you're serious: unfortunately a lot of times he has no clue what he's talking about. He doesn't even know how to code to begin with (he claimed so himself). One of the reasons I even argue with him in the first place, since it's public and misleading, but sometimes he's indeed right though, so I've nothing personal or such. Doesn't seem like a bad guy, just sometimes goes into things way out of his league.

...and no I don't want this to be a flame bait so it's my only reply to that context.

[Hi-Angel]

In case you're serious: unfortunately a lot of times he has no clue what he's talking about. He doesn't even know how to code to begin with (he claimed so himself). One of the reasons I even argue with him in the first place, since it's public and misleading, but sometimes he's indeed right though, so I've nothing personal or such. Doesn't seem like a bad guy, just sometimes goes into things way out of his league.

...and no I don't want this to be a flame bait so it's my only reply to that context.

It's always hard to argue on the internet, because the other side might bear in mind something you don't see, and then confusion arises when both sides think they understand each other, though actually neither of them does.

When this happens, it's likely that both sides think the other one is stupid or something. It's a displeasure to participate in such discussion, however it's an interesting read for a bystander, because lots of useful info usually arises.

I've noticed a similar pattern at your discussion somewhere at starting few pages, and simply stopped following it. Instead I've cursory read through oiaohm's explanations and docs on mutices and futices, since they were pretty elaborate — and it does make sense to me. I do admit a possibility there's something wrong, like, maybe there's some other way to implement them, which is used somewhere — but as far as generic explanation on a possible implementation goes, it seems okay to me, and it was interesting.

[Weasel]

It's always hard to argue on the internet, because the other side might bear in mind something you don't see, and then confusion arises when both sides think they understand each other, though actually neither of them does.

When this happens, it's likely that both sides think the other one is stupid or something. It's a displeasure to participate in such discussion, however it's an interesting read for a bystander, because lots of useful info usually arises.

I've noticed a similar pattern at your discussion somewhere at starting few pages, and simply stopped following it. Instead I've cursory read through oiaohm's explanations and docs on mutices and futices, since they were pretty elaborate — and it does make sense to me. I do admit a possibility there's something wrong, like, maybe there's some other way to implement them, which is used somewhere — but as far as generic explanation on a possible implementation goes, it seems okay to me, and it was interesting.

The thing is, what sounds interesting doesn't necessarily make it a fact. Sometimes boring is how stuff actually works. (well I guess "boring" and "interesting" are opinions anyway, can't really argue with that)

I don't know why I hadn't thought of simply linking wikipedia before when I was arguing with him: https://en.wikipedia.org/wiki/Futex

All you need to read is the beginning description and particularly the last phrase there (emphasis mine):

A properly programmed futex-based lock will not use system calls except when the lock is contended; since most operations do not require arbitration between processes, this will not happen in most cases.

But I guess that's too simple and not interesting enough. ;-)

[oiaohm]

But I guess that's too simple and not interesting enough. ;-)

Wikipedia did not tell you what those syscalls do or what current glibc and kernel do or how the arbitration is performed. You read the wikipedia and presumed you knew how it worked. There is a structure shared between glibc and the kernel for arbitration usage. What about the case of a lock being repeatedly contended do you want to syscall each time or do you syscall once to allocate the status structure and reuse next time your code is needing that lock.

Weasel you presume that the syscall is a yield when a syscall is used when with a futex that is not the case.

[Weasel]

Wikipedia did not tell you what those syscalls do or what current glibc and kernel do or how the arbitration is performed. You read the wikipedia and presumed you knew how it worked. There is a structure shared between glibc and the kernel for arbitration usage. What about the case of a lock being repeatedly contended do you want to syscall each time or do you syscall once to allocate the status structure and reuse next time your code is needing that lock.

Weasel you presume that the syscall is a yield when a syscall is used when with a futex that is not the case.

No, every programmer that does any sort of multi-threading knows how a futex works in theory (and even implemented their own). I didn't "read Wikipedia" since I literally just found out about that right now, else I would've linked it to you far earlier.

Why does it matter what a syscall does? A syscall means a context switch and that's all there is to it. It doesn't matter what it does. And yes, it actually does the yielding since it's only the kernel which can put threads to sleep, but that doesn't matter and I gave up trying to convince you of that fact.

I never said you repeatedly call into the kernel to initialize the mutex. You do that once. But lock being contended means that you have to wait, and to avoid a spinlock / busy waiting loop, you need to go to sleep. And the kernel is the only one who can put threads to sleep.

[oiaohm]

No, every programmer that does any sort of multi-threading knows how a futex works in theory (and even implemented their own). I didn't "read Wikipedia" since I literally just found out about that right now, else I would've linked it to you far earlier.

Why does it matter what a syscall does? A syscall means a context switch and that's all there is to it. It doesn't matter what it does. And yes, it actually does the yielding since it's only the kernel which can put threads to sleep, but that doesn't matter and I gave up trying to convince you of that fact.

But its not the syscall that does the yield. Its the fact the syscall registered information that scheduler latter uses to perform the yield. glibc is able to reuse the allocated structure for a later usage of the same lock in the code.

Its the kernel sheduler that has to do the yield and has to know it has to yield. There is no need need to create a scheduler notification structure if you acquired the lock and once the notification structure exists there is only a need to update not create another one.

With a futex you can decide to create notification structure before you use a futex. Do note the wikipedia said properly programmed you don't have to properly program a futex. So you can do your syscalls before you find out if futex is contended or not.

How the syscall works makes the difference between having to syscall every time you have the lock contended or only have to syscall the first time the lock is contented and recycle the structure after that.

And yes, it actually does the yielding since it's only the kernel which can put threads to sleep
The syscall creates a means to have a notification to yield not the yield or directly triggering the process to yield. Yes the kernel is the only thing that can do a yield how it notified can be done a few different ways.

[Weasel]

But its not the syscall that does the yield. Its the fact the syscall registered information that scheduler latter uses to perform the yield. glibc is able to reuse the allocated structure for a later usage of the same lock in the code.

Arguing semantics?

The kernel does the yield and puts the thread to sleep.
The syscall is communication with the kernel to let it know to do that.

And that needs a context switch no matter how you look at it.

[oiaohm]

The syscall is communication with the kernel to let it know to do that..

There is more than 1 communication path to kernel.

https://www.kernel.org/doc/Documentation/circular-buffers.txt

Circular buffers is kind of like the stunt futex on Linux is upto.

You need a syscall to allocate shared memory between kernel space and userspace but once that is set up you can put many yield messages back in the same allocation. Same reason logs in the linux kernel is in circular buffer if it was a syscall every time logging program need to get log results you would have way too many context switches.

And that needs a context switch no matter how you look at it.

This is wrong. You might need to-do a context switch. So you set that thread need to yield you wait in a spin lock if lock comes acquirable you cancel the yield message.

Calling a syscall to yield has already cost you a context switch and their is no way to back out of it if the lock comes acquirable before allocated time slice is up.

Messaging/share memory between kernel and application has different behaviour to syscall(interrupts and traps).

When I say registering information its registering for where the scheduler should look with the thread to see if there is a lock issue. So basically futex has used a syscall when a yield could be required to set up a message system between application and scheduler.

So you have two major ways to implement a yield event when you design operating systems. Shared memory message system to kernel or Syscall to kernel. Syscalls are for events that need to happen instantly so cause context switch straight away where a shared memory message system are events that program can keep on running but the kernel need to know about/update at normal scheduler interruptions. Feature of shared memory message systems is that you can cancel the message to kernel as long as the scheduler/other party has not kicked in yet. Feature of a syscall is when you have done the syscall there is no cancelling it.

Yes there are some operating systems out there that don't have syscalls all they have is a shared memory message bus and they do everything over it.

syscall means system call right. Call think pick up phone and talk straight to the person on the other end syscall you have to context switch to kernel because syscall is like the person on the other end has to answer straight away. Shared memory message is more like sms where the kernel or other things in the shared memory message don't have to answer straight away or respond straight away. How instant does the yield when you are sitting on a spinlock really need to be futex suggest not that instant so shared memory message seams good enough.