Linux Work Culminating On A "READFILE" Syscall For Reading Small Files Efficiently

[rene]

How is that different than io_uring?

certainly much simpler when you look at the io_uring code, synchronous, and last but not least work at all, as uring currently does not allow to re-use return values of previous syscalls, e.g. opened file-descriptors for the next systemically.

[coder]

Instead what would be amazing would be vectored / bundled system calls that would also be useful for actual high performance stuff

How is that different than io_uring?

[rene]

IMHO this is really bad system call and kernel ABI design. Pointless endeavour for non mition critical / high performance use case. Instead what would be amazing would be vectored / bundled system calls that would also be useful for actual high performance stuff, you know databases, fileserver and fun stuff like that: https://www.youtube.com/watch?v=84Uyh3KwY-w

[zyxxel]

There are many (really MANY) more indirect dispatches involved during the processing of a single call. VFS, LSM, namespace resolution are but a few that pop up on my mind. Adding another one makes exactly zero measurable difference in processing performance but if it means having a cleaner and more coherent kernel interface I'm all for it.

So if you go from 50 to 51 dispatches, that's still not zero cost. It's a 2% additional overhead, wasted just for the heck of it.

[markg85]

This is a wrapper over splice and doesn't work the way I want. You need to wait on in_fd and trigger the transfer for every batch of data (with unnecessary user-kernel trampoline every time), in_fd can't be a socket etc. It works for what it's intended to do, which is a file server, but it doesn't work for something like an on-demand tunnel manager. What I would like is to be able to "attach" an input file descriptor to an output one and have all data simply pass through (for example from socket to socket, or from local socket to inet socket etc.) with no further intervention.

I see what you mean. I'd like to have that feature too!

[Weasel]

+1 to Justin.

Can others stfu about ioctls already? They operate on file descriptors. That means you need to open(), do the ioctl() then close(). What's the fucking difference compared to a normal read?

The whole point of this syscall is to avoid the open() and close() syscalls.

[JustinTurdeau]

There are many (really MANY) more indirect dispatches involved during the processing of a single call. VFS, LSM, namespace resolution are but a few that pop up on my mind. Adding another one makes exactly zero measurable difference in processing performance

This is just the "computers are fast, so who gives a shit" argument.

All those other things actually have a purpose and provide some kind of value, whereas pointlessly doing this the slow and ugly way because "meh, some other things are slow too" is not a very engineer-like mindset. I would expect that kind of thing from schmuck JavaScript "developers", not kernel hackers.

[joshuarli]

Pretty excited about this; io_uring and readfile combined is going to significantly speedup anything that heavily reads from things like procfs. I'm currently in the early stages of implementing a stripped-down top clone in Rust for fun, and can't wait to use those technologies.

[jacob]

Right, but ioctls are essentially doing double dispatch. Once for the ioctl syscall itself and then again depending on the arguments. Why do double dispatch for no reason? Just to be contrarian?

There are many (really MANY) more indirect dispatches involved during the processing of a single call. VFS, LSM, namespace resolution are but a few that pop up on my mind. Adding another one makes exactly zero measurable difference in processing performance but if it means having a cleaner and more coherent kernel interface I'm all for it.

[kreijack]

ioctls are also implemented in drivers rather than in the VFS, so ignoring the reality that it is not possible to do this via an ioctl (possibly via a device node in /dev) and somehow doing it anyway would be a mess. You really want an ioctl when controlling a device, not when doing a VFS operation.

There are ioctl also for filesystem: for example look at fiemap ioctl, or at the XFS/BTRFS ones.. The former is generic (even tough the filesystem has to provide the fiemap() inode method) the latter are filesystem specific.
Even tough nobody likes the ioctl, there are a lot of cases where an ioctl is the unique way to go...