Uncached Buffered I/O Aims To Be Ready For Linux 6.14 With Big Gains
Collapse
X
-
Originally posted by coder View PostRWF_UNCACHED gives you the benefit of the kernel doing writeback, like it normally does with cached I/O, but the blocks just won't stick around in the cache once they're no longer dirty.
Leave a comment:
-
-
Originally posted by hwertz View Post...
O_DIRECT also means you need to use AIO or io_uring, in order to avoid blocking writes. I'm not sure, but I doubt the kernel's readahead optimization works with it, since I assume that just populates the page cache. So, yeah, it's a pain to use.
RWF_UNCACHED gives you the benefit of the kernel doing writeback, like it normally does with cached I/O, but the blocks just won't stick around in the cache once they're no longer dirty.
Leave a comment:
-
-
So to elaborate slightly, O_DIRECT does direct I/O, it reads/writes straight to/from the file or disk, bypassing the page cache (disk cache) entirely. But it's a bit "too direct", your memory buffer usually has to be page-aligned (on Intel systems a page is 4KB), and (apparently filesystem-depdendent..) your reads and writes often have to be page-sized. And there's complications if you have multiple programs accessing that file (if one is O_DIRECT so it's reads and writes are bypassing the cache, and one is using regular I/O so it has it's data going into the page cache... well you can see how that could potentially run into problems.) So it's a bit of a PITA to use.
So this uncached buffered I/O avoids these issues, you can just set this flag and carry on like you were doing regular I/O, while still having the data not crap up your cache.Last edited by hwertz; 26 December 2024, 12:38 AM.
Leave a comment:
-
-
Originally posted by coder View PostThis. It's for the type of apps which already might've been using O_DIRECT (or wanted to, if not for some of its unfortunate side-effects and complications).
Most userspace apps should continue to use buffered I/O, as normal. What end users can do to tune their system's use of the block cache is to play with the parameters the kernel already exposes, such as swapiness and different I/O schedulers.
Leave a comment:
-
-
Originally posted by stormcrow View PostEnd users don't. It's a set of system call options meant for those kinds of applications with IO patterns that might benefit. If you don't know what the difference is between a cache and a buffer then this isn't going to educate you nor is this option for you.
Most userspace apps should continue to use buffered I/O, as normal. What end users can do to tune their system's use of the block cache is to play with the parameters the kernel already exposes, such as swapiness and different I/O schedulers.
Leave a comment:
-
-
Originally posted by Kjell View PostHow can you enable this to reap the benefits?
Using it from applications is trivial - just set RWF_DONTCACHE for the read or write, using pwritev2(2) or preadv2(2). For io_uring, same thing, just set RWF_DONTCACHE in sqe->rw_flags for a buffered read/write operation. And that's it.
Leave a comment:
-
-
Originally posted by Kjell View PostHow can you enable this to reap the benefits?
Leave a comment:
-
-
Guest repliedthese are minor changes that do not change the linux I/O concept itself. but if, as I've said more than once, the Linux kernel is 30 years old, then the concept definitely needs to be revised. I/O best to transfer it to the GPU first. There are more opportunities on the GPU to implement something new, thanks to the threads and the blocks in which these threads fit.
at least from the point of view of programming for Nvidia GPUs. but it seems to me that most Linux users use just these GPUs.Last edited by Guest; 24 December 2024, 09:35 AM.
Leave a comment:
-
Leave a comment: