Uncached Buffered I/O Aims To Be Ready For Linux 6.14 With Big Gains

[phoronix]

Phoronix: Uncached Buffered I/O Aims To Be Ready For Linux 6.14 With Big Gains

Linux I/O expert and storage expert Jens Axboe of Meta is hoping to have the uncached buffered I/O support squared away for Linux 6.14 -- a feature that's been a half-decade in the making...

https://www.phoronix.com/news/Uncached-Buffered-IO-Linux-6.14

[Kjell]

How can you enable this to reap the benefits?

[pWe00Iri3e7Z9lHOX2Qx]

How can you enable this to reap the benefits?

Sounds like user space applications have to specifically ask for reads / writes of this type.

[JEBjames]

Michael

typo in link

"fpr big speed improvements" should be "for"

[stormcrow]

How can you enable this to reap the benefits?

End users don't. It's a set of system call options meant for those kinds of applications with IO patterns that might benefit. If you don't know what the difference is between a cache and a buffer then this isn't going to educate you nor is this option for you.

Using it from applications is trivial - just set RWF_DONTCACHE for the read or write, using pwritev2(2) or preadv2(2). For io_uring, same thing, just set RWF_DONTCACHE in sqe->rw_flags for a buffered read/write operation. And that's it.

[coder]

End users don't. It's a set of system call options meant for those kinds of applications with IO patterns that might benefit. If you don't know what the difference is between a cache and a buffer then this isn't going to educate you nor is this option for you.

This. It's for the type of apps which already might've been using O_DIRECT (or wanted to, if not for some of its unfortunate side-effects and complications).

Most userspace apps should continue to use buffered I/O, as normal. What end users can do to tune their system's use of the block cache is to play with the parameters the kernel already exposes, such as swapiness and different I/O schedulers.

[Kjell]

This. It's for the type of apps which already might've been using O_DIRECT (or wanted to, if not for some of its unfortunate side-effects and complications).

Most userspace apps should continue to use buffered I/O, as normal. What end users can do to tune their system's use of the block cache is to play with the parameters the kernel already exposes, such as swapiness and different I/O schedulers.

QEMU-IO is the first thing that comes to mind, it can be used with io_uring. E.g. -drive aio=io_uring. I wonder if this patch helps this usecase

[hwertz]

So to elaborate slightly, O_DIRECT does direct I/O, it reads/writes straight to/from the file or disk, bypassing the page cache (disk cache) entirely. But it's a bit "too direct", your memory buffer usually has to be page-aligned (on Intel systems a page is 4KB), and (apparently filesystem-depdendent..) your reads and writes often have to be page-sized. And there's complications if you have multiple programs accessing that file (if one is O_DIRECT so it's reads and writes are bypassing the cache, and one is using regular I/O so it has it's data going into the page cache... well you can see how that could potentially run into problems.) So it's a bit of a PITA to use.

So this uncached buffered I/O avoids these issues, you can just set this flag and carry on like you were doing regular I/O, while still having the data not crap up your cache.

[coder]

...

Yeah, this was explained in an earlier article. Another big caveat of O_DIRECT than the ones you mentioned is that it's not supported by all filesystems and one big omission is NFS (not sure if this was patched, but it traditionally couldn't do O_DIRECT).

O_DIRECT also means you need to use AIO or io_uring, in order to avoid blocking writes. I'm not sure, but I doubt the kernel's readahead optimization works with it, since I assume that just populates the page cache. So, yeah, it's a pain to use.

RWF_UNCACHED gives you the benefit of the kernel doing writeback, like it normally does with cached I/O, but the blocks just won't stick around in the cache once they're no longer dirty.

[hwertz]

RWF_UNCACHED gives you the benefit of the kernel doing writeback, like it normally does with cached I/O, but the blocks just won't stick around in the cache once they're no longer dirty.

All the better, given the massive speed benefit that can be had just by combining a few smaller writes into a larger one and all that.