Uncached Buffered I/O Aims To Be Ready For Linux 6.14 With Big Gains

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • MorrisS.
    replied
    IO_uring improvements work or not for end-users?

    Leave a comment:


  • hwertz
    replied
    Originally posted by coder View Post
    RWF_UNCACHED gives you the benefit of the kernel doing writeback, like it normally does with cached I/O, but the blocks just won't stick around in the cache once they're no longer dirty.
    All the better, given the massive speed benefit that can be had just by combining a few smaller writes into a larger one and all that.

    Leave a comment:


  • coder
    replied
    Originally posted by hwertz View Post
    ...
    Yeah, this was explained in an earlier article. Another big caveat of O_DIRECT than the ones you mentioned is that it's not supported by all filesystems and one big omission is NFS (not sure if this was patched, but it traditionally couldn't do O_DIRECT).

    O_DIRECT also means you need to use AIO or io_uring, in order to avoid blocking writes. I'm not sure, but I doubt the kernel's readahead optimization works with it, since I assume that just populates the page cache. So, yeah, it's a pain to use.

    RWF_UNCACHED gives you the benefit of the kernel doing writeback, like it normally does with cached I/O, but the blocks just won't stick around in the cache once they're no longer dirty.

    Leave a comment:


  • hwertz
    replied
    So to elaborate slightly, O_DIRECT does direct I/O, it reads/writes straight to/from the file or disk, bypassing the page cache (disk cache) entirely. But it's a bit "too direct", your memory buffer usually has to be page-aligned (on Intel systems a page is 4KB), and (apparently filesystem-depdendent..) your reads and writes often have to be page-sized. And there's complications if you have multiple programs accessing that file (if one is O_DIRECT so it's reads and writes are bypassing the cache, and one is using regular I/O so it has it's data going into the page cache... well you can see how that could potentially run into problems.) So it's a bit of a PITA to use.

    So this uncached buffered I/O avoids these issues, you can just set this flag and carry on like you were doing regular I/O, while still having the data not crap up your cache.
    Last edited by hwertz; 26 December 2024, 12:38 AM.

    Leave a comment:


  • Kjell
    replied
    Originally posted by coder View Post
    This. It's for the type of apps which already might've been using O_DIRECT (or wanted to, if not for some of its unfortunate side-effects and complications).

    Most userspace apps should continue to use buffered I/O, as normal. What end users can do to tune their system's use of the block cache is to play with the parameters the kernel already exposes, such as swapiness and different I/O schedulers.
    QEMU-IO is the first thing that comes to mind, it can be used with io_uring. E.g. -drive aio=io_uring. I wonder if this patch helps this usecase

    Leave a comment:


  • coder
    replied
    Originally posted by stormcrow View Post
    End users don't. It's a set of system call options meant for those kinds of applications with IO patterns that might benefit. If you don't know what the difference is between a cache and a buffer then this isn't going to educate you nor is this option for you.
    This. It's for the type of apps which already might've been using O_DIRECT (or wanted to, if not for some of its unfortunate side-effects and complications).

    Most userspace apps should continue to use buffered I/O, as normal. What end users can do to tune their system's use of the block cache is to play with the parameters the kernel already exposes, such as swapiness and different I/O schedulers.

    Leave a comment:


  • stormcrow
    replied
    Originally posted by Kjell View Post
    How can you enable this to reap the benefits?
    End users don't. It's a set of system call options meant for those kinds of applications with IO patterns that might benefit. If you don't know what the difference is between a cache and a buffer then this isn't going to educate you nor is this option for you.

    Using it from applications is trivial - just set RWF_DONTCACHE for the read or write, using pwritev2(2) or preadv2(2). For io_uring, same thing, just set RWF_DONTCACHE in sqe->rw_flags for a buffered read/write operation. And that's it.

    Leave a comment:


  • JEBjames
    replied
    Michael

    typo in link

    "fpr big speed improvements" should be "for"

    Leave a comment:


  • pWe00Iri3e7Z9lHOX2Qx
    replied
    Originally posted by Kjell View Post
    How can you enable this to reap the benefits?
    Sounds like user space applications have to specifically ask for reads / writes of this type.

    Leave a comment:


  • Guest
    Guest replied
    these are minor changes that do not change the linux I/O concept itself. but if, as I've said more than once, the Linux kernel is 30 years old, then the concept definitely needs to be revised. I/O best to transfer it to the GPU first. There are more opportunities on the GPU to implement something new, thanks to the threads and the blocks in which these threads fit.
    at least from the point of view of programming for Nvidia GPUs. but it seems to me that most Linux users use just these GPUs.
    Last edited by Guest; 24 December 2024, 09:35 AM.

    Leave a comment:

Working...
X