Announcement

Collapse
No announcement yet.

EXT4 File-System Picking Up New Direct I/O Read Implementation

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • EXT4 File-System Picking Up New Direct I/O Read Implementation

    Phoronix: EXT4 File-System Picking Up New Direct I/O Read Implementation

    Among other EXT4 file-system changes en route for the upcoming Linux 5.5 kernel is a new direct I/O read implementation...

    http://www.phoronix.com/scan.php?pag...Direct-IO-Read

  • #2
    Is this an alternative to io_uring, or is this to work in tandem with that?
    If some software want to use this, will they be dependent on ext4, or it just somehow works transparently or how does this work?
    And what kind of applications would benefit from this?

    Comment


    • #3
      Originally posted by uid313 View Post
      Is this an alternative to io_uring, or is this to work in tandem with that?
      If some software want to use this, will they be dependent on ext4, or it just somehow works transparently or how does this work?
      And what kind of applications would benefit from this?
      Yes, that's one of the reasons dropbox dropped support for non-ext4 filesystems on Linux (at least for a while, now they support the most common ones again). They are using filesystem-specific API.

      This specific functionality allows to request reads that bypass the cache, i.e. you are sure you are reading what is actually on disk.

      Comment


      • #4
        What effect will it have on performance?
        What about robustness?

        Comment


        • #5
          Originally posted by Royi View Post
          What effect will it have on performance?
          What about robustness?
          Calling reads directly from storage is slower, but of course more robust as you are sure that this is actually in permanent storage.

          It's slower as you aren't doing read-ahead and storing read material in RAM cache for near-future use. In many cases the system will not just read ONLY the thing you need but a few hundred KB or some MB more so that in the reasonably likely chance that the application asks for it it's already there.

          Comment


          • #6
            Originally posted by Royi View Post
            What effect will it have on performance?
            What about robustness?
            I can see a use case here where the application "knows what it is doing" - e.g. "I need this 1TB file once and only once" and you don't want it being kept in slab cache.

            fairly niche use case - but something I could see happening

            Comment


            • #7
              Originally posted by starshipeleven View Post
              This specific functionality allows to request reads that bypass the cache, i.e. you are sure you are reading what is actually on disk.
              That's what directio does for every filesystem, that supports this (btrfs, ext, jfs, reiserfs, xfs...) It's a common API at VFS-layer. No need for fs-specific APIs.

              Comment


              • #8
                Originally posted by PuckPoltergeist View Post

                That's what directio does for every filesystem, that supports this (btrfs, ext, jfs, reiserfs, xfs...) It's a common API at VFS-layer. No need for fs-specific APIs.
                Hey, Michael needs those fs-specific APIs.

                Us bickering over them helps contribute to his kid's college tuition.

                But, yeah, I can't think of an actual reason other than another file system going "Well, we can do this. Can you?".

                So, like, how long or how many more features until they "git switch -c ext5"?

                Comment


                • #9
                  Originally posted by boxie View Post

                  I can see a use case here where the application "knows what it is doing" - e.g. "I need this 1TB file once and only once" and you don't want it being kept in slab cache.

                  fairly niche use case - but something I could see happening
                  Not that niche.
                  Direct I/O is used by most database applications.
                  They implement their own internal caching/read ahead/durability algorithms and don't rely on OS caching.

                  Comment


                  • #10
                    Originally posted by starshipeleven View Post
                    It's slower
                    nobody would use it if it were slower. it gives control over caching to application, that's all
                    as was said it's mostly for databases, but can also be used for scanning large files and avoiding pushing everything else out of page cache for example

                    Comment

                    Working...
                    X