Announcement

Collapse
No announcement yet.

UFS File-Based Optimization Patches For Linux: Shot Down As "Complete & Utter Madness"

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • UFS File-Based Optimization Patches For Linux: Shot Down As "Complete & Utter Madness"

    Phoronix: UFS File-Based Optimization Patches For Linux: Shot Down As "Complete & Utter Madness"

    JEDEC recently outlined an extension to Universal Flash Storage (UFS) for File-Based Optimizations (FBO) to enhance the performance of UFS devices. A Xiaomi engineer sent out a set of Linux kernel patches for implementing UFS FBO in the name of better performance, but with almost immediate rejection by a veteran Linux kernel maintainer...

    https://www.phoronix.com/news/UFS-Fi...d-Optimization

  • #2
    But wait, isn't JEDEC that proposed this stupid thing?

    Comment


    • #3
      and the it
      I wonder what was there before the "it".

      Comment


      • #4
        Haven't BSD guys hired author of ReiserFS to do the right thing? You want order in filesystem names and acronyms.

        Comment


        • #5
          Originally posted by Danny3 View Post
          But wait, isn't JEDEC that proposed this stupid thing?
          Going by the title ( "Universal Flash Storage (UFS) File Based Optimizations (FBO) Extension, Version 1.0" ), yes. It seems the the specs do indeed call for the host to be file-aware instead of just a dumb addressing space and the Xiaomi engineer simply implemented it per the specs. That said, the maintainer might mean it should be done by a user land, root privileged, service rather than the kernel like how continues de-fragmentation is done in Windows.

          Comment


          • #6
            Originally posted by c117152 View Post

            Going by the title ( "Universal Flash Storage (UFS) File Based Optimizations (FBO) Extension, Version 1.0" ), yes. It seems the the specs do indeed call for the host to be file-aware instead of just a dumb addressing space and the Xiaomi engineer simply implemented it per the specs. That said, the maintainer might mean it should be done by a user land, root privileged, service rather than the kernel like how continues de-fragmentation is done in Windows.
            Yeah, this really looks like a job for fstrim.

            Comment


            • #7
              Heh. That reads like a comment I'd write when pissed about something

              Comment


              • #8
                Problem here is turtles all the way down kind of problem.

                LBA with flash storage is abstraction with wear leveling and the like. Yes you have fragmentation inside file system and you have fragmentation at the flash storage level with the transform from LBA to real flash block storage. Yes the LBA to flash storage stuff being fragmented can be causing performance overhead.

                This UFS FBO design has some in face faults.

                First question why needing multi LBA ranges for a file for performance. A de-fragmented file that is going to have effective OS operations is going to be a single LBA range if you are not defraging the inode entries as well yes the UFS FBO proposal is not defraging the inode entries and the like.

                Sending LBA information to the storage device to defragment storage side as part of the file system defragmentation would make sense as option. Doing storage side defragmentation while the file in the file system by LBA address is still fragmented in majority of cases is going to be wrong. Think about it you are doing DMA reads from storage device continuous LBA ranges of files make this more effective.

                The problem with UFS FBO is it a half done solution. File system fragmentation side can in inodes and other file system structures that are not file and file system can be defragmenting them all at as LBA ranges when they are continuous.

                This is a true multi level problem. Also remember auto defragment in lots file systems will notice that file is using two or more lba ranges then notice hey we have a open lba range over here large enough to take the complete file so move the file into that. Also notice that inodes of a directory are spread all over the place and move them back into groups.

                This UFS FBO stuff I really see as should be a block level feature that any file system defragmentation system on flash storage can use when even it has solved a items to a single lba range yes this could be files or could be collections of inodes that make up directories.

                Notice I said single lba range. this UFS FBO implementation has sending across multi LBA ranges per file that means the file is fragmented so will be defragmented and moved in future and you have just wasted writes defragmenting something at the device level that will have to be moved in future to get the LBA range down to a single LBA range so its defragmented at the file system level. Basically the implementation does not make sense.

                The problem here you file system level fragmentation and the device storage level fragmentation.

                The file system level fragmentation will show up at the file and other items being stored across multi LBA ranges when the item could be stored as a single LBA range.

                Device storage fragmentation is in the translation from LBA to physical storage.

                This is also like saying putting the horse before the cart. Have the device sort out device level defragmentation before sorting out file system fragmentation is doing things in the wrong order. File system defragmentation to reduce the LBA ranges need to be performed before performing the device level defragmentation.

                JEDEC over the years has proposed many completely stupid things. This case there is a valid need to perform device level defragmentation the stupid thing here is the proposal at JEDEC was kind of designed to operate in isolation without considering how filesystems do in fact need to operate with LBA ranges usage for effective DMA operations. This is what is leading to the straight up NAK.

                Yes what happened is someone who understands the hardware level and has a light understanding of the file system level has proposed what they think is a solution at JEDEC. Yes it has something that need to be done. Problem here it overlooked key item how file systems need to operate when using LBA using storage devices to be CPU operation effective.

                There is more than 1 performance trap here. Solving the performance traps here in the wrong order is absolutely not ideal.

                Comment


                • #9
                  Originally posted by willmore View Post
                  Yeah, this really looks like a job for fstrim.
                  fstrim only tell what blocks are not in use. Does not tell what blocks are likely to be read as a single group. There is a need to be able to give a little bit more information to flash storage devices to allow more ideal placement.

                  But its also like fstrim . You don't want to perform a trim right if the area has not be released from use from the file system right. This case you don't want to be performing a block device level defragmentation/optimisation if the file is still able to be de-fragmented at the file system level. Basically there is a order of operation here that has been over looked.

                  Comment


                  • #10
                    Originally posted by oiaohm View Post

                    fstrim only tell what blocks are not in use. Does not tell what blocks are likely to be read as a single group. There is a need to be able to give a little bit more information to flash storage devices to allow more ideal placement.

                    But its also like fstrim . You don't want to perform a trim right if the area has not be released from use from the file system right. This case you don't want to be performing a block device level defragmentation/optimisation if the file is still able to be de-fragmented at the file system level. Basically there is a order of operation here that has been over looked.
                    Currently fstrim only does that, but there's no reason it (or a very similar program) couldn't do this task. You'd want to fstrim before performing this file level 'defragmentation' anyway to give the flash controller as much free space to work with as possible. Then you'd just need to walk the inodes and give the device a list of LBAs associated with each file.

                    It would be ironic if the filesystem had fragmentation but the physical storage for files ended up being contiguous.

                    Since the FTL already has a block rename ability, what I'd like to see is a stage where some user space tool could walk the inodes and 'rename' blocks. For example, if you had a file using blocks 2-4, and 6 (and 5 was now unallocated), you could tell the FTL to just swap 5 and 6. Now, at the FS level the allocation would be 2-5 and all the FTL had to do is swap around a few pointers. Then, if you though it beneficial, you could walk the filesystem and do a file by file 'defragemntation' of the actual physical storage blocks.

                    But, I don't see the point of that step. The FTL already has to do a block level granulatiry redirection of *every* block. Is there much benefit to having the physical arrangement of blocks completely sequential? Keep in mind how few R/W cycles these flash chips have. Just how fragmented can they get? Aren't most UFS devices used in things like Android phones? Aren't they already using flash allocation friendly/aware FSs like F2FS? After all F2FS was designed by Samsung--who knows a bit about flash.

                    Comment

                    Working...
                    X