Announcement

Collapse
No announcement yet.

EXT4 In Linux 5.6 To See Big Write Performance Boost For Direct I/O

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • #21
    Originally posted by polarathene View Post
    How does this Direct I/O block overwriting thing work at that speed? With HDDs you overwrite the blocks, but on SSDs, you're only overwriting a virtual block mapping, while the SSD itself is managing the blocks under the hood, and afaik never overwrites data inline, it just writes to it's own blocks, then erases the old references to free up for writing to again, it's CoW like SMR drives can be?
    this patch changes how the kernel handles the filesystem mapping of blocks, not the hardware mapping on the device

    when writing to the filesystem, the kernel needs to map the blocks that will be written, as some some blocks may be newly allocated or released and it will start a transaction inside the ext4 journal to account these changes

    but if all the blocks are already allocated and will be overwritten, the blocks layout will not change and thus there is no need to start a transaction, so this patch will be useful on any high throughput device

    Comment


    • #22
      Originally posted by phuclv View Post

      which task requires you to view 200 tabs at once? I bet there are many unnecessary tabs that you can close. Some tabs can be delayed load and some others can be removed from memory temporarily. AFAIK newer Chrome versions automatically unload many tabs from memory, and Firefox will delay load tabs from the previous session

      But the first suggestion is to use zram (which is far faster than normal swap) and lower the priority of the swap partition
      One other issue I note is that the computer continues to use hard drive memory after I close the programs so to reduce the RAM usage... indeed every time I work on the computer the hard drive is involved making slow the tasks. There is some error in the management of memory resources.

      Comment


      • #23
        Originally posted by CommunityMember View Post

        You *can* add a swapfile on ext4 (a swapfile is not a swap partition of course, but they both provide swap space to the kernel and many people are ignorant about the differences since they operate the same).
        I know you can, I'm using one right now. But the OP said/wrote swap partition
        Either way, he clarified his issue is actually using a combo of Chrome+hundreds of open pages. It's like he's trying to cause an OOM.

        Comment


        • #24
          Originally posted by timofonic View Post
          Will there be a filesystem as fast or faster than EXT4
          XFS, for many use cases.

          Originally posted by timofonic View Post
          yet providing all the features of more modern filesystems such as ZFS and Btrfs?
          Not possible. Many features of ZFS and BTRFS depend on copy-on-write, which carries performance penalties, in certain cases. Yes, you can disable CoW, but not without losing features.

          Comment


          • #25
            Originally posted by coder View Post
            XFS, for many use cases.


            Not possible. Many features of ZFS and BTRFS depend on copy-on-write, which carries performance penalties, in certain cases. Yes, you can disable CoW, but not without losing features.
            So what about the bcachefs promise?

            Comment


            • #26
              Originally posted by timofonic View Post
              So what about the bcachefs promise?
              I'm not familiar with it.

              If you can find some recent benchmarks that prove me wrong, I'd be glad to hear about it.

              Comment


              • #27
                Originally posted by phuclv View Post

                which task requires you to view 200 tabs at once? I bet there are many unnecessary tabs that you can close. Some tabs can be delayed load and some others can be removed from memory temporarily. AFAIK newer Chrome versions automatically unload many tabs from memory, and Firefox will delay load tabs from the previous session

                But the first suggestion is to use zram (which is far faster than normal swap) and lower the priority of the swap partition
                many thanks how to use zram?

                Comment


                • #28
                  Originally posted by Azrael5 View Post

                  many thanks how to use zram?
                  On Ubuntu just run sudo apt-get install zram-config, it'll handle the setup and priority things for you
                  On Lubuntu and likely a few other distros it's enabled by default, no config needed. On others you need to use a different package, or config zram manually with zramctl command. Read the kernel's documentation for more info. You may want to change the default lzo compression algorithm to lz4 for even better performance. See the benchmark on the lz4 repo

                  See also Running out of RAM in Ubuntu? Enable ZRAM

                  Windows and macOS have similar compression techniques by default before swapping out unused pages to disk
                  Last edited by phuclv; 15 January 2020, 09:19 AM.

                  Comment


                  • #29
                    Originally posted by phuclv View Post

                    On Ubuntu just run sudo apt-get install zram-config, it'll handle the setup and priority things for you
                    On Lubuntu and likely a few other distros it's enabled by default, no config needed. On others you need to use a different package, or config zram manually with zramctl command. Read the kernel's documentation for more info. You may want to change the default lzo compression algorithm to lz4 for even better performance. See the benchmark on the lz4 repo

                    See also Running out of RAM in Ubuntu? Enable ZRAM

                    Windows and macOS have similar compression techniques by default before swapping out unused pages to disk
                    many thanks. Lubuntu is an excellent operating system.

                    Comment


                    • #30
                      Originally posted by Azrael5 View Post

                      many thanks. Lubuntu is an excellent operating system.
                      you can check if zram is being used by running cat /proc/swaps

                      Comment

                      Working...
                      X