No announcement yet.

Real World Benchmarks Of The EXT4 File-System

  • Filter
  • Time
  • Show
Clear All
new posts

  • #31
    Hmm this makes me want to try XFS again but only if my drives support write barriers. I've just been trying to find a way to determine whether the drive supports them but I can't find one anywhere. Is there any way besides actually creating an XFS partition?


    • #32
      I'd like to see boot time, app start time, package manager, and maybe file indexer/searching service benchmarks.

      Configuring each filesystem with settings commonly recommended by the community and including Reiser4 and JFS would be very interesting too.


      • #33
        I have XFS filesystem on my / partition on the desktop computer mounted with nobarrier option - I've been told this option is dangerous, but I haven't experienced any filesystem corruption since then, even with few unexpected reboots. I look forward to trying out the little XFS tweaks from this thread ^^

        I use ext4dev on my laptop, and it's working normally since 2.6.26 - I had some problems with block allocation on 2.6.25, but no files were corrupted or lost.


        • #34
          Originally posted by kebabbert View Post
          ext4 seems cool! Does it protect against silent corruption? Typically 20% of a modern hard drive is devoted to error correcting codes. Once in a while, you will run into a problem that is not correctable, or what is worse; not detectable. You dont even know that there was some error in your files:

          And Ive heard about a large ext3 filesystem being fsck, it took one week. Does ext4 suffer from the same problem?
          No, ext4 does not. The only filesystems that can detect data corruption are ZFS and BtrFS currently.


          • #35
            (includes Reiser4 and Ext4)


            RESULT: With compression, REISER4, absolutely SMASHED the other filesystems.

            No other filesystem came close (not even remotely close).

            Using REISER4 (gzip), rather than EXT2/3/4, saves you a truly amazing 816 - 213 = 603 MB (a 74% saving in disk space), and this, with little, or no, loss of performance when storing 655 MB of raw data. In fact, substantial performance increases were achieved in the bonnie++ benchmarks.

            We use the following filesystems:

            REISER4 gzip: Reiser4 using transparent gzip compression.
            REISER4 lzo: Reiser4 using transparent lzo compression.
            REISER4 Standard Reiser4 (with extents)
            EXT4 default Standard ext4.
            EXT4 extents ext4 with extents.
            NTFS3g Szabolcs Szakacsits' NTFS user-space driver.
            NTFS NTFS with Windows XP driver.

            Disk Usage in megabytes. Time in seconds. SMALLER is better.

            |File         |Disk |Copy |Copy |Tar  |Unzip| Del |
            |System       |Usage|655MB|655MB|Gzip |UnTar| 2.5 |
            |Type         | (MB)| (1) | (2) |655MB|655MB| Gig |
            |REISER4 gzip | 213 | 148 |  68 |  83 |  48 |  70 |
            |REISER4 lzo  | 278 | 138 |  56 |  80 |  34 |  84 |
            |REISER4 tails| 673 | 148 |  63 |  78 |  33 |  65 |
            |REISER4      | 692 | 148 |  55 |  67 |  25 |  56 |
            |NTFS3g       | 772 |1333 |1426 | 585 | 767 | 194 |
            |NTFS         | 779 | 781 | 173 |   X |   X |   X |
            |REISER3      | 793 | 184 |  98 |  85 |  63 |  22 |
            |XFS          | 799 | 220 | 173 | 119 |  90 | 106 |
            |JFS          | 806 | 228 | 202 |  95 |  97 | 127 |
            |EXT4 extents | 806 | 162 |  55 |  69 |  36 |  32 |
            |EXT4 default | 816 | 174 |  70 |  74 |  42 |  50 |
            |EXT3         | 816 | 182 |  74 |  73 |  43 |  51 |
            |EXT2         | 816 | 201 |  82 |  73 |  39 |  67 |
            |FAT32        | 988 | 253 | 158 | 118 |  81 |  95 |

            The raw data (without filesystem meta-data, block alignment wastage, etc) was 655MB.
            It comprised 3 different copies of the Linux kernel sources.

            Disk Usage: The amount of disk used to store the data.
            Copy 655MB (1): Time taken to copy the data over a partition boundary.
            Copy 655MB (2): Time taken to copy the data within a partition.
            Tar Gzip 655MB: Time taken to Tar and Gzip the data.
            Unzip UnTar 655MB: Time taken to UnGzip and UnTar the data.
            Del 2.5 Gig: Time taken to Delete everything just written (about 2.5 Gig).

            Each test was preformed 5 times and the average value recorded.

            To get a feel for the performance increases that can be achieved by using compression, we look at the total time (in seconds) to run the test:

            bonnie++ -n128:128k:0 (bonnie++ is Version 1.93c)

            | FILESYSTEM | TIME |
            |REISER4 lzo |  1938|
            |REISER4 gzip|  2295|
            |REISER4     |  3462|
            |EXT4        |  4408|
            |EXT2        |  4092|
            |JFS         |  4225|
            |EXT3        |  4421|
            |XFS         |  4625|
            |REISER3     |  6178|
            |FAT32       | 12342|
            |NTFS-3g     |>10414|
            The top two results use Reiser4 with compression. Since bonnie++ writes test files which are almost all zeros, compression speeds things up dramatically. That this is not the case in real world examples can be seen in the first test above where compression often does not speed things up. However, more importantly, it does not slow things down much, either.

            Last edited by Jade; 04 December 2008, 12:20 AM.


            • #36
              A couple points:

              1. Reiserfs certainly isn't the first FS to support online compression. I remember there being compressed file system options going back to DOS days.

              2. Most big files that everybody cares about... things like:

              * audio files
              * image files
              * document files (like's)
              * video files
              * game archives

              Are already compressed prior to being written to disk. Usually they use specialized compression that is much better for that specific format then generic ones like Gzip or Lzo. Ergo for anything that is especially sensitive to I/O speeds or are large are going to gain very little to no benefit from compression at the file system level.

              So most the benefit you'll gain will be from large text files and executables. Maybe from certain types of database actions.

              So you can get some boosts in performance in terms of application start up and whatnot. But the killer for startup performance is mostly seek times, not read limitations.. which compression does nothing to address and may actually make worse. Likely make latency worse.

              Having optional online compression support is certainly a good thing, (I would love to have it) but it's no homerun hit like those benchmarks misleadingly seem to indicate.

              Oh an point 3:

              Reiserfs is bad for your data's health.


              BTW, Btrfs has transparent compression support as one of it's listed features.


              • #37
                point 3 is so wrong. reiserfs (3.X) had some problems in early 2.4 because of constant breakage introduced by vm changes (the in kernel code will be fixed when it is affected lie). But later 2.4 and 2.6 reiserfs is one of the most stable fs out there. Only bug fixes, no features. Just look on lkml. Weekly XFS bugs, monthly ext3 bugs, once in a while a non-reproducible reiserfs bug.
                And reiser4 already works better for me than ext3 ever did.


                • #38
                  Originally posted by Jade View Post
                  (includes Reiser4 and Ext4)
                  Sorry, let me clarify. I'd like to see filesystem benchmarks that include current versions of Reiser4, ext4, and so on all configured as commonly recommended by their fans.


                  • #39
                    fsck time


                    first of all, thanks for the articel and benchmark.

                    We are planning to buy a new raid system with around 4TB of storage capacity (actual we have 2TB on ext3). On monthly scheduled administration days we reboot the main server for maintenance (new kernel, surely kick all nfs clients ...). So, from time to time, the raid system will check (tunefs could avoid this, but for safety reasons we perform the complete disk check) the data. This needs hours where you just can wait and wait ....

                    So, if ext4 would reduce this checking time, i would immediatley change.

                    Any experiences or a possibility to check this???

                    Thanks in advance


                    • #40
                      Originally posted by energyman View Post
                      point 3 is so wrong. reiserfs (3.X) had some problems in early 2.4 because of constant breakage introduced by vm changes (the in kernel code will be fixed when it is affected lie). But later 2.4 and 2.6 reiserfs is one of the most stable fs out there. Only bug fixes, no features. Just look on lkml. Weekly XFS bugs, monthly ext3 bugs, once in a while a non-reproducible reiserfs bug.
                      And reiser4 already works better for me than ext3 ever did.

                      Reiser and friends refused to support and maintain Reiserfsv3 once they started work on Reiserfsv4. All file systems are very complex and have problems that are going to crop up over time, refusing to support it means that bugs and problems went unfixed, unacknowledged, and forced other developers to pick up their work if it happened at all.

                      Rfsv4 isn't ever going to get into the kernel at this point. At least not in any substantial way. It has no serious backing from the core Linux developers or any of the corporations or distributions that depend on Linux. Without this testing and support it's never going to mature to the point were I am going to trust it.

                      If Reiser was able to relate and get along with other people then it probably would of had a good chance, but this is just not how it worked out.

                      And speed, btw, is extremely secondary to data protection and reliability.

                      Ask yourself why, if companies like Redhat or IBM, that depend on the competitive nature of Linux against operating systems like Solaris and Windows 2008, have continued to put substantial time and effort into maintaining Ext3 and evolving it to Ext4 instead of just adopting the 'much better' Reiserfsv4.

                      Suse was a early adopter and proponent of ReiserFsv3. They have ReiserFS developers on staff. They show no sign of moving to support v4 in any meaningful way. They too depend heavily on the ability of Linux to compete with Unix, Windows, and especially Redhat. So you would think that if v4 offered a substantial advantage over the more mundane Linux file systems then they would jump at the chance to push their OS forward.

                      And why Intel, HP, IBM, Oracle, and Redhat are currently spending a great deal of money, developer time, and expertise working and hacking on Btrfs. People who make their living designing and developing file systems... things like Ext2/3, Advfs, Ocfsv2, Reiserfs, Lustre, GFS, and XFS.

                      Because they have very good reasons for this sort of thing and understanding why is critical to really knowing how to gauge file systems. Now they are not gods, are imperfect, with biases, and lots of decisions are driven by politics as much as technical reasons, but there is method behind all this madness.


                      For example:

                      Ext3 fsck is something that is rather powerful at protecting and repairing file systems. It's robust, proven, and reliable.

                      Ext3 journalling also has the capabilities to monitor and detect issues with actual files. Sure it's not as good as checksums, but it does have the ability to do journalling for files. Now other journalling FSs, like Reiser and XFS only monitor file system metadata (that is they monitor the FS and ignore the data store using the FS) and that is all they are capable of dealing with. (unfortunately with Ext3 there are bugs for a long time that tended to limit the usefulness of this feature)

                      If you run Reiserfsck on a file system with a file with a loopback file with Reiserfs on it.. then it will often try to stitch your loopback file into the file system. Which is the sort of thing you very much do not want.

                      That's a problem with v3 that was never fixed. It isn't suppose to be a problem v4, but keep in mind that unlike Ext2->Ext3->Ext4 each new Reiser file system is rewritten from scratch and are not related to one another in any direct manner.
                      Last edited by drag; 04 December 2008, 03:41 AM.