Announcement

Collapse
No announcement yet.

Large HDD/SSD Linux 2.6.38 File-System Comparison

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • #31
    Originally posted by drag View Post
    Yeah sure. Didn't know if anybody cared. :P
    Great elaboration on the subject drag! I didn't ask for it but sure appreciated it.

    Comment


    • #32
      Originally posted by Michael View Post
      Michael , i meant to ask before but forget, why dont you also mount and run a generic ram disk test , if nothing else it serves as a baseline, and perhaps shows us if there's any DMA regressions in a given test set...

      perhaps you should set one up and add it to the results for that machine.

      Comment


      • #33
        Originally posted by drag View Post
        If you want a summary of what is the best FS for you to use... use Ext4. It's a safe file system.

        JFS is effectively unsupported. It's a port of a file system from OS/2 Warp... the AIX JFS is a entirely different beast. It was interesting when it was new, but besides a few fixes here and there it has essentially been unmaintained for years.

        XFS is good if you need big datasets. If you have multiple TB-large file systems then XFS is a good choice. It's fast, it scales well, and it behaves well when dealing with large amounts of data. You'll want to have very good hardware for it... it's not nearly as robust as Ext4 is.

        BTRFS is good if you want something to play around with. Otherwise leave it alone until distros start using it by default.
        I could not agree more, please don't read this article and begin to think like: hey! I want to use JFS / NILFS2 because I see on phoronix that it has more performance than ext4. Unless you know what are u dealing with, use ext4, it is good, and actively developed by google and Theodore Ts'o.

        Comment


        • #34
          The color of the EXT4 and JFS bars should be different, it is too similar.

          Comment


          • #35
            If you would like to recommend a particular set of mount options for each file-system, I would be happy to carry out such tests under those conditions as well to complement the default options.
            Michael, if you could, will you benchmark the various file systems with different schedulers/elevators? I'm performing my own comparisons, but I'd like to see noop, cfq, and deadline on the various filesystems compared.

            I think this would be a good thing for those of us with multiple types of hard drives to determine which scheduler to use for which hard drive. Should a 2TB use the same as an 80gb drive, or should ssd drives use noop or deadline?

            Also, what's the performance difference between relatime, noatime, and atime?

            How much difference do these disk benchmarks make depending on your amount of ram? I noticed months ago, part of the reason I went with 12gb instead of 6gb of ram, that the 8gb read tests were WAY higher if you had 8gb or more of ram.

            Separate issue... can you make it possible for us to choose colors of each bar on the benchmark graphs? I've done tests where there are 3 systems, and 2 of the 3 have the same color bar, instead of each one having it's own color.

            Thanks,

            Skeetre

            Comment


            • #36
              ext4 settings

              Michael I read your recently article about why we should leave default settings. I mostly agree with everything, based on the fact that most of the people are gonna use it that way.

              But not in this precise scenario: Ext4 + SSD.

              90% of the SSD users use Ext4 with noatime and discard options. This is almost mandatory to preserve disk's life. I guess if it were an ssd-mode as in btrfs, this would be as default. so in this case i think default options dont apply here, at least when we talk about number of people using it.

              I don't know if results were gonna change much, just wanted to point it out.

              Comment


              • #37
                Originally posted by drag View Post
                Yeah sure. Didn't know if anybody cared. :P

                Some of the things you need to keep very careful of is the data set size is correct for the test.

                Like, for example, if I am measuring raw I/O speeds for read/write and I have 4 GB of RAM and the dataset I am working with is only 4-5GB then your not really measuring the file systems as much as measuring the file system cache.
                ... and if you are testing a copy-on-write or logging file system, such as btrfs or nilfs2, the benchmark needs to write (and delete) at least 2-3 times the free space if you want to fairly test the effects of the garbage collector (in the case of a logging file system) or test the effects of the how well the file system holds up against fragmentation (in the case of copy-on-write file systems, such as ZFS and btrfs).

                And of course, if the benchmark isn't testing with the same size files as what you plan to use (small files? typical file sizes as seen by Open Office? MP3 sized files? Video files?), the results may also be not the same.

                Finally, for all file systems, how full the file system is during the test will also make a huge difference. It's very easy for file systems to exhibit good performance if you're only using the first 10-20% of the file system. But how does the file system perform when it's 50% full? 75% full?

                I can tell you that ext4's performance can be quite bad if you take an ext4 file system which is 1 or 2TB, and fill it up to 95% with file sizes are mostly 8-64 megabytes in size, especially if this happens where many 8-64 meg files are getting deleted, and then replaced with other 8-64 meg files, and the disk gradually fills up until it's 95% full. Since this is a workload (using ext4 as a backend store for cluster file systems) is one that I'm paid to care about at $WORK, I'm currently working on an extension to ext4 to help address this particular case.

                Measuring how file systems' performance fall off as the disk utilization increases is hard, yes. But the question is whether the benchmarking is being done primarily for entertainment's sake (i.e., to drive advertising dollars, like a NASCAR race without the car crashes), or to help users make valid decisions about which file system to use, or to help drive improvements in one or more file systems. Depending on your goals, how you approach the file system benchmarking task will be quite different.

                -- Ted

                Comment


                • #38
                  Originally posted by tytso View Post
                  And of course, if the benchmark isn't testing with the same size files as what you plan to use (small files? typical file sizes as seen by Open Office? MP3 sized files? Video files?), the results may also be not the same.

                  Finally, for all file systems, how full the file system is during the test will also make a huge difference. It's very easy for file systems to exhibit good performance if you're only using the first 10-20% of the file system. But how does the file system perform when it's 50% full? 75% full?

                  I can tell you that ext4's performance can be quite bad if you take an ext4 file system which is 1 or 2TB, and fill it up to 95% with file sizes are mostly 8-64 megabytes in size, especially if this happens where many 8-64 meg files are getting deleted, and then replaced with other 8-64 meg files, and the disk gradually fills up until it's 95% full. Since this is a workload (using ext4 as a backend store for cluster file systems) is one that I'm paid to care about at $WORK, I'm currently working on an extension to ext4 to help address this particular case.
                  No benchmark, no matter how thorough, is going to cover every scenario and every possible usage of a file system. If you are in a position such as you are where you have a very narrow set of requirements and usage scenarios for a file system, shouldn't you be running your own benchmarks and not relying on this set? No offense, but your situation doesn't apply to me and it never will. I wouldn't want a benchmark for your particular scenario because I'll never enter a situation like that, and I would venture that a majority of users would not either; any such data would skew opinions of file systems unnecessarily. I don't care how well a Corolla tows a 3 ton camper, just tell me how well it drives in basic conditions (city, highway) and I'll go from there.

                  Comment


                  • #39
                    Originally posted by cruiseoveride View Post
                    Where are the overall performance charts? I've been asking for these sort of charts since pts was in beta. These sort of articles are useful for individual metrics but are useless for the average reader who is just trying to choose the best overall.

                    If you take the time to tabulate and average out relative performance, you will see that NILFS2 was the best overall for a HDD, but reading this article won't tell you that. Is this so much to ask from pts?
                    I've found this also. Wrong description under SQLite graph. NIL2FS was fastest

                    Comment


                    • #40
                      Originally posted by cruiseoveride View Post
                      Where are the overall performance charts? I've been asking for these sort of charts since pts was in beta. These sort of articles are useful for individual metrics but are useless for the average reader who is just trying to choose the best overall.
                      They aren't shown in Phoronix articles as it would detract in page views... If you want averages, you can go to the respective page on OpenBenchmarking.org, and opt to view the averages, see the overall table, calculate geometric / harmonic / aggregate sums, etc etc.
                      Michael Larabel
                      https://www.michaellarabel.com/

                      Comment

                      Working...
                      X