Announcement

Collapse
No announcement yet.

Real World Benchmarks Of The EXT4 File-System

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • #61
    just look at lkml.
    Every month you will see reports for fs corruption and bugs - xfs and ext3.
    You hardly ever see a report for reiserfs.
    And it wasn't dropped because it was unstable - it was dropped because Namesys wasn't willing to add any new features. They wanted to put reiserfs in maintanace only mode. ext3 is much 'better' constantly adding features - and bugs.

    Comment


    • #62
      Originally posted by energyman View Post
      just look at lkml.
      Every month you will see reports for fs corruption and bugs - xfs and ext3.
      You hardly ever see a report for reiserfs.
      And it wasn't dropped because it was unstable - it was dropped because Namesys wasn't willing to add any new features. They wanted to put reiserfs in maintanace only mode. ext3 is much 'better' constantly adding features - and bugs.
      Here is the exact reasons why suse dropped it.
      Hi all -

      We've been using ReiserFS as our default installation file system for
      the last 6-7 years now, and it's served us well in that time.
      Unfortunately, there are a number of problems with it, some purely
      technical, some more related to maintenance. I'll outline a few of the
      larger issues and offer my solution as a conclusion.

      ReiserFS has serious scalability problems. David Chinner's talk at OLS
      really underscored the problem well for a single, large, high bandwidth
      file system. While I realize that XFS-style scalability isn't a real
      goal for most users, ans isn't a target workload for reiserfs, the
      scalability problems are real. ReiserFS uses the BKL for synchronization
      everywhere, and since it's system-global lock, the problem doesn't go
      away when you split the file system into smaller ones. Lock contention
      alone is one problem, but it's made worse by cache bouncing between
      processors on larger systems.

      ReiserFS has serious performance problems with extended attributes and
      ACLs. (Yes, this one is my own fault, see numerous flamewars on lkml and
      reiserfs-list for my opinion on this.) xattrs are backed by normal files
      rooted in a hidden directory structure. This is bad for performance and
      in rare cases is deadlock prone due to lock inversions between pdflush
      and the xattr code
      . The quota code gets around this, but the fix would
      result in huge amounts of wasted space with ReiserFS. With increasing
      deployment of SLES as samba servers, and perhaps NFSv4 servers, the use
      of extended attributes will only increase.

      ReiserFS has a small and shrinking development community. Right now, the
      only developers really working with ReiserFS are Chris Mason, Jan Kara
      (internally), a rotating member of Hans Reiser's team, and myself. All
      of us have projects we're very much more interested in than working with
      ReiserFS. While Jan and I will be continuing to support ReiserFS for
      SUSE, Hans is increasingly (hard to believe) pushing people to use
      reiser4. Chris has moved on to Oracle and has expressed his opinions on
      leaving ReiserFS behind.

      ReiserFS v3 is a dead end. Hans has been pushing reiser4 for years now
      and declared Reiser3 in maintenance mode. Any changes that aren't bug
      fixes are met with violent resistance. Reiser4 is not an incremental
      update and requires a reformat, which is unreasonable for most people.
      Reiser3 lacks a number of features that other file systems either have
      or are adding soon, such as extents and growth beyond current limits.
      Since it's in maintenance mode, that's unlikely to change. I view
      reiser4 as an interesting research file system, but that's about as far
      as it goes. I've been unimpressed with its stability so far. I don't
      know how advanced the recovery tools are yet, but I suspect that the
      complexity of the format and the ability to essentially define the
      format on-the-fly with plugins will make a useful fsck extremely difficult.

      The solution for replacing an aging file system isn't to switch to a
      brand new unproven file system, but rather a proven one with a clear
      upgrade path. That file system is ext3.

      Ext3's performance in some situations may not be on par with Reiser3,
      but it scales better and Andi mentioned the other day that there is
      quite a bit of research going into improving the locking and general
      performance of ext3 going on right now, and since reiser3 is stagnant, I
      don't doubt they'll pass them soon.

      Ext3 has a much larger development community out there. Most other
      distributions use ext3 as their default file system, so bugs that don't
      end up getting reported to us and are fixed by other developers, we get
      for free - most Reiser3 fixes originate from Chris or I.

      Ext3 has a clear upgrade path. There is quite a bit of interest in the
      community in improving ext3, and ext4 is already under development. Like
      the upgrade path from ext2 to ext3, the path to ext4 is clearly defined.
      Existing file systems can be updated easily, and new files will be able
      to take advantage of the new features. Features already written and
      queued up include extents, a 64-bit journal, and 64-bit file sizes.

      Most of the institutional knowledge of reiserfs is bouncing around in my
      head. Jan has been getting his hands dirty a little bit, but beyond
      that, finding additional developers with reiserfs experience will be
      extremely difficult and I'd call training additional developers a wasted
      effort. Since reiserfs is in maintenance mode, the effort needed to
      continue to support it in future releases should be shrinking.

      To be clear, my long term goal is to use OCFS2 (or another CFS if one
      shows a clear adoption advantage) for the root file system. This would
      enable single-instance clustering at both the physical and the virtual
      distribution level and get us ease of management and flexibility in HA
      deployments. Realistically, though, desktop users are likely to continue
      to use ext[34] for the foreseeable future. Until we have OCFS2 (and the
      rest of the distribution) ready for such a deployment described above on
      larger servers, ext3 would be a suitable choice across the board.

      - -Jeff

      - --
      Jeff Mahoney
      SUSE Labs

      Comment


      • #63
        Originally posted by energyman View Post
        just look at lkml.
        Every month you will see reports for fs corruption and bugs - xfs and ext3.
        You hardly ever see a report for reiserfs.
        While that reflects the stability of those filesystems, it's also slanted by their popularity/exposure. In other words, given 2 filesystems of equal quality, one twice as popular should average about twice as many bug reports. Simply counting bug reports would mistake obscurity for stability and broad exposure for bugginess.
        Last edited by StringCheesian; 12-06-2008, 04:22 AM.

        Comment


        • #64
          and how many users use need acl/xattrs? I know many linux users personally - and not one needs or uses any of the two. BKL is a red herring too. The BKL is used in so many places in the kernel, that the BKL usage of reiserfs is just a drop into an ocean. Same for 'scalability'. Who cares that reiserfs does not deal as well as xfs with 10s of disks, and several terabytes of data?

          Comment


          • #65
            Sorry, but you're losing credibility. As others have said, these tests are mostly useless: CPU bound or huge files. Do we really have to point out some relevant benches such as kernel compilation, boot-time or rsync archives ?

            Comment


            • #66
              Originally posted by reavertm View Post
              I'm using reiserfs ('3') for at least 5 years and I never ever had any data loss nor filesystem corruption. It handled perfectly every power loss I occured. It's rock stable and is eating alive ext3 when it comes to performance.
              I'm using reiser3 for at about 7 years and I never ever had any data loss nor filesystem corruption. It handled perfectly every power loss that occured. It's rock stable and eats ext3 alive when it comes to performance.

              There was one exception when using QEMU (most likely due to various Gentoo saboteurs getting to the code) which you can read about here:

              http://linux.50webs.org/gentoo/qemu-gentoo.htm

              A quote from that page:

              "I would have usually formated /dev/hdb3 with the Reiser3 filesystem, however, this usually extremely reliable filesystem, has been deliberately sabotaged. This sabotage causes massive corruption of files after a short time. By comparison, I have used the Reiser3 filesystem for nearly ten years (actually about 7 or 8 years (there was no ext3 when I changed)) now and have never had any corruption, at all, on any of my many Reiser3 boxes."

              Comment


              • #67
                jade, i have found a _real_ conspiracy for you :

                check this out:

                http://repos.archlinux.org/viewvc.cg...LD?view=markup

                this is a gcc build script for arch linux :

                Code:
                if ! locale -a | grep ^de_DE; then
                    echo "You need the de_DE locale to build gcc."
                    return 1
                  fi
                it must be an german conspiracy to take over the world! ;-)

                that aside, it still beats me what does it need de_DE for ?

                Comment


                • #68
                  From LFS:
                  The following instructions will install the minimum set of locales necessary for the optimal coverage of tests:
                  It's for the test suites.

                  Comment


                  • #69
                    ReiserFS, with its balanced trees, is extremely sensitive to CPU and memory errors. Imagine a RAM module goes bad, and you only notice it a few days later. This can be too late already and half the file system is destroyed. It's what happened to me.

                    Anyway, I don't see what all the fuss is about. ext3/4 are stable, perform well and have an okay feature set for most uses.

                    Comment


                    • #70
                      ext3 is not stable, and ext4 is pre alpha code.

                      Comment


                      • #71
                        Originally posted by greg View Post
                        ReiserFS, with its balanced trees, is extremely sensitive to CPU and memory errors. Imagine a RAM module goes bad, and you only notice it a few days later. This can be too late already and half the file system is destroyed. It's what happened to me.
                        Yes... Though these issues are already known (except by people with marginal hardware who won't acknowledge the fact ;>). Reiserfs (v3) is imho easily the best linux filesystem. I've used it exclusively for over 5 years on dozens of installs and never have had a problem except for an overheating disk.

                        Having said that, A solution to any problem must address all issues, technical and social. Reiserfs v3 and v4 are unfortunately marginalised by
                        1) (above quote)
                        2) Hans' no-nonsense attitude to various people
                        3) Hans' legal problem
                        4) reiserfsck tree splicing bug.
                        5) Reiser4's non-inclusion in kernel due to (2) (3).

                        As for Ext3/4, I can't comment personally as my few experiences of ext3 were so sluggish i've hardly used it.... But i suspect energyman is right ( ;> ).

                        Comment


                        • #72
                          Originally posted by deanjo View Post
                          Here is the exact reasons why suse dropped it.
                          Actually, the real reason why SuSE dropped it is to be found in here somewhere:

                          Originally posted by drag
                          Suse was a early adopter and proponent of ReiserFsv3. They have ReiserFS developers on staff.
                          At least this statement of yours is true.

                          Suse has supported and distributed Reiser3 ever since January 2000 (in SuSE 6.3).


                          They show no sign of moving to support v4 in any meaningful way.
                          This is TOTAL CRAP. Suse supported and distributed Reiser4 for years.

                          They were almost the only ones supporting it.


                          They too depend heavily on the ability of Linux to compete with Unix, Windows, and especially Redhat.
                          This is TOTAL CRAP as well.

                          Reiser4 was supported by SuSE till they were bought out by the Jews. The Jews already owned Redhat, so there was no competition.


                          So you would think that if v4 offered a substantial advantage over the more mundane Linux file systems then they would jump at the chance to push their OS forward.
                          The (German company SuSE) did "jump at the chance," as a non-fairy tale version of history substantiates.

                          When the Jews bought it out, they worked hard on getting rid of KDE, Reiser3, Reiser4, mp3 support and NTFS support.

                          Destruction of Linux NTFS support got away from them when Szabolcs Szakacsits released his NTFS driver.

                          They removed mp3 support from SuSE 10. Thus I stopped using SuSE, so I don't know if it is still sabotaged in this way.

                          Reiser4 has been successfully shut down by sabotage of the Linux kernel code due to Andrew Morton.

                          They are still trying to kill Reiser3, but too many people know that for years it was the best filesystem available and it is proving hard for them to get rid of it.

                          There was a huge user rebellion against the move to Gnome and KDE stayed,... at least for now.

                          Comment


                          • #73
                            Originally posted by StringCheesian View Post
                            While that reflects the stability of those filesystems, it's also slanted by their popularity/exposure. In other words, given 2 filesystems of equal quality, one twice as popular should average about twice as many bug reports. Simply counting bug reports would mistake obscurity for stability and broad exposure for bugginess.
                            Esp. when you consider that probably a significant fraction of the data loss reports (for any filesystem) are partly due to flaky hardware (bad RAM, bad power supply, ...) more users = more corruption pretty much independently of the reliability of the filesystem code (on good hardware). Esp. distro default filesystems will get used by less-expert users who are probably more likely to have less-good hardware. (My theory here is that people who are computer-savvy enough to build a machine with quality parts might be more likely to choose non-default filesystems.)

                            edit: forgot to say: My guess is that people usually switch to a new filesystem when the set up a new machine and do a fresh install, so their experience of one filesystem is on one machine, and of another filesystem on another machine. If one of those machines sometimes had memory errors, or data errors on the way from disk to memory, that will affect their impression. So everyone out there who's had bad luck with a certain FS, think back and see if you remember trying other filesystems on the same hardware. If not, maybe it was marginal hardware causing the problems.

                            Sensitivity to memory errors should be a factor in filesystem choice if you run with a cheapo power supply and RAM you don't really trust. Or if you overclock. I try to use quality hardware, since having a computer that gets wrong answers completely defeats the purpose of a having a dumb but fast machine. Also, I want to use svn/git versions of things and make bug reports without wondering if maybe they crashed because my hardware is flaky. If I just wanted my commercial games to run fast, I'd probably be a lot more interested in overclocking.

                            Other than disk failures, I've never experienced FS corruption that lost any data on my own systems. I guess I've been lucky. I've used ext3 quite a bit in the past, and I would still use ext3 with data=journal for a FS that I _really_ didn't want to lose anything on after a crash. I use ext3 on my laptop because it dual-boots winxp, and there aren't windoze drivers for any other good filesystems. I have a ~10 year old system (P3-450) as a router/server on my cable modem, and I ssh to it to read my mail (with mutt) from anywhere in the world. It's been running Debian with ext3, and hasn't been re-installed since March 2001. It uses ext3 everywhere, and I've never had a problem with FS corruption.

                            I've used reiserfs a few times, and it's been ok. It's not extent-based, so it's slow to delete large files (like ext3).

                            I currently use JFS as my root filesystem on my desktop. I've also used it on a few HPC cluster machines I've set up at my former work.

                            I think reiser4 has some very interesting ideas. Years ago, I read some of the papers on namesys.com about making the filesystem "smarter". I've always been a fan of adding interesting features to GNU/Linux, more than having it be a 100% perfect re-implementation of Unix. If reiser4 had caught on, and file managers had started using its features to search files by tags or something, I might be using a file manager instead of continuing to be happy with typing commands in an xterm-equivalent. (I can type faster than I can click, and most of the interesting programs I know how to use, like sed, grep, find, are all traditional Unix command line stuff anyway. )

                            I'm lazy, and I haven't compiled my own kernels for a couple years now. I use Ubuntu's kernels, and they don't include reiser4. I could compile my own, of course, but I partly want to do QA testing for Ubuntu.

                            It's a shame that the reiser4 project isn't getting much development funding, compared to other FSes, but it's hardly the only case of interesting/good stuff dying because Linux's corporate backers have their own plans. See http://sabi.co.uk/blog/anno06-3rd.html#060702. I've been reading Sabi's blog quite a bit, and he frequently points out how GNU/Linux is suffering from Microsoft cultural hegemony, and occasional land-grabs by kernel devs. e.g. GNU/Linux has grown a few different registry-work-alikes, and more and more things use binary databases instead of editable config files.

                            XFS is my filesystem of choice these days. Esp. with dual-core CPUs being standard, XFS's scalability starts to come into play. ext3 will interleave two files written at the same time on different CPUs, but XFS's delayed allocation doesn't have that problem. I don't have big-iron nearly as big as this, though:
                            http://oss.sgi.com/projects/xfs/pape...esentation.pdf
                            and someone's blog about it:
                            http://sabi.co.uk/blog/anno06-4th.html#061022

                            edit: forgot to mention this:
                            XFS has survived lots of crashes (e.g. when testing out Mesa's i965 DRI driver...), but I very rarely have power interruptions, thanks to a UPS. So I run with write-caches enabled and no write barriers (-o nobarrier). Actually, I use LVM2, and device-mapper doesn't support barriers, so I don't explicitly say nobarrier. On a read-mostly filesystem, like the root or /usr fs, where corruption is a bigger headache (although catchable with debsums), I would enable write barriers. Note that JFS doesn't support write barriers, but ext3 does. http://sabi.co.uk/Notes/linuxFS.html#fsFeats Anyone know about any others?

                            BTW, I was hoping someone would reply to my brain-dump about XFS I posted previously, before it got pushed off the last page by reiser4 consiracy theories (and blaming things on Jewish people? WTF , there's no call for that. If there are specific Jewish people you think are doing bad things, that's fine. There are lots of Jews who are not nice people, same for most groups. But talking about them as "the Jews" is not sensible, IMHO. Isn't this board moderated, or is that only for first posts from new users. I guess I'll find out, since this'll be my second.) It's probably not a conspiracy, just Linux corporations spending their money on development of what they think will be in their best interest. And unfortunately that's not reiser4.

                            I spent maybe a couple hours writing http://www.phoronix.com/forums/showp...7&postcount=46. I should probably put it somewhere instead of leaving it buried in this forum, but still, I was hoping someone would read it...
                            Last edited by llama; 12-06-2008, 10:17 PM.

                            Comment


                            • #74
                              @llama
                              I find your information about XFS most insightful, I'm waiting for the first occasion to try out your tweaks
                              I had problems with XFS with earlier kernels (2.6.24,2.6.25) I've experienced a few nasty data corruptions (one almost broke my system -_-), but from 2.6.26 onward, everything is well, even with nobarrier... it's rock solid now, and I'm really happy if I can safely enhance its performance.

                              @Jade
                              Please, use any other color than red... it's totally unreadable on my CRT monitor...

                              Comment


                              • #75
                                Still, no matter what Jade actually prays for (and others shoud at him for this), proper filesytem tests is a really good idea.
                                So, what I wanted to see there is:
                                ext3, ext4, jfs, xfs, reiserfs, reiser4.
                                And please don't skip reisers as they are fast. We need to test what's out there w/o any exception!
                                Tests might include all Phoronix tests and some SVN checkout deletion (like wine or smth big really), compressing a lot of small files, the same with large and so.
                                I'm really looking forward to see some objective tests!

                                Comment

                                Working...
                                X