Announcement

Collapse
No announcement yet.

New XFS Programs Update Supports New XFS On-Disk Format

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • New XFS Programs Update Supports New XFS On-Disk Format

    Phoronix: New XFS Programs Update Supports New XFS On-Disk Format

    After a year of development, xfsprogs 3.2.0 has been released as the latest version of the user-space program and other components for the XFS file-system. The big addition to xfsprogs 3.2.0 is supporting a new on-disk format...

    http://www.phoronix.com/vr.php?view=MTY5MTY

  • Apopas
    replied
    Checked. It does.

    Leave a comment:


  • Apopas
    replied
    Originally posted by microchip8 View Post
    No and no. you have to reformat the partition with the xfsutils that support the v5 format, ie by passing -m crc=1 to mkfs.xfs
    If I format the partition with this option, will I be able to mount it with a 3.12 kernel?

    Leave a comment:


  • jabl
    replied
    Originally posted by Paul-L View Post
    Again, since ext4 is good with little files (like the ones created on /tmp while compiling anything), the performance difference is vast for me
    If you're using the gcc family compilers, you really should look into the -pipe option. Or failing that, just put /tmp on a tmpfs. Or do both.

    Leave a comment:


  • xeekei
    replied
    I am a big fan of XFS, and it's my go to filesystem for most things. My /home is XFS, and all my extra storage hard drives is XFS. I've been thinking about going with an XFS /usr too. /boot is Ext2 though, and the root partition is Ext4. I love that I can pick and choose filesystems so freely in Linux.

    Leave a comment:


  • microchip8
    replied
    Originally posted by Paul-L View Post
    Again, since ext4 is good with little files (like the ones created on /tmp while compiling anything), the performance difference is vast for me, but I am not denying that XFS is far better for files that are bigger and are more critical than just a make file or a C source code file. Also, I watched the video, it was a good explaination of everything and I am reconsidering even to change some of my disks back to XFS now (thanks for that lol).
    There really is no need to change FS if EXT4 is giving you everything you need. I'm not advocating that. Just trying to explain the differences between the two FSes. It basically comes to this. EXT4 is designed for the low-end stuff while XFS for the mid- and high-end. Meaning, EXT4 is fine most of the time for regular desktop loads and maybe some other things too. While the theoretical maximum volume of an EXT4 partition is 1 Exabytes (EB), the max recommended is 16 Terabytes (TB) (same for max file size) due to architectural deficiencies. EXT4 is basically an improved/enhanced EXT3 borrowing some things from XFS (delayed allocation, extents and persistent pre-allocation) but it still suffers from an '80s-era design file system philosophy. If you need more than 16 TB, then XFS is much better at handling that. 16 TB or above may sound like a lot now, but in the near future we'll see such scenarios becoming more common in specific households. XFS's max volume size is 16 EB while its max file size is 8 EB. That's why people choose XFS for big storage, among other things.
    Last edited by microchip8; 05-16-2014, 06:42 PM.

    Leave a comment:


  • Paul-L
    replied
    Originally posted by microchip8 View Post
    I don't know about Fedora, but Red Hat is making XFS the default in their upcoming RHEL version. This may influence the Fedora decision to make it the default too. I don't know since I can't predict the future. Maybe Red Hat got biased somehow because Dave is working for them but it also could be that they see the benefit in XFS as RHEL is really targeted at entry-level servers up to big iron stuff and XFS was born for such tasks

    For SSDs, according to Dave Chinner, if you watch his video ( https://www.youtube.com/watch?v=FegjLbCnoBw ), XFS does not need any tweaks for SSDs. It works just fine on them and there really is no need to optimize. This was his answer at the end of the talk to a question asked by someone in the audience.

    Yes, AGs were invented specifically for parallel workloads. When you create an XFS file system, it "decides" how many AGs to create. If you're doing it on an RAID array, XFS will globally "see" one big volume as is with single disks. The difference is that if you're on a RAID array, the AGs will be spread across the disks, which allows XFS to push workload (eg, reads/writes) simultaneously to multiple disks at a time. Also, each AG is tracking its own data and free space. Think of AGs as "mini partitions" embedded in your original partition. In single disk scenarios, having parallel-based design with AGs brings little advantages, thus AGs are less effective compared to having many disks (thus many heads and platters) where everything gets spread out from the POV of XFS, which in turn can write to multiple locations concurrently due to the AGs being spread out across the many disks, but I'm repeating myself :P

    Another thing is, if you go for RAID (and even if you don't) do NOT use the CFQ disk scheduler. It destroys almost all of the parallelism XFS offers since some kernel versions back - quoting the XFS wiki, "As of kernel 3.2.12, the default i/o scheduler, CFQ, will defeat much of the parallelization in XFS.". The deadline, or even the noop scheduler, are recommended. As XFS has recently gone through a lot of changes, one of the thing it has also gained is to order the pending data waiting to be flushed from memory to disk in a way that it's basically doing the work of the scheduler. No, this is not to be compared with the data=ordered method offered by EXT4. Watch the video and you will understand

    PS: I should correct myself. The presentation is from 2012 and NOT for 2009 like I remembered. Must have confused it with another video I saw back in 2009.


    Again, since ext4 is good with little files (like the ones created on /tmp while compiling anything), the performance difference is vast for me, but I am not denying that XFS is far better for files that are bigger and are more critical than just a make file or a C source code file. Also, I watched the video, it was a good explaination of everything and I am reconsidering even to change some of my disks back to XFS now (thanks for that lol).

    Leave a comment:


  • microchip8
    replied
    Originally posted by liam View Post
    Thanks for this info, it's interesting. I wonder if this will change the minds of those devs in fedora to make it the default for Workstation.
    About AGs, I assume this is part of their well known, highly threaded io scheme? What happens when ssds are used? I ask because I know that btrfs made lots of design choices based on the assumption of platter-based media, and their ssd mode SEEMS to change very performance characteristics very little (though, when using highly queued work the threading helps, and, add that is exactly an enterprise load, it makes sense they'd optimize for that).
    I don't know about Fedora, but Red Hat is making XFS the default in their upcoming RHEL version. This may influence the Fedora decision to make it the default too. I don't know since I can't predict the future. Maybe Red Hat got biased somehow because Dave is working for them but it also could be that they see the benefit in XFS as RHEL is really targeted at entry-level servers up to big iron stuff and XFS was born for such tasks

    For SSDs, according to Dave Chinner, if you watch his video ( https://www.youtube.com/watch?v=FegjLbCnoBw ), XFS does not need any tweaks for SSDs. It works just fine on them and there really is no need to optimize. This was his answer at the end of the talk to a question asked by someone in the audience.

    Yes, AGs were invented specifically for parallel workloads. When you create an XFS file system, it "decides" how many AGs to create. If you're doing it on an RAID array, XFS will globally "see" one big volume as is with single disks. The difference is that if you're on a RAID array, the AGs will be spread across the disks, which allows XFS to push workload (eg, reads/writes) simultaneously to multiple disks at a time. Also, each AG is tracking its own data and free space. Think of AGs as "mini partitions" embedded in your original partition. In single disk scenarios, having parallel-based design with AGs brings little advantages, thus AGs are less effective compared to having many disks (thus many heads and platters) where everything gets spread out from the POV of XFS, which in turn can write to multiple locations concurrently due to the AGs being spread out across the many disks, but I'm repeating myself :P

    Another thing is, if you go for RAID (and even if you don't) do NOT use the CFQ disk scheduler. It destroys almost all of the parallelism XFS offers since some kernel versions back - quoting the XFS wiki, "As of kernel 3.2.12, the default i/o scheduler, CFQ, will defeat much of the parallelization in XFS.". The deadline, or even the noop scheduler, are recommended. As XFS has recently gone through a lot of changes, one of the thing it has also gained is to order the pending data waiting to be flushed from memory to disk in a way that it's basically doing the work of the scheduler. No, this is not to be compared with the data=ordered method offered by EXT4. Watch the video and you will understand

    PS: I should correct myself. The presentation is from 2012 and NOT for 2009 like I remembered. Must have confused it with another video I saw back in 2009.

    Leave a comment:


  • liam
    replied
    Originally posted by microchip8 View Post
    Currently, since I don't work in some big datacenter or enterprise, my guidelines are just two: what kind of stuff are you going to store on the FS? For small to moderate large files, EXT4 does fine and is actually faster than XFS on small files. For big files (think of HD or Full HD films, or ISO files of Blu-Ray discs which can take up to 50GB) I'd choose XFS without much thinking as I know it excels with them. The second is on scalability. There is currently just one stable FS, and that is XFS, which scales to extreme volumes. I mention "stable" as BTRFS is not there yet and it too scales extremely well. So if you're going to build some RAID array with lots of storage, you'll not only gain scalability but also much better performance than EXT4 due to XFS's AGs (allocation groups) specifically designed for multi-platter RAID spinning disks (= parallel workload).

    About your question on hybrid-disks. Sorry, cannot answer it as I've never used such disks yet but maybe there's some article for Linux on how to format the partition/FS. You can drop in #xfs on Freenode IRC and ask around. Either Dave, Brian or Eric will answer your question

    EDIT: it's worthy to mention that XFS has the most efficient free space handling of all Linux FSes out there
    Thanks for this info, it's interesting. I wonder if this will change the minds of those devs in fedora to make it the default for Workstation.
    About AGs, I assume this is part of their well known, highly threaded io scheme? What happens when ssds are used? I ask because I know that btrfs made lots of design choices based on the assumption of platter-based media, and their ssd mode SEEMS to change very performance characteristics very little (though, when using highly queued work the threading helps, and, add that is exactly an enterprise load, it makes sense they'd optimize for that).

    Leave a comment:


  • microchip8
    replied
    Originally posted by Paul-L View Post
    1. Changes nothing then.

    2. 2009 is very far away my friend, both have been developed bastly by now.

    3. both are suited for my task too, but I found in real-world workloads that ext4 does better, could be because my disk or something odd, dunno why.

    EXT4 has mostly seen bugfixes and code improvements, with only two big feature added to it which is the bigalloc feature (which has its downsides, too) and journal checksums. XFS has seen major improvements including a whole on-disk data structure changes in order to support CRCs and self-describing metadata. No, 2009 is not very far away. In the video you'll see Dave talking of what is coming next for XFS and that is the CRC/metadata stuff. The performance problems were fixed by then so he only talked about performance comparison and the upcoming CRC/metadata which is now stable to use

    That said, if you have no issues with EXT4 then keep using it. I'm not here to change your mind and if you say it feels faster for you then so be it. When I switched to XFS on all my computers, I couldn't really notice a speed degradation from when I was on EXT4. So to me and my regular workload, I can't notice a difference between the two FSes. Why I switched to XFS? Simple. I work a lot with big files such as HD/Full HD stuff and ISOs and I can definitely feel a performance improvements while on XFS. For everything else, like I said, I can't notice much difference between EXT4 and XFS

    Leave a comment:

Working...
X