New XFS Programs Update Supports New XFS On-Disk Format

[shmerl]

No and no. you have to reformat the partition with the xfsutils that support the v5 format, ie by passing -m crc=1 to mkfs.xfs

Thanks.

[microchip8]

1. What I meant, slowly developed in comparison of ext4.
2. Check the benchmarks on I/O in this very site. It's not the same.
3. As a home user and hobbyst on programming, ext4 has become faster (atleast for me, obviously) dealing with my personal problems; I know that XFS is targeted to enterprise environments and that's why I like it so much, redundancy.

I meant no offense of course, just that in my point of view, patches submitted that are not merged mean nothing; and for some reason ext4 has become faster on my volumes while XFS has become slower (with some performance improvements on some kernel releases). Again, this is my personal point of view, not to say that what you written is not true for your particular enviroment.

1. it's only slower developed compared to EXT4 because the EXT4 has a bigger community and is considered the default FS for Linux. But, devs from XFS provide patches to EXT4 and the other way around

2. I don't visit Phoronix for benchmarks as most of the time they don't measure real-world workloads. I come here mostly for other news. I highly recommend the Dave Chinner video from 2009 (you can find it on youtube) where he explains and shows how benchmarks can be very deceiving. He compared mostly BTRFS with XFS, but it holds true for other FSes too.

3. I use both XFS and EXT4 for different reasons. When it comes to choosing an FS for a specific environment, after considering the needs, I'd suggest either XFS or EXT4 is better suited for the task. My post above might have seen as I'm biased towards XFS, but it was just a post trying to go away with some of the misconceptions about XFS

[shmerl]

3. I use both XFS and EXT4 for different reasons. When it comes to choosing an FS for a specific environment, after considering the needs, I'd suggest either XFS or EXT4 is better suited for the task. My post above might have seen as I'm biased towards XFS, but it was just a post trying to go away with some of the misconceptions about XFS

What are your general guidelines about when to use either of them? What about working with advanced format drives?

I installed a new system recently (Debian testing), using a hybrid advanced format WD hard drive ("black"). I set the root (home is on the same partition) to XFS and used 4 KB sector size in mkfs.xfs since I found somewhere that it's a recommended setting for the hybrid advanced format disks which have 4 KB physical and 512 B logical sectors (and supposedly default in the newest versions mkfs.xfs). However it seems to me that reading with such setting is even slower than with my older disks (I didn't benchmark it in detail yet, just found the bonnie++ tool which I plan to use to drill down to actual performance). Any idea if what I did was correct, or setting sector size to 4 KB wasn't right?

[Paul-L]

1. it's only slower developed compared to EXT4 because the EXT4 has a bigger community and is considered the default FS for Linux. But, devs from XFS provide patches to EXT4 and the other way around

2. I don't visit Phoronix for benchmarks as most of the time they don't measure real-world workloads. I come here mostly for other news. I highly recommend the Dave Chinner video from 2009 (you can find it on youtube) where he explains and shows how benchmarks can be very deceiving. He compared mostly BTRFS with XFS, but it holds true for other FSes too.

3. I use both XFS and EXT4 for different reasons. When it comes to choosing an FS for a specific environment, after considering the needs, I'd suggest either XFS or EXT4 is better suited for the task. My post above might have seen as I'm biased towards XFS, but it was just a post trying to go away with some of the misconceptions about XFS

1. Changes nothing then.

2. 2009 is very far away my friend, both have been developed bastly by now.

3. both are suited for my task too, but I found in real-world workloads that ext4 does better, could be because my disk or something odd, dunno why.

[microchip8]

What are your general guidelines about when to use either of them? What about working with advanced format drives?

I installed a new system recently (Debian testing), using a hybrid advanced format WD hard drive ("black"). I set the root (home is on the same partition) to XFS and used 4 KB sector size in mkfs.xfs since I found somewhere that it's a recommended setting for the hybrid advanced format disks which have 4 KB physical and 512 B logical sectors (and supposedly default in the newest versions mkfs.xfs). However it seems to me that reading with such setting is even slower than with my older disks (I didn't benchmark it in detail yet, just found the bonnie++ tool which I plan to use to drill down to actual performance). Any idea if what I did was correct, or setting sector size to 4 KB wasn't right?

Currently, since I don't work in some big datacenter or enterprise, my guidelines are just two: what kind of stuff are you going to store on the FS? For small to moderate large files, EXT4 does fine and is actually faster than XFS on small files. For big files (think of HD or Full HD films, or ISO files of Blu-Ray discs which can take up to 50GB) I'd choose XFS without much thinking as I know it excels with them. The second is on scalability. There is currently just one stable FS, and that is XFS, which scales to extreme volumes. I mention "stable" as BTRFS is not there yet and it too scales extremely well. So if you're going to build some RAID array with lots of storage, you'll not only gain scalability but also much better performance than EXT4 due to XFS's AGs (allocation groups) specifically designed for multi-platter RAID spinning disks (= parallel workload).

About your question on hybrid-disks. Sorry, cannot answer it as I've never used such disks yet but maybe there's some article for Linux on how to format the partition/FS. You can drop in #xfs on Freenode IRC and ask around. Either Dave, Brian or Eric will answer your question

EDIT: it's worthy to mention that XFS has the most efficient free space handling of all Linux FSes out there

[microchip8]

1. Changes nothing then.

2. 2009 is very far away my friend, both have been developed bastly by now.

3. both are suited for my task too, but I found in real-world workloads that ext4 does better, could be because my disk or something odd, dunno why.

EXT4 has mostly seen bugfixes and code improvements, with only two big feature added to it which is the bigalloc feature (which has its downsides, too) and journal checksums. XFS has seen major improvements including a whole on-disk data structure changes in order to support CRCs and self-describing metadata. No, 2009 is not very far away. In the video you'll see Dave talking of what is coming next for XFS and that is the CRC/metadata stuff. The performance problems were fixed by then so he only talked about performance comparison and the upcoming CRC/metadata which is now stable to use

That said, if you have no issues with EXT4 then keep using it. I'm not here to change your mind and if you say it feels faster for you then so be it. When I switched to XFS on all my computers, I couldn't really notice a speed degradation from when I was on EXT4. So to me and my regular workload, I can't notice a difference between the two FSes. Why I switched to XFS? Simple. I work a lot with big files such as HD/Full HD stuff and ISOs and I can definitely feel a performance improvements while on XFS. For everything else, like I said, I can't notice much difference between EXT4 and XFS

[liam]

Currently, since I don't work in some big datacenter or enterprise, my guidelines are just two: what kind of stuff are you going to store on the FS? For small to moderate large files, EXT4 does fine and is actually faster than XFS on small files. For big files (think of HD or Full HD films, or ISO files of Blu-Ray discs which can take up to 50GB) I'd choose XFS without much thinking as I know it excels with them. The second is on scalability. There is currently just one stable FS, and that is XFS, which scales to extreme volumes. I mention "stable" as BTRFS is not there yet and it too scales extremely well. So if you're going to build some RAID array with lots of storage, you'll not only gain scalability but also much better performance than EXT4 due to XFS's AGs (allocation groups) specifically designed for multi-platter RAID spinning disks (= parallel workload).

About your question on hybrid-disks. Sorry, cannot answer it as I've never used such disks yet but maybe there's some article for Linux on how to format the partition/FS. You can drop in #xfs on Freenode IRC and ask around. Either Dave, Brian or Eric will answer your question

EDIT: it's worthy to mention that XFS has the most efficient free space handling of all Linux FSes out there

Thanks for this info, it's interesting. I wonder if this will change the minds of those devs in fedora to make it the default for Workstation.
About AGs, I assume this is part of their well known, highly threaded io scheme? What happens when ssds are used? I ask because I know that btrfs made lots of design choices based on the assumption of platter-based media, and their ssd mode SEEMS to change very performance characteristics very little (though, when using highly queued work the threading helps, and, add that is exactly an enterprise load, it makes sense they'd optimize for that).

[microchip8]

Thanks for this info, it's interesting. I wonder if this will change the minds of those devs in fedora to make it the default for Workstation.
About AGs, I assume this is part of their well known, highly threaded io scheme? What happens when ssds are used? I ask because I know that btrfs made lots of design choices based on the assumption of platter-based media, and their ssd mode SEEMS to change very performance characteristics very little (though, when using highly queued work the threading helps, and, add that is exactly an enterprise load, it makes sense they'd optimize for that).

I don't know about Fedora, but Red Hat is making XFS the default in their upcoming RHEL version. This may influence the Fedora decision to make it the default too. I don't know since I can't predict the future. Maybe Red Hat got biased somehow because Dave is working for them but it also could be that they see the benefit in XFS as RHEL is really targeted at entry-level servers up to big iron stuff and XFS was born for such tasks

For SSDs, according to Dave Chinner, if you watch his video ( https://www.youtube.com/watch?v=FegjLbCnoBw ), XFS does not need any tweaks for SSDs. It works just fine on them and there really is no need to optimize. This was his answer at the end of the talk to a question asked by someone in the audience.

Yes, AGs were invented specifically for parallel workloads. When you create an XFS file system, it "decides" how many AGs to create. If you're doing it on an RAID array, XFS will globally "see" one big volume as is with single disks. The difference is that if you're on a RAID array, the AGs will be spread across the disks, which allows XFS to push workload (eg, reads/writes) simultaneously to multiple disks at a time. Also, each AG is tracking its own data and free space. Think of AGs as "mini partitions" embedded in your original partition. In single disk scenarios, having parallel-based design with AGs brings little advantages, thus AGs are less effective compared to having many disks (thus many heads and platters) where everything gets spread out from the POV of XFS, which in turn can write to multiple locations concurrently due to the AGs being spread out across the many disks, but I'm repeating myself :P

Another thing is, if you go for RAID (and even if you don't) do NOT use the CFQ disk scheduler. It destroys almost all of the parallelism XFS offers since some kernel versions back - quoting the XFS wiki, "As of kernel 3.2.12, the default i/o scheduler, CFQ, will defeat much of the parallelization in XFS.". The deadline, or even the noop scheduler, are recommended. As XFS has recently gone through a lot of changes, one of the thing it has also gained is to order the pending data waiting to be flushed from memory to disk in a way that it's basically doing the work of the scheduler. No, this is not to be compared with the data=ordered method offered by EXT4. Watch the video and you will understand

PS: I should correct myself. The presentation is from 2012 and NOT for 2009 like I remembered. Must have confused it with another video I saw back in 2009.

[Paul-L]

I don't know about Fedora, but Red Hat is making XFS the default in their upcoming RHEL version. This may influence the Fedora decision to make it the default too. I don't know since I can't predict the future. Maybe Red Hat got biased somehow because Dave is working for them but it also could be that they see the benefit in XFS as RHEL is really targeted at entry-level servers up to big iron stuff and XFS was born for such tasks

For SSDs, according to Dave Chinner, if you watch his video ( https://www.youtube.com/watch?v=FegjLbCnoBw ), XFS does not need any tweaks for SSDs. It works just fine on them and there really is no need to optimize. This was his answer at the end of the talk to a question asked by someone in the audience.

Yes, AGs were invented specifically for parallel workloads. When you create an XFS file system, it "decides" how many AGs to create. If you're doing it on an RAID array, XFS will globally "see" one big volume as is with single disks. The difference is that if you're on a RAID array, the AGs will be spread across the disks, which allows XFS to push workload (eg, reads/writes) simultaneously to multiple disks at a time. Also, each AG is tracking its own data and free space. Think of AGs as "mini partitions" embedded in your original partition. In single disk scenarios, having parallel-based design with AGs brings little advantages, thus AGs are less effective compared to having many disks (thus many heads and platters) where everything gets spread out from the POV of XFS, which in turn can write to multiple locations concurrently due to the AGs being spread out across the many disks, but I'm repeating myself :P

Another thing is, if you go for RAID (and even if you don't) do NOT use the CFQ disk scheduler. It destroys almost all of the parallelism XFS offers since some kernel versions back - quoting the XFS wiki, "As of kernel 3.2.12, the default i/o scheduler, CFQ, will defeat much of the parallelization in XFS.". The deadline, or even the noop scheduler, are recommended. As XFS has recently gone through a lot of changes, one of the thing it has also gained is to order the pending data waiting to be flushed from memory to disk in a way that it's basically doing the work of the scheduler. No, this is not to be compared with the data=ordered method offered by EXT4. Watch the video and you will understand

PS: I should correct myself. The presentation is from 2012 and NOT for 2009 like I remembered. Must have confused it with another video I saw back in 2009.

Again, since ext4 is good with little files (like the ones created on /tmp while compiling anything), the performance difference is vast for me, but I am not denying that XFS is far better for files that are bigger and are more critical than just a make file or a C source code file. Also, I watched the video, it was a good explaination of everything and I am reconsidering even to change some of my disks back to XFS now (thanks for that lol).

[microchip8]

Again, since ext4 is good with little files (like the ones created on /tmp while compiling anything), the performance difference is vast for me, but I am not denying that XFS is far better for files that are bigger and are more critical than just a make file or a C source code file. Also, I watched the video, it was a good explaination of everything and I am reconsidering even to change some of my disks back to XFS now (thanks for that lol).

There really is no need to change FS if EXT4 is giving you everything you need. I'm not advocating that. Just trying to explain the differences between the two FSes. It basically comes to this. EXT4 is designed for the low-end stuff while XFS for the mid- and high-end. Meaning, EXT4 is fine most of the time for regular desktop loads and maybe some other things too. While the theoretical maximum volume of an EXT4 partition is 1 Exabytes (EB), the max recommended is 16 Terabytes (TB) (same for max file size) due to architectural deficiencies. EXT4 is basically an improved/enhanced EXT3 borrowing some things from XFS (delayed allocation, extents and persistent pre-allocation) but it still suffers from an '80s-era design file system philosophy. If you need more than 16 TB, then XFS is much better at handling that. 16 TB or above may sound like a lot now, but in the near future we'll see such scenarios becoming more common in specific households. XFS's max volume size is 16 EB while its max file size is 8 EB. That's why people choose XFS for big storage, among other things.