Cross-platform filesystem benchmarking

[GreatEmerald]

I have a slow external HDD that I want to use for storing large files as well as databases. Since I want the files to be accessible from both Linux and, when required, Windows, I need to use a file system that both understand. From what I can see, what fits the bill are NTFS and UDF, also possibly ext3 (FAT32 is right out due to file size limitations). All three of them have similar features, and thus I don't prefer either one of them over the other (only perhaps ext3 is lower on the list due to needing an extra driver; I'm not even certain if it still works on Win10). Therefore, in order to determine what I should use, I decided to do it the benchmarking way!

I'll be doing the comparison on a laptop that has openSUSE 13.2 as well as Windows 10 installed. The HDD will be reformatted for each test using GPartEd, because Windows is stupid and doesn't understand multiple partitions for external media anyway. I'll use PTS, and only cross-platform tests will be run. I'll publish the results here.

If you have any suggestions before I start, do tell. One thing that would be good to know is which tests make sense to run. I know some tests have a hilarious amount of options and thus take days to do; I would probably rather avoid those. For the start I'll use Michael's filesystem tests as a reference, I think.

[GreatEmerald]

Hm, well, that lasted... PTS fails with "The system cannot find the path specified." whenever trying to run IOZone, and that's the only benchmark it supports on Windows.

I suppose I'm forced to run it manually. Perhaps find a way to run timed extraction manually too.

[GreatEmerald]

I managed to get it to start running, but it says "iozone interrupted" right in the middle of the test and fails. Sigh. Oh well, I guess no cross-platform tests then, I'll use platform-specific ones.

So far I completed the NTFS benchmark, as well as UDF benchmark on Windows. I'll post more details later, but generally it's looking good for UDF. On Windows, synthetic benchmarks do show a decrease in write performance when using UDF, but that's because they use 4KiB blocks, whereas UDF uses 512B blocks instead. But the Linux unpack test actually shows an increase in performance, oddly enough. Whereas on Linux, the throughput seems to be better in general.

The partial NTFS results, compared to one of Michael's tests, are here: http://openbenchmarking.org/result/1...GREA-140708545
(I think Threaded I/O Tester is using the wrong drive there. The / drive of the system is an SSD.)

[GreatEmerald]

Yeap, so UDF is definitely faster than NTFS. However, in CompileBench, it threw me an I/O error once. After removing the testing directory manually and restarting, it worked fine. Very strange; I'm not sure what caused that.

So one result is this: http://openbenchmarking.org/result/1...GREA-150803264
The CompileBench rerun is this: http://openbenchmarking.org/result/1...GREA-150803346

Results on Windows:

unpack-linux: NTFS: 24.5 seconds, UDF: 18.8 seconds
Anvil's benchmark: NTFS: 87 points, UDF: 78 points
CrystalDiskMark:
Sequential read: both 32 MB/s
Sequential write: both 29 MB/s
Random read 4 KiB: NTFS 109 IOPS, UDF 97 IOPS
Random write 4 KiB: NTFS 280 IOPS, UDF 71 IOPS

Both the sythetic benchmarks use 4 KiB blocks, whereas UDF used 512B blocks, so it was an artificial handicap. In the one real-world test UDF actually dared better than NTFS, and the sequential tests are equal between the two even on Windows. So basically, UDF wins in terms of performance everywhere. The only downside is that it doesn't have an fsck tool; thus errors, like the one I ran across during CompileBench, are hard to deal with on Linux. Windows does have a tool for checking the errors, however, so on a dual-boot system it should be OK.