SquashFS Getting Major Performance Enhancements

SquashFS is commonly used by Linux distributions as a compressed read-only file-system commonly used by the Live CD/DVD/USB versions of distributions like Ubuntu, Arch, Fedora, Debian, etc. There's some performance improvements to be merged into the Linux kernel that will make SquashFS even faster. SquashFS decompression can be multiple times faster with these patches.

Besides being used for Linux distribution Live CD/DVD/USB environments, the compressed read-only file-system is also used by lightweight embedded Linux distributions like OpenWRT so having faster performance for SquashFS is far reaching. The latest work on improving the performance of SquashFS is by Phillip Lougher, a lead developer of the file-system, and comes via a set of performance enhancements currently residing on the kernel mailing list.

Lougher is planning to get these SquashFS performance patches merged soon; it might be too late for Linux 3.13 but otherwise would be material for the Linux 3.14 kernel. The improvements to SquashFS in the Linux kernel include enhanced parallel I/O, multi-threaded decompression support using perCPU variables, directly decompressing into the page cache for file data, and other changes -- including the refactoring of the decompressor interface and code.

In the change to enhancing parallel I/O by allowing multiple decompressors, one example test done on the patch shows a system with two CPUs and 4GB of RAM going from taking 99 seconds to decompress four SquashFS file-systems to now taking just nine seconds! The multiple decompressor support for SquashFS right now though is hidden behind its own kernel configuration option (SQUASHFS_MULTI_DECOMPRESSOR).

The patch to directly decompress into the page cache for file data showed single-data decompression going from a 13 MB/s to now decompressing at 67 MB/s.