Announcement

Collapse
No announcement yet.

Transparent Decompression Support For EXT4

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • Transparent Decompression Support For EXT4

    Phoronix: Transparent Decompression Support For EXT4

    A Mozilla engineer has been developing transparent decompression support for the EXT4 file-system to benefit Firefox on Android, among other use-cases...

    Phoronix, Linux Hardware Reviews, Linux hardware benchmarks, Linux server benchmarks, Linux benchmarking, Desktop Linux, Linux performance, Open Source graphics, Linux How To, Ubuntu benchmarks, Ubuntu hardware, Phoronix Test Suite

  • #2
    ehhhh it may have its uses, but modifying the kernel for your user app? really?

    Comment


    • #3
      Great!
      It also doesn't decompress all the bytes properly, but this is on purpose to gain speed, on average the decompression mangles about 1% of the file contents while gaining 3% of speed. You can even gain up to 7% of speed if you're OK with the decompression overwriting some parts of some near-by files, to enable this pass the "elephant-in-the-vessel-store" option. But since the project is in alpha the numbers aren't final yet.

      Comment


      • #4
        Originally posted by pheldens View Post
        ehhhh it may have its uses, but modifying the kernel for your user app? really?
        Actually the topic of a generic compression / decompression framework in kernel got brought up in the lz4 discussion. this may be the first step towards getting it
        All opinions are my own not those of my employer if you know who they are.

        Comment


        • #5
          Why backport btrfs features to ext4 when you could just work on getting btrfs mainstream-ready? (yeah, I know Android would never switch filesystem at this point). Also, I'm curious why Android isn't looking at f2fs considering Samsungs work to make it really good for the kind of internal flash phones use.

          Comment


          • #6
            Reminds me of the good old days of DoubleDisk in MS-DOS 6.

            Comment


            • #7
              Originally posted by pheldens View Post
              ehhhh it may have its uses, but modifying the kernel for your user app? really?
              I don't have statistics, but I'd wager that the greater bulk of kernel contributions are made with specific use cases as the primary motivator. I don't think that just because this particular one targets improved performance for a single app makes it any less legitimate a reason for the work. This work has the potential to benefit a great deal more use cases than just the one that is trying to be improved through this effort.

              Comment


              • #8
                Is there any advantage of this over upx? http://upx.sourceforge.net/

                Comment


                • #9
                  Pack libraries

                  Originally posted by TheOne View Post
                  Is there any advantage of this over upx?
                  UPX compresses executable binaries.
                  So UPX would be good, for example, to compress "/usr/bin/firefox" (or "/usr/libexec/firefox-bin", depending on your distro ;-) )

                  This is used to compress dynamically loaded libraries (.so files), which is not and can't be supported by UPX (due to the way libraries are loaded).
                  So this would be good, for exemple to compress "/usr/lib64/xulrunner/libxul.so"

                  Think of it as a UPX-for-SO-files, which need to be put into the kernel so it intercepts the necessary hooks.
                  (Like UPX, and unlike BTRFS' various Zlib/LZO/LZ4 compressors, files need to be compressed in advance offline
                  Also the decompression happens at a very last step in the file system when the .so file is being streamed into memory. It doesn't work with memory maps or seeks.
                  Only this time the decompressor is in the filesystem module and not embed inside the so-file (like upx on elfs) because that simply is technically impossible).

                  Comment


                  • #10
                    Windows

                    Note that, on windows, UPX *IS* able to compress DLL files, because they function completely differently.
                    (They are basically just a different type of executable, the same PE/COFF format as used in the .EXE files, and are loaded and executed the same).

                    Comment

                    Working...
                    X