Announcement

Collapse
No announcement yet.

LZ4 For Btrfs Arrives While Its FSCK Remains M.I.A.

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • #11
    I've modified the patch set to work with linux-3.2. Just wanted to trial it really so for ease I wanted to build a debian kernel package. If anyone else is interested you can grab the patch here ....


    http://pastebin.com/Nsfzifvw

    I've removed the snappy stuff so it's just lz4.

    Comment


    • #12
      Hey guys,

      I wrote a patch so people can benchmark with linux 3.2 as I had troubles compiling with DKMS. I think my last post was removed for linking pastebin so I'll paste here

      root@drxxn:/home/zxxxn# uname -a
      Linux drxxn 3.2.0-1-amd64 #2 SMP Wed Feb 29 17:13:01 GMT 2012 x86_64 GNU/Linux
      root@drxxn:/home/zxxxn# mount | grep btrfs
      /dev/disk/by-uuid/59xxxxxx-xxxx-xxxx-xxxx-xxxxxxx on / type btrfs (rw,noatime,thread_pool=36,compress=lz4,ssd,discar d,space_cache,inode_cache)
      /dev/sdb2 on /var type btrfs (rw,relatime,compress=lz4,nospace_cache)
      /dev/sdb1 on /home type btrfs (rw,relatime,thread_pool=36,compress=lz4,space_cac he,inode_cache)

      hxxp: / / pastebinDOTcom/Nsfzifvw

      You will have to put the url back together lol

      Comment


      • #13
        > I wrote a patch so people can benchmark with linux 3.2

        Great ! Thanks !

        By the way, when will Phoronix publish its updated BTRFS benchmarks, as said in the article ?

        Comment


        • #14
          I haven't got a clue I just read the article and I wanted to test it out I haven't been able to run a full bench yet but here's one I get 3/4 of the way through on my SATA2 laptop using a crucial M4 SSD.


          Current Description: Intel Core i7 720Q testing with a HP 1449 v65.38 and AMD Radeon HD 6500M/5600/5700 1024MB on Debian unstable via the Phoronix Test Suite.

          New Description:

          Estimated Run-Time: 9 Hours, 39 Minutes



          Gzip Compression:
          pts/compress-gzip-1.1.0
          Test 1 of 20
          Estimated Trial Run Count: 3
          Estimated Test Run-Time: 3 Minutes
          Estimated Time To Completion: 9 Hours, 39 Minutes
          Running Pre-Test Script @ 10:21:15
          Started Run 1 @ 10:21:19
          Started Run 2 @ 10:21:50
          Started Run 3 @ 10:22:16 [Std. Dev: 0.45%]
          Running Post-Test Script @ 10:22:41

          Test Results:
          24.415318012238
          24.383520126343
          24.213171958923

          Average: 24.34 Seconds


          SQLite 3.7.3:
          pts/sqlite-1.8.0
          Test 2 of 20
          Estimated Trial Run Count: 3
          Estimated Test Run-Time: 26 Minutes
          Estimated Time To Completion: 9 Hours, 37 Minutes
          Running Pre-Test Script @ 10:22:49
          Started Run 1 @ 10:22:49
          Running Interim Test Script @ 10:30:39
          Started Run 2 @ 10:30:41
          Running Interim Test Script @ 10:38:31
          Started Run 3 @ 10:38:33
          Running Interim Test Script @ 10:46:53 [Std. Dev: 3.60%]
          Started Run 4 @ 10:46:55
          Running Interim Test Script @ 10:55:15 [Std. Dev: 3.52%]
          Started Run 5 @ 10:55:17 [Std. Dev: 3.41%]
          Running Post-Test Script @ 11:03:39

          Test Results:
          470.08013391495
          470.52692008018
          500.24402809143
          499.56982016563
          502.06850004196

          Average: 488.50 Seconds


          Apache Benchmark 2.2.21:
          pts/apache-1.4.0
          Test 3 of 20
          Estimated Trial Run Count: 3
          Estimated Test Run-Time: 4 Minutes
          Estimated Time To Completion: 9 Hours, 12 Minutes
          Running Pre-Test Script @ 11:03:46
          Started Run 1 @ 11:03:52
          Started Run 2 @ 11:04:54
          Started Run 3 @ 11:05:58 [Std. Dev: 1.34%]
          Running Post-Test Script @ 11:07:00

          Test Results:
          11764.3
          11629.11
          11454.78

          Average: 11616.06 Requests Per Second


          PostgreSQL pgbench 9.0.4:
          pts/pgbench-1.3.0
          Test 4 of 20
          Estimated Trial Run Count: 3
          Estimated Test Run-Time: 39 Minutes
          Estimated Time To Completion: 9 Hours, 8 Minutes
          Started Run 1 @ 11:07:11
          Started Run 2 @ 11:39:17
          Started Run 3 @ 12:11:07 [Std. Dev: 3.77%]

          Test Results:
          127.401032
          129.557515
          120.46904

          Average: 125.81 TPS


          Compile Bench 0.6:
          pts/compilebench-1.0.0 [Test: Initial Create]
          Test 5 of 20
          Estimated Trial Run Count: 3
          Estimated Test Run-Time: 39 Minutes
          Estimated Time To Completion: 8 Hours, 30 Minutes
          Started Run 1 @ 12:45:46
          Started Run 2 @ 12:48:25
          Started Run 3 @ 12:50:12 [Std. Dev: 4.44%]
          Started Run 4 @ 12:51:53 [Std. Dev: 3.95%]
          Started Run 5 @ 12:53:48 [Std. Dev: 3.78%]
          Started Run 6 @ 12:56:07 [Std. Dev: 3.74%]

          Test Results:
          76.44
          73.37
          80.16
          74.37
          78.94
          73.76

          Average: 76.17 MB/s


          Compile Bench 0.6:
          pts/compilebench-1.0.0 [Test: Compile]
          Test 6 of 20
          Estimated Trial Run Count: 3
          Estimated Test Run-Time: 39 Minutes
          Estimated Time To Completion: 7 Hours, 52 Minutes
          Started Run 1 @ 12:58:00
          Started Run 2 @ 12:58:00
          Started Run 3 @ 12:58:00 [Std. Dev: 1.46%]

          Test Results:
          638.12
          644.8
          626.47

          Average: 636.46 MB/s


          Compile Bench 0.6:
          pts/compilebench-1.0.0 [Test: Read Compiled Tree]
          Test 7 of 20
          Estimated Trial Run Count: 3
          Estimated Test Run-Time: 39 Minutes
          Estimated Time To Completion: 7 Hours, 14 Minutes
          Started Run 1 @ 12:58:00
          Started Run 2 @ 12:58:00
          Started Run 3 @ 12:58:00 [Std. Dev: 1.49%]

          Test Results:
          691.98
          703.04
          682.4

          Average: 692.47 MB/s


          IOzone 3.347:
          pts/iozone-1.7.0 [8GB Write Performance]
          Test 8 of 20
          Estimated Trial Run Count: 3
          Estimated Test Run-Time: 11 Minutes
          Estimated Time To Completion: 6 Hours, 35 Minutes
          Started Run 1 @ 12:58:06
          Started Run 2 @ 12:58:28
          Started Run 3 @ 12:58:58 [Std. Dev: 2.93%]

          Test Results:
          1207.3681640625
          1221.7529296875
          1155.138671875

          Average: 1194.75 MB/s


          IOzone 3.347:
          pts/iozone-1.7.0 [8GB Read Performance]
          Test 9 of 20
          Estimated Trial Run Count: 3
          Estimated Test Run-Time: 11 Minutes
          Estimated Time To Completion: 6 Hours, 25 Minutes
          Started Run 1 @ 12:59:41
          Started Run 2 @ 13:01:51
          Started Run 3 @ 13:03:58 [Std. Dev: 3.28%]

          Test Results:
          424.9775390625
          407.875
          435.3203125

          Average: 422.72 MB/s


          Dbench 4.0:
          pts/dbench-1.0.0 [1 Clients]
          Test 10 of 20
          Estimated Trial Run Count: 3
          Estimated Test Run-Time: 46 Minutes
          Estimated Time To Completion: 6 Hours, 15 Minutes
          Started Run 1 @ 13:06:18
          Started Run 2 @ 13:18:20
          Started Run 3 @ 13:30:22 [Std. Dev: 0.02%]

          Test Results:
          29.3254
          29.3368
          29.3358

          Average: 29.33 MB/s


          Dbench 4.0:
          pts/dbench-1.0.0 [12 Clients]
          Test 11 of 20
          Estimated Trial Run Count: 3
          Estimated Test Run-Time: 46 Minutes
          Estimated Time To Completion: 5 Hours, 29 Minutes
          Started Run 1 @ 13:42:30
          Started Run 2 @ 13:54:34
          Started Run 3 @ 14:06:36 [Std. Dev: 5.33%]
          Started Run 4 @ 14:18:43 [Std. Dev: 4.76%]
          Started Run 5 @ 14:30:46 [Std. Dev: 7.34%]
          Started Run 6 @ 14:43:09 [Std. Dev: 6.57%]

          Test Results:
          168.063
          165.689
          152.079
          156.021
          183.136
          167.378

          Average: 165.39 MB/s


          Dbench 4.0:
          pts/dbench-1.0.0 [48 Clients]
          Test 12 of 20
          Estimated Trial Run Count: 3
          Estimated Test Run-Time: 46 Minutes
          Estimated Time To Completion: 4 Hours, 43 Minutes
          Started Run 1 @ 14:55:18
          Started Run 2 @ 15:08:21
          Started Run 3 @ 15:20:26 [Std. Dev: 5.73%]
          Started Run 4 @ 15:32:32 [Std. Dev: 5.11%]
          Started Run 5 @ 15:45:04 [Std. Dev: 4.67%]
          Started Run 6 @ 15:57:55 [Std. Dev: 4.17%]

          Test Results:
          362.931
          376.576
          405.737
          367.06
          391.53
          382.344

          Average: 381.03 MB/s

          Comment


          • #15
            Full results here !!!

            http://openbenchmarking.org/result/1...AR-BTRFSLZ4124

            Comment


            • #16
              Great work drgr33n !!

              Clear and detailed results !

              Is there any way to make these "comparable", say with non-compressed & gzip-compressed settings for example ?

              Comment


              • #17
                Yes working on it. I've just patched my kernel against the new r58 lz4 code that has a small improvement in speed. I'm going to compare against that and LZO compression over the weekend skipping some of the benchmarks because some write zeros to the disk and compress too well which leads to misleading results.

                Comment


                • #18
                  Here's my comparison between LZ4 and LZO using 7 benchmarks.

                  Apache
                  Compile
                  postgresql (failed) will try again.
                  SQLite
                  Unpacking the linux kernel

                  http://openbenchmarking.org/result/1...AR-1203043AR76

                  Comment


                  • #19
                    Thanks for the link,

                    though i don't understand the results to be fair

                    Is that a comparison between lz4 & lzo ?
                    If yes, why then is there one config called "btrfs-lzo-lz4"
                    and the other one called "2012-03-04 13:47" ?

                    Rgds

                    Comment


                    • #20
                      not merged

                      I believe the lz4 stuff was never merged is that right?

                      Comment

                      Working...
                      X