Announcement

Collapse
No announcement yet.

LZ4 For Btrfs Arrives While Its FSCK Remains M.I.A.

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • LZ4 For Btrfs Arrives While Its FSCK Remains M.I.A.

    Phoronix: LZ4 For Btrfs Arrives While Its FSCK Remains M.I.A.

    The proper fsck utility for the Btrfs file-system remains M.I.A. while a contribution from an independent developer introduces LZ4 compression support to this next-generation Linux file-system...

    http://www.phoronix.com/vr.php?view=MTA1OTQ

  • #2
    And what is that: https://git.kernel.org/?p=linux/kern...gerdonteveruse

    Comment


    • #3
      Originally posted by stikonas View Post
      For me, a branch that says "danger..dont ever use"...and a branch that doesn't compile on the latest stable kernel ( 3.2.6 ) .

      Comment


      • #4
        Originally posted by stikonas View Post
        Going by the URL, it's dangerous and shouldn't ever be used.

        Hopefully that's a temporary thing.

        Comment


        • #5
          Originally posted by stikonas View Post
          Thanks for the link!

          That is interesting. Unless I'm missing something, those commits do not add full repair capability, but they do add a repair option that can fix extent allocation errors:

          "This also includes a new --repair btrfsck option. For now it can only fix errors in the extent allocation tree."

          Comment


          • #6
            I won't be using BTRFS on any of my servers before mid 2013. I really can't trust it and most importantly I don't trust this "rush for the fsck utility". We don't need rush, we need stable things on servers, there is nothing to play when you manage data.

            Comment


            • #7
              Fsck Btrfs. Seriously.

              I can't believe it's still not ready.

              Comment


              • #8
                Compression is all nice and dandy, but I don't see Btrfs outperforming ext4 on most benchmarks even if they shave off a couple more percentage points from Btrfs+snappy.

                (That is not to deny there may be other features Btrfs provides that ext4 doesn't.)

                Comment


                • #9
                  Originally posted by bulletxt View Post
                  I don't trust this "rush for the fsck utility". We don't need rush, we need stable things on servers, there is nothing to play when you manage data.
                  It is hardly a rush. IIRC, btrfsck (with full repair capability) has been "coming soon" for more than a year now.

                  Comment


                  • #10
                    Originally posted by bastiaan View Post
                    Compression is all nice and dandy, but I don't see Btrfs outperforming ext4 on most benchmarks even if they shave off a couple more percentage points from Btrfs+snappy.

                    (That is not to deny there may be other features Btrfs provides that ext4 doesn't.)
                    BTRFS has never been about speed, it's more about durability (data integrity), snapshotting and the capability to handle very large data sets. So IMHO it's not that much of a client FS and more of a stable server FS.

                    Comment


                    • #11
                      I've modified the patch set to work with linux-3.2. Just wanted to trial it really so for ease I wanted to build a debian kernel package. If anyone else is interested you can grab the patch here ....


                      http://pastebin.com/Nsfzifvw

                      I've removed the snappy stuff so it's just lz4.

                      Comment


                      • #12
                        Hey guys,

                        I wrote a patch so people can benchmark with linux 3.2 as I had troubles compiling with DKMS. I think my last post was removed for linking pastebin so I'll paste here

                        root@drxxn:/home/zxxxn# uname -a
                        Linux drxxn 3.2.0-1-amd64 #2 SMP Wed Feb 29 17:13:01 GMT 2012 x86_64 GNU/Linux
                        root@drxxn:/home/zxxxn# mount | grep btrfs
                        /dev/disk/by-uuid/59xxxxxx-xxxx-xxxx-xxxx-xxxxxxx on / type btrfs (rw,noatime,thread_pool=36,compress=lz4,ssd,discar d,space_cache,inode_cache)
                        /dev/sdb2 on /var type btrfs (rw,relatime,compress=lz4,nospace_cache)
                        /dev/sdb1 on /home type btrfs (rw,relatime,thread_pool=36,compress=lz4,space_cac he,inode_cache)

                        hxxp: / / pastebinDOTcom/Nsfzifvw

                        You will have to put the url back together lol

                        Comment


                        • #13
                          > I wrote a patch so people can benchmark with linux 3.2

                          Great ! Thanks !

                          By the way, when will Phoronix publish its updated BTRFS benchmarks, as said in the article ?

                          Comment


                          • #14
                            I haven't got a clue I just read the article and I wanted to test it out I haven't been able to run a full bench yet but here's one I get 3/4 of the way through on my SATA2 laptop using a crucial M4 SSD.


                            Current Description: Intel Core i7 720Q testing with a HP 1449 v65.38 and AMD Radeon HD 6500M/5600/5700 1024MB on Debian unstable via the Phoronix Test Suite.

                            New Description:

                            Estimated Run-Time: 9 Hours, 39 Minutes



                            Gzip Compression:
                            pts/compress-gzip-1.1.0
                            Test 1 of 20
                            Estimated Trial Run Count: 3
                            Estimated Test Run-Time: 3 Minutes
                            Estimated Time To Completion: 9 Hours, 39 Minutes
                            Running Pre-Test Script @ 10:21:15
                            Started Run 1 @ 10:21:19
                            Started Run 2 @ 10:21:50
                            Started Run 3 @ 10:22:16 [Std. Dev: 0.45%]
                            Running Post-Test Script @ 10:22:41

                            Test Results:
                            24.415318012238
                            24.383520126343
                            24.213171958923

                            Average: 24.34 Seconds


                            SQLite 3.7.3:
                            pts/sqlite-1.8.0
                            Test 2 of 20
                            Estimated Trial Run Count: 3
                            Estimated Test Run-Time: 26 Minutes
                            Estimated Time To Completion: 9 Hours, 37 Minutes
                            Running Pre-Test Script @ 10:22:49
                            Started Run 1 @ 10:22:49
                            Running Interim Test Script @ 10:30:39
                            Started Run 2 @ 10:30:41
                            Running Interim Test Script @ 10:38:31
                            Started Run 3 @ 10:38:33
                            Running Interim Test Script @ 10:46:53 [Std. Dev: 3.60%]
                            Started Run 4 @ 10:46:55
                            Running Interim Test Script @ 10:55:15 [Std. Dev: 3.52%]
                            Started Run 5 @ 10:55:17 [Std. Dev: 3.41%]
                            Running Post-Test Script @ 11:03:39

                            Test Results:
                            470.08013391495
                            470.52692008018
                            500.24402809143
                            499.56982016563
                            502.06850004196

                            Average: 488.50 Seconds


                            Apache Benchmark 2.2.21:
                            pts/apache-1.4.0
                            Test 3 of 20
                            Estimated Trial Run Count: 3
                            Estimated Test Run-Time: 4 Minutes
                            Estimated Time To Completion: 9 Hours, 12 Minutes
                            Running Pre-Test Script @ 11:03:46
                            Started Run 1 @ 11:03:52
                            Started Run 2 @ 11:04:54
                            Started Run 3 @ 11:05:58 [Std. Dev: 1.34%]
                            Running Post-Test Script @ 11:07:00

                            Test Results:
                            11764.3
                            11629.11
                            11454.78

                            Average: 11616.06 Requests Per Second


                            PostgreSQL pgbench 9.0.4:
                            pts/pgbench-1.3.0
                            Test 4 of 20
                            Estimated Trial Run Count: 3
                            Estimated Test Run-Time: 39 Minutes
                            Estimated Time To Completion: 9 Hours, 8 Minutes
                            Started Run 1 @ 11:07:11
                            Started Run 2 @ 11:39:17
                            Started Run 3 @ 12:11:07 [Std. Dev: 3.77%]

                            Test Results:
                            127.401032
                            129.557515
                            120.46904

                            Average: 125.81 TPS


                            Compile Bench 0.6:
                            pts/compilebench-1.0.0 [Test: Initial Create]
                            Test 5 of 20
                            Estimated Trial Run Count: 3
                            Estimated Test Run-Time: 39 Minutes
                            Estimated Time To Completion: 8 Hours, 30 Minutes
                            Started Run 1 @ 12:45:46
                            Started Run 2 @ 12:48:25
                            Started Run 3 @ 12:50:12 [Std. Dev: 4.44%]
                            Started Run 4 @ 12:51:53 [Std. Dev: 3.95%]
                            Started Run 5 @ 12:53:48 [Std. Dev: 3.78%]
                            Started Run 6 @ 12:56:07 [Std. Dev: 3.74%]

                            Test Results:
                            76.44
                            73.37
                            80.16
                            74.37
                            78.94
                            73.76

                            Average: 76.17 MB/s


                            Compile Bench 0.6:
                            pts/compilebench-1.0.0 [Test: Compile]
                            Test 6 of 20
                            Estimated Trial Run Count: 3
                            Estimated Test Run-Time: 39 Minutes
                            Estimated Time To Completion: 7 Hours, 52 Minutes
                            Started Run 1 @ 12:58:00
                            Started Run 2 @ 12:58:00
                            Started Run 3 @ 12:58:00 [Std. Dev: 1.46%]

                            Test Results:
                            638.12
                            644.8
                            626.47

                            Average: 636.46 MB/s


                            Compile Bench 0.6:
                            pts/compilebench-1.0.0 [Test: Read Compiled Tree]
                            Test 7 of 20
                            Estimated Trial Run Count: 3
                            Estimated Test Run-Time: 39 Minutes
                            Estimated Time To Completion: 7 Hours, 14 Minutes
                            Started Run 1 @ 12:58:00
                            Started Run 2 @ 12:58:00
                            Started Run 3 @ 12:58:00 [Std. Dev: 1.49%]

                            Test Results:
                            691.98
                            703.04
                            682.4

                            Average: 692.47 MB/s


                            IOzone 3.347:
                            pts/iozone-1.7.0 [8GB Write Performance]
                            Test 8 of 20
                            Estimated Trial Run Count: 3
                            Estimated Test Run-Time: 11 Minutes
                            Estimated Time To Completion: 6 Hours, 35 Minutes
                            Started Run 1 @ 12:58:06
                            Started Run 2 @ 12:58:28
                            Started Run 3 @ 12:58:58 [Std. Dev: 2.93%]

                            Test Results:
                            1207.3681640625
                            1221.7529296875
                            1155.138671875

                            Average: 1194.75 MB/s


                            IOzone 3.347:
                            pts/iozone-1.7.0 [8GB Read Performance]
                            Test 9 of 20
                            Estimated Trial Run Count: 3
                            Estimated Test Run-Time: 11 Minutes
                            Estimated Time To Completion: 6 Hours, 25 Minutes
                            Started Run 1 @ 12:59:41
                            Started Run 2 @ 13:01:51
                            Started Run 3 @ 13:03:58 [Std. Dev: 3.28%]

                            Test Results:
                            424.9775390625
                            407.875
                            435.3203125

                            Average: 422.72 MB/s


                            Dbench 4.0:
                            pts/dbench-1.0.0 [1 Clients]
                            Test 10 of 20
                            Estimated Trial Run Count: 3
                            Estimated Test Run-Time: 46 Minutes
                            Estimated Time To Completion: 6 Hours, 15 Minutes
                            Started Run 1 @ 13:06:18
                            Started Run 2 @ 13:18:20
                            Started Run 3 @ 13:30:22 [Std. Dev: 0.02%]

                            Test Results:
                            29.3254
                            29.3368
                            29.3358

                            Average: 29.33 MB/s


                            Dbench 4.0:
                            pts/dbench-1.0.0 [12 Clients]
                            Test 11 of 20
                            Estimated Trial Run Count: 3
                            Estimated Test Run-Time: 46 Minutes
                            Estimated Time To Completion: 5 Hours, 29 Minutes
                            Started Run 1 @ 13:42:30
                            Started Run 2 @ 13:54:34
                            Started Run 3 @ 14:06:36 [Std. Dev: 5.33%]
                            Started Run 4 @ 14:18:43 [Std. Dev: 4.76%]
                            Started Run 5 @ 14:30:46 [Std. Dev: 7.34%]
                            Started Run 6 @ 14:43:09 [Std. Dev: 6.57%]

                            Test Results:
                            168.063
                            165.689
                            152.079
                            156.021
                            183.136
                            167.378

                            Average: 165.39 MB/s


                            Dbench 4.0:
                            pts/dbench-1.0.0 [48 Clients]
                            Test 12 of 20
                            Estimated Trial Run Count: 3
                            Estimated Test Run-Time: 46 Minutes
                            Estimated Time To Completion: 4 Hours, 43 Minutes
                            Started Run 1 @ 14:55:18
                            Started Run 2 @ 15:08:21
                            Started Run 3 @ 15:20:26 [Std. Dev: 5.73%]
                            Started Run 4 @ 15:32:32 [Std. Dev: 5.11%]
                            Started Run 5 @ 15:45:04 [Std. Dev: 4.67%]
                            Started Run 6 @ 15:57:55 [Std. Dev: 4.17%]

                            Test Results:
                            362.931
                            376.576
                            405.737
                            367.06
                            391.53
                            382.344

                            Average: 381.03 MB/s

                            Comment


                            • #15
                              Full results here !!!

                              http://openbenchmarking.org/result/1...AR-BTRFSLZ4124

                              Comment

                              Working...
                              X