LZ4 For Btrfs Arrives While Its FSCK Remains M.I.A.

[drgr33n]

I've modified the patch set to work with linux-3.2. Just wanted to trial it really so for ease I wanted to build a debian kernel package. If anyone else is interested you can grab the patch here ....

http://pastebin.com/Nsfzifvw

I've removed the snappy stuff so it's just lz4.

[drgr33n]

Hey guys,

I wrote a patch so people can benchmark with linux 3.2 as I had troubles compiling with DKMS. I think my last post was removed for linking pastebin so I'll paste here

root@drxxn:/home/zxxxn# uname -a
Linux drxxn 3.2.0-1-amd64 #2 SMP Wed Feb 29 17:13:01 GMT 2012 x86_64 GNU/Linux
root@drxxn:/home/zxxxn# mount | grep btrfs
/dev/disk/by-uuid/59xxxxxx-xxxx-xxxx-xxxx-xxxxxxx on / type btrfs (rw,noatime,thread_pool=36,compress=lz4,ssd,discar d,space_cache,inode_cache)
/dev/sdb2 on /var type btrfs (rw,relatime,compress=lz4,nospace_cache)
/dev/sdb1 on /home type btrfs (rw,relatime,thread_pool=36,compress=lz4,space_cac he,inode_cache)

hxxp: / / pastebinDOTcom/Nsfzifvw

You will have to put the url back together lol

[evergreen]

> I wrote a patch so people can benchmark with linux 3.2

Great ! Thanks !

By the way, when will Phoronix publish its updated BTRFS benchmarks, as said in the article ?

[drgr33n]

I haven't got a clue I just read the article and I wanted to test it out I haven't been able to run a full bench yet but here's one I get 3/4 of the way through on my SATA2 laptop using a crucial M4 SSD.


Current Description: Intel Core i7 720Q testing with a HP 1449 v65.38 and AMD Radeon HD 6500M/5600/5700 1024MB on Debian unstable via the Phoronix Test Suite.

New Description:

Estimated Run-Time: 9 Hours, 39 Minutes



Gzip Compression:
pts/compress-gzip-1.1.0
Test 1 of 20
Estimated Trial Run Count: 3
Estimated Test Run-Time: 3 Minutes
Estimated Time To Completion: 9 Hours, 39 Minutes
Running Pre-Test Script @ 10:21:15
Started Run 1 @ 10:21:19
Started Run 2 @ 10:21:50
Started Run 3 @ 10:22:16 [Std. Dev: 0.45%]
Running Post-Test Script @ 10:22:41

Test Results:
24.415318012238
24.383520126343
24.213171958923

Average: 24.34 Seconds


SQLite 3.7.3:
pts/sqlite-1.8.0
Test 2 of 20
Estimated Trial Run Count: 3
Estimated Test Run-Time: 26 Minutes
Estimated Time To Completion: 9 Hours, 37 Minutes
Running Pre-Test Script @ 10:22:49
Started Run 1 @ 10:22:49
Running Interim Test Script @ 10:30:39
Started Run 2 @ 10:30:41
Running Interim Test Script @ 10:38:31
Started Run 3 @ 10:38:33
Running Interim Test Script @ 10:46:53 [Std. Dev: 3.60%]
Started Run 4 @ 10:46:55
Running Interim Test Script @ 10:55:15 [Std. Dev: 3.52%]
Started Run 5 @ 10:55:17 [Std. Dev: 3.41%]
Running Post-Test Script @ 11:03:39

Test Results:
470.08013391495
470.52692008018
500.24402809143
499.56982016563
502.06850004196

Average: 488.50 Seconds


Apache Benchmark 2.2.21:
pts/apache-1.4.0
Test 3 of 20
Estimated Trial Run Count: 3
Estimated Test Run-Time: 4 Minutes
Estimated Time To Completion: 9 Hours, 12 Minutes
Running Pre-Test Script @ 11:03:46
Started Run 1 @ 11:03:52
Started Run 2 @ 11:04:54
Started Run 3 @ 11:05:58 [Std. Dev: 1.34%]
Running Post-Test Script @ 11:07:00

Test Results:
11764.3
11629.11
11454.78

Average: 11616.06 Requests Per Second


PostgreSQL pgbench 9.0.4:
pts/pgbench-1.3.0
Test 4 of 20
Estimated Trial Run Count: 3
Estimated Test Run-Time: 39 Minutes
Estimated Time To Completion: 9 Hours, 8 Minutes
Started Run 1 @ 11:07:11
Started Run 2 @ 11:39:17
Started Run 3 @ 12:11:07 [Std. Dev: 3.77%]

Test Results:
127.401032
129.557515
120.46904

Average: 125.81 TPS


Compile Bench 0.6:
pts/compilebench-1.0.0 [Test: Initial Create]
Test 5 of 20
Estimated Trial Run Count: 3
Estimated Test Run-Time: 39 Minutes
Estimated Time To Completion: 8 Hours, 30 Minutes
Started Run 1 @ 12:45:46
Started Run 2 @ 12:48:25
Started Run 3 @ 12:50:12 [Std. Dev: 4.44%]
Started Run 4 @ 12:51:53 [Std. Dev: 3.95%]
Started Run 5 @ 12:53:48 [Std. Dev: 3.78%]
Started Run 6 @ 12:56:07 [Std. Dev: 3.74%]

Test Results:
76.44
73.37
80.16
74.37
78.94
73.76

Average: 76.17 MB/s


Compile Bench 0.6:
pts/compilebench-1.0.0 [Test: Compile]
Test 6 of 20
Estimated Trial Run Count: 3
Estimated Test Run-Time: 39 Minutes
Estimated Time To Completion: 7 Hours, 52 Minutes
Started Run 1 @ 12:58:00
Started Run 2 @ 12:58:00
Started Run 3 @ 12:58:00 [Std. Dev: 1.46%]

Test Results:
638.12
644.8
626.47

Average: 636.46 MB/s


Compile Bench 0.6:
pts/compilebench-1.0.0 [Test: Read Compiled Tree]
Test 7 of 20
Estimated Trial Run Count: 3
Estimated Test Run-Time: 39 Minutes
Estimated Time To Completion: 7 Hours, 14 Minutes
Started Run 1 @ 12:58:00
Started Run 2 @ 12:58:00
Started Run 3 @ 12:58:00 [Std. Dev: 1.49%]

Test Results:
691.98
703.04
682.4

Average: 692.47 MB/s


IOzone 3.347:
pts/iozone-1.7.0 [8GB Write Performance]
Test 8 of 20
Estimated Trial Run Count: 3
Estimated Test Run-Time: 11 Minutes
Estimated Time To Completion: 6 Hours, 35 Minutes
Started Run 1 @ 12:58:06
Started Run 2 @ 12:58:28
Started Run 3 @ 12:58:58 [Std. Dev: 2.93%]

Test Results:
1207.3681640625
1221.7529296875
1155.138671875

Average: 1194.75 MB/s


IOzone 3.347:
pts/iozone-1.7.0 [8GB Read Performance]
Test 9 of 20
Estimated Trial Run Count: 3
Estimated Test Run-Time: 11 Minutes
Estimated Time To Completion: 6 Hours, 25 Minutes
Started Run 1 @ 12:59:41
Started Run 2 @ 13:01:51
Started Run 3 @ 13:03:58 [Std. Dev: 3.28%]

Test Results:
424.9775390625
407.875
435.3203125

Average: 422.72 MB/s


Dbench 4.0:
pts/dbench-1.0.0 [1 Clients]
Test 10 of 20
Estimated Trial Run Count: 3
Estimated Test Run-Time: 46 Minutes
Estimated Time To Completion: 6 Hours, 15 Minutes
Started Run 1 @ 13:06:18
Started Run 2 @ 13:18:20
Started Run 3 @ 13:30:22 [Std. Dev: 0.02%]

Test Results:
29.3254
29.3368
29.3358

Average: 29.33 MB/s


Dbench 4.0:
pts/dbench-1.0.0 [12 Clients]
Test 11 of 20
Estimated Trial Run Count: 3
Estimated Test Run-Time: 46 Minutes
Estimated Time To Completion: 5 Hours, 29 Minutes
Started Run 1 @ 13:42:30
Started Run 2 @ 13:54:34
Started Run 3 @ 14:06:36 [Std. Dev: 5.33%]
Started Run 4 @ 14:18:43 [Std. Dev: 4.76%]
Started Run 5 @ 14:30:46 [Std. Dev: 7.34%]
Started Run 6 @ 14:43:09 [Std. Dev: 6.57%]

Test Results:
168.063
165.689
152.079
156.021
183.136
167.378

Average: 165.39 MB/s


Dbench 4.0:
pts/dbench-1.0.0 [48 Clients]
Test 12 of 20
Estimated Trial Run Count: 3
Estimated Test Run-Time: 46 Minutes
Estimated Time To Completion: 4 Hours, 43 Minutes
Started Run 1 @ 14:55:18
Started Run 2 @ 15:08:21
Started Run 3 @ 15:20:26 [Std. Dev: 5.73%]
Started Run 4 @ 15:32:32 [Std. Dev: 5.11%]
Started Run 5 @ 15:45:04 [Std. Dev: 4.67%]
Started Run 6 @ 15:57:55 [Std. Dev: 4.17%]

Test Results:
362.931
376.576
405.737
367.06
391.53
382.344

Average: 381.03 MB/s

[drgr33n]

Full results here !!!

http://openbenchmarking.org/result/1203022-AR-BTRFSLZ4124

[evergreen]

Great work drgr33n !!

Clear and detailed results !

Is there any way to make these "comparable", say with non-compressed & gzip-compressed settings for example ?

[drgr33n]

Yes working on it. I've just patched my kernel against the new r58 lz4 code that has a small improvement in speed. I'm going to compare against that and LZO compression over the weekend skipping some of the benchmarks because some write zeros to the disk and compress too well which leads to misleading results.

[drgr33n]

Here's my comparison between LZ4 and LZO using 7 benchmarks.

Apache
Compile
postgresql (failed) will try again.
SQLite
Unpacking the linux kernel

http://openbenchmarking.org/result/1203040-AR-1203043AR76

[evergreen]

Thanks for the link,

though i don't understand the results to be fair

Is that a comparison between lz4 & lzo ?
If yes, why then is there one config called "btrfs-lzo-lz4"
and the other one called "2012-03-04 13:47" ?

Rgds

[rogerdpack]

not merged

I believe the lz4 stuff was never merged is that right?