Btrfs vs. F2FS Multi-SSD Performance On Linux 4.11

[milkylainen]

Just some notes:

- ext4 is fast, which is its only advantage compared to btrfs
- ext4 doesn't have data compression, btrfs+lzo provides about 10% extra space on my nearly full SSD compared to ext4
- ext4 doesn't have data checksumming which means it is unable to detect data corruption, /sbin/btrfs scrub
- a GPU crash caused hard-reset of the machine, ext4 partition got slightly corrupted, I switched from ext4 to btrfs because of its copy-on-write design which might be able to lower the probability of filesystem corruption due to hard reset or power outage
- I noticed that btrfs is slightly slower than ext4 after switching to btrfs, but the real impact of this on work is negligible

You are right, but the point I was trying to make is (with my personal experience and and what I read into the benchmarks) the erratic performance. For me, btrfs was resonable at times and with other operations it was really, really slow. Not only is ext4, fast, but from my perspective, very consistent in the performance department.

[benmoran]

What filesystem would you guys recommend when storing large git projects on SSD? I'm thinking of storing projects such as linux kernel, android (aosp), freedesktop projects. F2FS or BTRFS with LZO compression?

Many times I have end up in situations where it takes forever to delete thousands of small files.

If the performance of btrfs is OK for your use cases, use subvolumes. They essentially look and act like folders, so you can make one for each project . When you're done with the project, you can just "btrfs subvol delete <vol>" to get rid of it instantly. (btrfs-cleaner does it's business in the background transparantly).

Just be sure to remember that the subvolumes are not folders, or you'll wonder why you can't "rm -r" them

[starshipeleven]

Erratic, you mentally retarded yokel.

How you see performance as erratic again, you zyppy zebra? RAIDing it yelds same performance benefits, its performance tanks with many small read/writes. That's not erratic, it's a pattern that other filesystems also usually have.

I said both features and other qualities are more important.

Said in a sarcastic and ambiguous tone. Many people thought you were trolling.

But as you apparently thrive on personal insults.

Please remember you aren't the only troll here, and surely not the best.

[pal666]

milkylainen - idiot? All I see after all these years he is unable to understand that btrfs is copy on write

[pal666]

I'm thinking of storing projects such as linux kernel, android (aosp), freedesktop projects. F2FS or BTRFS with LZO compression?

Many times I have end up in situations where it takes forever to delete thousands of small files.

git stores history in packs, i.e. in small number of large compressed files

[DrYak]

Small reminder : RAID1 isn't optimized well for reading under BTRFS yet. (It doesn't load balance nicely)

This is due to how BTRFS and MD handle allocation respectively.

With MD, things are dead simple :
- it does its allocations by full block devices
- first copy, the "copy A" goes on disk 1 of the pool
- second copy, the "copy B" goes on disk 2 of the pool
If you receive multiple read request, spread them between copy A and copy B (as if it where a RAID 0) and only switch to the other copy in case the first read fails.
Thus reads will be nicely spread between drive 1 and drive 2. You get up to (theoretically) twice the read speed.

With BTRFS things are more complex :
- BTRFS has its own LV-like allocation layer
- BTRFS allocates data in block groups (of 1GiB each).
- applies RAID when it allocates a new block group. (Which also means the previous block groups can be in a different mode, until they are re-balanaced)
- there's not guaranteed mapping of the block groups. The only guarantee is that in RAID1 mode, copy A of the block group and copy B of the block group will go on 2 different drives of the pool (as opposed to the "dup" modes which can write 2 copies on the same driver, in order to do redundancy on a single spinning drive), but nothing tells you which drivers those will be.
- thus it's hard to load balance. When 2 concurrent reads arrive, you can't just spread them between "copy A" and "copy B" like above because due to random chances in the allocation history, both copies might be on the same disk.
- you need extra code to check on which disk is each copy to correctly choose which copy to read from in priority.
That code isn't there yet.

BTRFS is basically as if each extent you allocate with LVM had its own mini MD raid overlayed on top.

(You can now understand why RAID5/6 is even more complex, and isn't stable yet).

So that's why RAID1 currently doesn't show much performance improvement in BTRFS.
(As opposed to a classical MD's RAID1)
Unlike RAID0, which is by definition spread among drives and thus load-balances naturally.

there's only two tests where BTRFS has noticeably worse performance: The SQLite bench{...}

Which again is due to copy-on-write and log-structured filesystem not playing well with huge binary blobs that have their own inner file-system-like structure with its own recovery mecanism (e.g.: databases, virtual disk images, etc.).

Which is why systems like ZFS automatically detect this for you and turn off the CoW.

BTRFS on its side, offers you the option to manually create files with attribute:

chattr +C

Or since recently, it has an auto-defragger that can detect and optimize such situation (e.g.: multiple fragmentation within the same 64k-256k block)

The question is : how does F2FS manage its speed with SQLite while being a log-structured file system...

Btrfs has to be functional, not fast. Development isn't focusing on performance. Show me transparent compression, snapshots, copy on write or deduplication in F2FS or another "fast" file system.

Note that F2FS is a log-structured file system (closely related to CoW). It has similar crash-resilient properties. And is also similarly gentler to flash-media (the reason why F2FS was designed log-structured. Just like UDF, btw)

And regarding all the features :
That's also why Kent Overstreet is working on bcachefs. Hope is that he'll manage to make it, but with a FS that is a tad bit less weird to use.
But currently BCacheFS is far from having implemented the best but most complex bits of BTRFS and ZFS (no snapshotting yet, compression not really usable)

Making FS is hard.
Making FS with advanced features is even harder.

[thelongdivider]

BTRFS is perfect for me. On my very fast NVME boot ssd, it provides compression and subvolumes to dynamically allocate space between /, /var, /opt, and /home. On my HDDs, compression and snapshots are amazing to keep my files in a versioned type of way. Performance is good enough, achieving 2200 MB/s read/write on the SSD.