Btrfs Seeing Nice Performance Improvements For Linux 5.17

[skeevy420]

Calm down. People may care more about space efficiency than throughput, e.g. if you compress more, then you write less and consequently enhance the lifespan of your SSD or squeeze more stuff on your ram disk.

Their whole shtick is performance and speed which would be raw throughput for a ram disk. You lose more performance and speed with a Zstd ramdisk than with an LZ4 one.

The the SSD is the calm down part. That only matters if you use Zswap and a swap partition. Zram, what CachyOS uses, put swap in ram and is used for omitting swap on a disk...you can have both, but that's not how Zram is designed to work -- the ram and disk be two separate swap devices and which one is used depends on how you set the priority.

Zswap uses ram and then spills over to the swap on disk once the ram is used. That scenario is what you're thinking of and is where using Zstd could potentially bring SSD lifespan benefits. IMHO, you're better off using Zram and not having an SSD based swap if you're worried about drive life.

[pheldens]

sticking to softraid with ext4

[pal666]

I'm disappointed at Btrfs's extended file attributes support (a posix standard) - in that it only supports ~20KiB or less (per file)

btrfs has limit per xattr, not per file - you can have as many xattrs per file as you'd like(just split your thumbnail). and this limit can be increased at mkfs time, but see below

XFS on the other hand supports much larger extended file attributes

not much, vfs places 64kb limit on all filesystems

[yump]

Their whole shtick is performance and speed which would be raw throughput for a ram disk. You lose more performance and speed with a Zstd ramdisk than with an LZ4 one.

[...]

Zram, what CachyOS uses, put swap in ram and is used for omitting swap on a disk...you can have both, but that's not how Zram is designed to work -- the ram and disk be two separate swap devices and which one is used depends on how you set the priority.

Swap-on-zram uses the regular kernel swap code, which was originally written for swap-on-disk. Compared to zram of any kind, disk has infinity compression ratio and is extremely slow.

Zstd, which is closest to the design backing device, is clearly the correct choice.

[PeeJay]

5.16 Btrfs is already noticeably faster with less latency and less write spikes. Might be time for some benchmarks again

[cl333r]

not much, vfs places 64kb limit on all filesystems

All thumbnails are 128x128 or less and <=64KiB is just enough for ~100% of cases (90% of the time they're 30-35KiB so there's plenty of space for other x attrs), whereas 20KiB is not enough most of the time. That's big for this usecase, which is why XFS is much better in this regard.
Again - splitting, putting together - fuck that. It would be understandable if XFS was substantially slower because of this, but in fact XFS is one of the fasted filesystems, hence Btrfs has to fix its shit, though it's not only Btrfs.

[yump]

All thumbnails are 128x128 or less and <=64KiB is just enough for ~100% of cases (90% of the time they're 30-35KiB so there's plenty of space for other x attrs), whereas 20KiB is not enough most of the time. That's big for this usecase, which is why XFS is much better in this regard.

Honestly, I don't think storing thumbnails in extended attributes is a good idea. It could amount to a lot of data if the source files are small and numerous, which the user might have trouble identifying as the cause of their disk being full. And even the filesystems that *can* handle large objects stored that way, I expect high write amplification because of needing one fs operation per thumbnailed file.

Better, perhaps, to store thumbnails in a per-filesystem SQLite database, as close to the root of the filesystem as you can get without losing write permissions.

That lets you batch thumbnail cache updates into a single flash-friendly sequential write, makes the space occupied by thumbnails more transparent to the user and possible to in/exclude from backups, and still has the security properties of storing thumbnails alongside the files. Plus you don't have to exercise any semi-exotic filesystem features.

[cl333r]

Honestly, I don't think storing thumbnails in extended attributes is a good idea. It could amount to a lot of data if the source files are small and numerous, which the user might have trouble identifying as the cause of their disk being full. And even the filesystems that *can* handle large objects stored that way, I expect high write amplification because of needing one fs operation per thumbnailed file.

Better, perhaps, to store thumbnails in a per-filesystem SQLite database, as close to the root of the filesystem as you can get without losing write permissions.

That lets you batch thumbnail cache updates into a single flash-friendly sequential write, makes the space occupied by thumbnails more transparent to the user and possible to in/exclude from backups, and still has the security properties of storing thumbnails alongside the files. Plus you don't have to exercise any semi-exotic filesystem features.

Nonsense.
Not to mention extended file attributes are standard posix.

[pal666]

Again - splitting, putting together - fuck that. It would be understandable if XFS was substantially slower because of this, but in fact XFS is one of the fasted filesystems, hence Btrfs has to fix its shit, though it's not only Btrfs.

are you writing file manager for yourself, or you are also expecting other people to use it? if for yourself, you can increase limit during mkfs. if for others, you can't expect xfs anyway

[cl333r]

are you writing file manager for yourself, or you are also expecting other people to use it? if for yourself, you can increase limit during mkfs. if for others, you can't expect xfs anyway

For myself and others to use, but that's a different story. I'm not "expecting XFS", it writes the thumbnails to ext attributes (which succeed on XFS) and if it fails then it writes them as separate little files to a temp location that survives reboot and once in a few days the older thumbnails are automatically removed to avoid (the unlikely case) of using up too much space.
That's why ext attrs are much better - you don't have to do any housekeeping cause when the user deletes the file he automatically deletes the thumbnail and no matter where he copies the file he also copies the thumbnail (it doesn't remain a dangling file in the temp folder).

But in some cases it's not desirable, for example if the files are part of a shared git project and the user wouldn't like the file browser to touch the (image) files of the project. In such cases one can delete such thumbnails by selecting said files and pressing Ctrl+Shift+U. It's easy to see which files have thumbnails because they're subtly marked as such with a green dot [1].

But of course some people won't like to deal with this either and would like to just disable saving thumbnails to extended file attributes, for this I'll add an option in the prefs menu soon.


[1] https://imgur.com/a/U3X1coJ