Btrfs Changes For Linux 4.21 Prepped With Swapfile Support, Logging Improvements

[phoronix]

Phoronix: Btrfs Changes For Linux 4.21 Prepped With Swapfile Support, Logging Improvements

Due to the Linux 4.21 merge window expected to open up next week just prior to Christmas, some kernel subsystem maintainers who won't be around in the days ahead have been sending in their pull requests early. Among those with early feature pulls is David Sterba continuing to oversee the Btrfs file-system development...

http://www.phoronix.com/scan.php?page=news_item&px=Linux-4.21-Btrfs-Changes

[jacob]

I wonder why is swapfile compression not supported?

OTOH not being able to snapshot the swapfile doesn't surprise me. It's a very weird and virtually totally useless idea anyway.

[kimixa]

I wonder why is swapfile compression not supported?

Doing stuff with swap is a bit more difficult to do things that /may/ take a long time or need memory to be allocated - as it's often called in contexts where you're already out of memory, trying to allocate more would cause more to be attempted to be swapped out, calling back into your swap code causing an infinite loop. I guess they avoided things that may cause new blocks to be allocated on disk - so compression (as the new data may compress less well, requiring more disk space) and snapshotting (allocating new blocks for the copy-on-write behavior of a snapshotted file) being disabled might make that simpler.

[man-walking]

The smartest enhancement for swapfile would be instead independent from BTRFS, that is to write chunks of data on disk compressed with LZ4 or ZStandard

[pal666]

I wonder why is swapfile compression not supported?

because compression is sequential process

[oiaohm]

I wonder why is swapfile compression not supported?

The smartest enhancement for swapfile would be instead independent from BTRFS, that is to write chunks of data on disk compressed with LZ4 or ZStandard

You two of you require the same kind answer. man-walking need to know what feature is called.

You don't compress swapfile/swappartition for Linux because swap data going to disc could already be compressed if a particular feature is enabled.

https://www.kernel.org/doc/Documentation/vm/zswap.txt

If zswap is enabled first when you need to use swap its attempt to be compressed into ram so avoiding the swap device complete if possible. Now if zswap need to send pages it already compressed to swapfile it does not decompress them it sends them across compressed.

So lets say you have zswap enabled and you have compress at file system level as well. You now have a nightmare if lets say both are set to the same compression format(as they can be).

1) zswap send something straight uncompressed to swapfile it did that because it would not compress now at file system level you attempt to compress again so totally wasting time.
2) zswap found it could compress it and need to evict compressed area from memory now high odds it also uncompressable and you are also consuming more cpu time.

The number are horrible attempt to compress will fail over 90% of the time if zswap is enabled using the same compressing format as under file system.

Zswap makes compressing swapfile at file system or swap device at block device quite pointless. zswap will reduce your swap IO way more than compressing the swap data at block/file system level.

Of course someone might implement compressed swapfile on btrfs even if someone does most cases this will be pointless and you would have been better of enabling zswap and not compressing the swapfile. I cannot think of how btrfs offering compressed swap is going to provide any advantage over just enabling zswap. Hey duplicate features exist in the Linux kernel so it could be there one day.

Basically the feature man-walking is talking about is mainline Linux feature zswap.

OTOH not being able to snapshot the swapfile doesn't surprise me. It's a very weird and virtually totally useless idea anyway.

jacob snapshot on swapfile is more useful than compression but you would be talking about like a virtual machine running on host file system. So snapshot when the virtual machine is suspended. Yes is a weird idea but useful in a limited corner case. That limited corner case is more likely to be useful than case of having the file system compress on the swapfile.

Basically I don't really think compression missing of swapfile really need a major mention due to being basically pointless. Snapshot has some justification.

because compression is sequential process

Compression can be done in a block based process this is what Virtual Data Optimizer (VDO) does on block devices from redhat. Zswap does block base compression for swap data. So sequential process is not the problem. Discs you read in blocks and those blocks are read in sequential process.

Doing compression per block does not give you as high as compression as doing compression over complete area but these are the trade offs.

[geearf]

Are you sure zswap sends compressed pages to swap? It doesn't seem like it when reading the kernel doc or the Arch wiki.

[man-walking]

Oiaohm, when zswap stores pages on disk they are first uncompressed (read the docs), anyway what I wanted to point out is that swapfile compression active or not should be managed at file-content level by the linux memory management itself regardless the filesystem hosting the swapfile (whatever would be EXT4, BTRFS or such...), as the system is aware about which bounds/chunk size choose to both optimize compression and seek/read/write performance.
Usually RAM content should be highly compressible as any data is in flat format, on the worst case anyway LZ4 should do the job very well, almost on par as writing raw data on disk.

[starshipeleven]

I wonder why is swapfile compression not supported?

OTOH not being able to snapshot the swapfile doesn't surprise me. It's a very weird and virtually totally useless idea anyway.

As explained in another thread, swapfiles are a trick where the filesystem allocates space which is then passed RAW to the kernel by address, to be used however it sees fit for swapping.

After this happens, the filesystem has no type of control of what happens in this allocated space, it only knows it must be left alone. This is valid for every filesystem.

If you want to compress swap you need to look for ways that compress swapped pages. Afaik nothing does that. I'd guess that it's not worth investing time into, since even allocating insane amounts like 16 or 32GB of swap is a drop in the bucket for modern storage devices.

[pcxmac]

been waiting for swapfile support FOR A LONG TIME !!! OMG THANK YOU.