Debcow Optimizing Debian Packages For Copy-On-Write File-Systems

[phoronix]

Phoronix: Debcow Optimizing Debian Packages For Copy-On-Write File-Systems

Debcow is an experimental implementation of deploying Debian packages on copy-on-write file-systems like Btrfs and Bcachefs. Debcow adapts DPKG to use reflinks for installing packages. With reflinking to the file contents from the Debian package archives rather than copying of files, it can lead to a dramatic speed-up of installing Debian packages: as much as 6x faster on CoW file-systems...

https://www.phoronix.com/news/Debcow-Debian-DEBs-CoW-FS

[Chugworth]

I've known for a while now that the way most updates are handled doesn't take advantage of CoW, and in fact with snapshots it can cost you more disk space. When you replace a file (as is the common practice with updates), you end up with a completely separate file from what's in the snapshot. The better way to update a file is to do the equivalent of rsync's "--inplace" switch which only replaces the changed data in the existing file. That would save storage space because the snapshots wouldn't be holding entirely separate copies of the same file.

From the sound of it, this is not really addressing that problem though. It sounds like they are just taking advantage of reflinks to avoid copying all of the data.

[down1]

The snap installer should do this for the userfile migrations too.

[V1tol]

I would say they need to change .deb package format. Anyone who opened a .deb file finds inside at least two more archives (control.tar.gz and data.tar.gz) which should be unpacked too. I noticed that behavior when did some PKGBUILDS and wondered why package build folder consumes more space than expected. That's one of the reasons why I prefer to use RPMs, AppImages or plain tar.gzs if available to pack software I need and use DEBs for that only if nothing else is available.

[shmerl]

Did Debian repos enable zstd in packages already? That should speed up installation times too.

[microchip8]

Will it work on XFS too? It supports reflinks.

EDIT: yeah, it will. Missed XFS while reading

[FluxTape]

I don't really see the point of this, file copy times have never been an issue for me on btrfs. But maybe it makes a difference for people that use an HDD as their boot drive.

[jstoik]

Anyone who opened a .deb file finds inside at least two more archives (control.tar.gz and data.tar.gz) which should be unpacked too.

The .deb itself is just an AR archive so doesn't add any compression itself. The control, data, etc. sub-archives are compressed separately to allow them to be unpacked separately. (Except for installation, most of the time it's just the main CONTROL file being parsed/referenced.)

Did Debian repos enable zstd in packages already? That should speed up installation times too.

Yeah, Ubuntu uses zstd by default, and it's supported in the current Debian stable too.

[shmerl]

Yeah, Ubuntu uses zstd by default, and it's supported in the current Debian stable too.

Supported or used in Debian repos? Those are two different things. I think Debian was still stuck with not using it even if it's supported in theory.

[jstoik]

Supported or used in Debian repos? Those are two different things. I think Debian was still stuck with not using it even if it's supported in theory.

Bookworm still uses tar.xz by default, but has no trouble parsing/installing archives from my own repo that uses tar.zst (ultra 22) instead.

Ubuntu switched to tar.zst (with lower compression) back in Jammy, I believe? They've been using it for a while now.