Fedora Considers Dropping Delta RPMs

[darkdragon-001]

Reproducible builds are essential for delta rpms to work well. Bazel requires this to work, so I can imagine this is one reason why it works well for Chromium/Chrome.

[stormcrow]

It's not the choice of Fedora package maintainers how the systems push updates. Maintainers are still building updates on top of the previous, but the systems (for various reasons) do not keep the old updates around on the mirrors. So at any given point, you have GA and latest updates, but nothing in between.

Except that's quite literally not what Neal said. He said they explicitly don't build the update packages on top of previous packages. They're built from upstream releases, not delta/diffed from GA release, I assume even when upstream diffs are available. We just don't know how far back in history the method Neal's describing goes. Perhaps at one time they kept things closer to the delta RPM build procedure, but now they don't. It's probably a mistake to believe every maintainer does it the same way. Perhaps some still make proper delta RPMs, but the general feel from the discussion is that deltas haven't even been consistently built for a long time in Fedora.

[King InuYasha]

Except that's quite literally not what Neal said. He said they explicitly don't build the update packages on top of previous packages. They're built from upstream releases, not delta/diffed from GA release, I assume even when upstream diffs are available. We just don't know how far back in history the method Neal's describing goes. Perhaps at one time they kept things closer to the delta RPM build procedure, but now they don't. It's probably a mistake to believe every maintainer does it the same way. Perhaps some still make proper delta RPMs, but the general feel from the discussion is that deltas haven't even been consistently built for a long time in Fedora.

Update packages are built on top of previous packages in the Koji build system, but the updates system doesn't work that way. Update repos are composed on top of GA repos only.

You can see this in the Koji build system by looking at the build logs of any package. The flatpak package is a good example.

[cen1]

I experimented with binary delta updates on a small desktop application I wrote and I agree it is not worth it. With today's CDNs and internet speeds it is no longer a problem. If your builds are not reproducible, the deltas are not that great either. And the biggest problem of all is that you now need to create a matrix of deltas because folks can update from different versions of a package to latest. Unless your system requires incremental package updates, which dnf does not(?), this becomes a complete nightmare.

[piotrj3]

Google chrome & derivates should be the ones vocal about binary delta updates ... just imagine how much CO2 would be saved if the whole world would not need to download 100mb+ every other day when chrome decides it's outdated ...

https://www.chromium.org/developers/design-documents/software-updates-courgette/

They are. Actually out of curiosity i was looking who else was using courgette and found Nvidia uses it as well.

[Britoid]

Perhaps using some tools like rsync/zsync will be a more robust solution to this?

Fedora already has ostree which provides delta updates for free, although its only used in Silverblue, Core and Flatpak for now.

[elatllat]

...zsync...

Is used by Ubuntu for some things;

https://help.ubuntu.com/community/ZsyncCdImage

I wonder if zsync could used with metalinks like OpenSuse uses;

https://en.opensuse.org/SDB:Download_help#Downloading_via_Metalinks

[Weasel]

So they messed up. Makes more sense than "it's not needed now when the Internet is faster", because I have yet to find a person that would download 600MB of updates, instead of 60MB. As a rolling-release user, I see that big a download quite often.

Imagine Michael ever admitting Fedora or GNOME mess up. Always someone else's fault.

[Weasel]

The issue from point of view of maintener of packages is that to be efficient you would have to generate more then 1 diff. What if someone upgrades not from version 0.9 to 0.10 but from version 0.5 to 0.10? At that point downloading 5 diffs might be more costly then direct entire redownload. Balancing how many diffs you generate is quite hard task.

That's not hard at all, you literally just look at all diffs and count the size you'd have to download. If it's larger than direct entire redownload, go for direct redownload of latest version. Steam also does it.

[mercster]

I am not familiar with the deeper technological issues here, but I think it's important not to assume everyone has very fast Internet. A few months ago, due to financial difficulties, the only Internet I could afford was a shoddy 3M/sec ADSL connection. It's very frustrating when most sites/distribution systems assume everyone has good Internet. Linux is also very useful in lower income places where Internet may not be as good.

I recognize that there were other technical considerations, and I can't speak to that. If the system wasn't working the way it was supposed to, that's a consideration. I do think we ought not to assume that everyone has good Internet, because not everyone does.