Richard Hughes Developing New "Passim" Local Caching Server

[phoronix]

Phoronix: Richard Hughes Developing New "Passim" Local Caching Server

Richard Hughes is the Red Hat developer who is most prominently known for leading the Linux Vendor Firmware Service (LVFS) and Fwupd development as well as formerly being behind the ColorHug monitor color calibration hardware effort and PackageKit, among other open-source software. He's recently been developing a new software project called Passim that today he announced to the world...

https://www.phoronix.com/news/Passim-Local-Caching-Server

[tildearrow]

Typo:

For now Passim is quite simple and is serving the caches files

[adriansev]

for quite some years there is cvmfs (https://cvmfs.readthedocs.io/en/stable/) that allows things like https://cernvm.cern.ch/appliance/ which is a ~99% remote os files installation.

[caligula]

Does this also work in IPv6 networks?

[skeevy420]

I'm having a Picard headpalm moment reading the github page.

So, NEVER ADD FILES TO THE CACHE THAT YOU DO NOT WANT TO SHARE. Even the filename may give more clues than you wanted to provide, e.g. sharing madison.json might get you in trouble with a family member. This can be more subtle; if you download a security update for a Lenovo P1 Gen 3 laptop and share it with other laptops on your LAN -- it also tells the attacker your laptop model and also that you're running a system firmware that isn't patched against the latest firmware bug.

My reccomendation here is only to advertise files that are common to all machines. For instance:

• AdBlocker metadata
• Firmware update metadata
• Remote metadata for update frameworks, e.g. apt-get/dnf etc.

About the only thing common in that list is AdBlocker metadata. Firmware updates allow an attacker to figure out a system's hardware and remote metadata for update frameworks will let an attacker know every freaking thing the users have installed so they know what to target. Anything is an attack vector if you're patient enough. (Yeah, I'm paraphrasing the dildo bravery line)

While this seems like a good idea, especially for the needs that RHEL wants to serve, I think future versions should focus on security/privacy and not assuming the local network is safe. I also noticed that the assumption that the local network is safe also assumes that everyone in the organization with network/Passim access has the same security and privacy credentials, basically nothing at all, since nothing would be private.

[Xake]

About the only thing common in that list is AdBlocker metadata. Firmware updates allow an attacker to figure out a system's hardware and remote metadata for update frameworks will let an attacker know every freaking thing the users have installed so they know what to target. Anything is an attack vector if you're patient enough. (Yeah, I'm paraphrasing the dildo bravery line)

Except you apparently do not understand what metadata in this context is, and what he warns against.
Metadata is that information you computer fetches when running "apt-get update", "yum makecache", "dnf makecache", "fwupdmgr update" or on windows clicking "look for updates" in "windows update".
In this context it is *only* information about what exists in an upstream repo in a form of a couple of files containing list of files and hashes.
Those files says *nothing* about what actually runs on the machine it self.

What he warns against is putting anything else but this kind of generic data on there unless you really know what you are doing.


If you need an explanation about metadata in this context and what the point is:
If you for example look at https://dl.fedoraproject.org/pub/fed...ch64/repodata/
There is a file named repomd.xml and it is rather small at just under 5k.
Or look at https://archive.raspbian.org/raspbian/dists/bookworm/ where InRelease is only 15k.
These files themselves contain a filename and its hash telling the packagemanager to also fetch the other files that actually contains information about the packages in the repo themselves. Those other files can range up to some megabytes each.
They are not really that large. But say you have a couple of machines running the same version of the same distro in your home lab. Not enough to try to set up a mirror of your own, but enough that it will become a couple of calls against the upstream repo, and a couple of megabytes fetches for every time your machines checks for updates?
Now say that there was a service you can run on you network that shares a list of filenames and their hashes telling your machine that "if apt-get says you want this file you can fetch it faster from here instead of from internet as I already have it".

Now say you are running a repository or a public mirror. Would you not love if only one client per network fetched all the heavy repository metadata and the rest of the machines did not? That would at least save you a couple of gigabytes in data, and I guess that is why windows already does this.

[amxfonseca]

Does this also work in IPv6 networks?

It should. It says that it relies on mDNS, which uses UDP packets on the port 5353 to communicate. It runs fine on both IPv4/IPv6 networks.

So it is up to the http server to listen on IPv6 and announce the correct endpoint via mDNS.

[timofonic]

Except you apparently do not understand what metadata in this context is, and what he warns against.
Metadata is that information you computer fetches when running "apt-get update", "yum makecache", "dnf makecache", "fwupdmgr update" or on windows clicking "look for updates" in "windows update".
In this context it is *only* information about what exists in an upstream repo in a form of a couple of files containing list of files and hashes.
Those files says *nothing* about what actually runs on the machine it self.

What he warns against is putting anything else but this kind of generic data on there unless you really know what you are doing.


If you need an explanation about metadata in this context and what the point is:
If you for example look at https://dl.fedoraproject.org/pub/fed...ch64/repodata/
There is a file named repomd.xml and it is rather small at just under 5k.
Or look at https://archive.raspbian.org/raspbian/dists/bookworm/ where InRelease is only 15k.
These files themselves contain a filename and its hash telling the packagemanager to also fetch the other files that actually contains information about the packages in the repo themselves. Those other files can range up to some megabytes each.
They are not really that large. But say you have a couple of machines running the same version of the same distro in your home lab. Not enough to try to set up a mirror of your own, but enough that it will become a couple of calls against the upstream repo, and a couple of megabytes fetches for every time your machines checks for updates?
Now say that there was a service you can run on you network that shares a list of filenames and their hashes telling your machine that "if apt-get says you want this file you can fetch it faster from here instead of from internet as I already have it".

Now say you are running a repository or a public mirror. Would you not love if only one client per network fetched all the heavy repository metadata and the rest of the machines did not? That would at least save you a couple of gigabytes in data, and I guess that is why windows already does this.

It's quite limited in scope, it should add security and privacy. A project like this but without those limitations would be quite interesting for too many use cases.

[Vorpal]

Yeah, if this was a bit more secure (e.g encryption and authentication between computers, this would be wonderful as a way to only having to download the actual apt/dnf/pacman/what-have-you updates once and then share it peer-to-peer. Would be lovely as I have 5 or so computers all running Arch.

My experience is that downloading e.g. the Linux kernel is much slower than the index (metadata), making it not worth the effort for most end users.

If it worked similar to syncthing, but was on demand rather than sync everything eagerly that would be perfect for this usage (as some machines have slightly different sets of packages, I don't want to use syncthing itself directly on the package cache directories).

[mb_q]

There is like, BitTorrent for that, right? This is how huge monolithic server farms powering Facebook and Twitter pull updates from each other, at least due to some 10yo+ blog posts I vaguely remember. IPFS also has less-hassle clones...