Feature debate IPFS for benchmark downloads?

[RobinIT]

Hey Michael,

Just want to mention, more modern tests I typically don't have any problems with and I find PTS very helpful. Thank you for putting it out there.
I'm only sharing the suggestion since it seems IPFS is made to tackle exactly the kind of problems I did run into.

Not familiar with IPFS but anyhow:

- When broken download links are reported, they are corrected.

Haven't looked into reporting them, as I wouldn't know where to begin searching for mirrors for some of the relics I tried to run.
Content addressing could make this easier to maintain though. The hash is immutable, so as long as anyone you have a network path to still has the data, you can get a copy.

- The LAN 'feature' for test distribution is not relevant since PTS/Phoromatic already address this for being able to have all files cached locally and can setup your own download cache or Phoromatic Server.

Hm, I disagree with it being irrelevant on two points there.

One is that IPFS would require very little configuration to function as local caches. With the default config, if you have the data and the daemon is running, you're sharing. So that also means between testing machines. Perhaps not so interesting for people with caches already set up, but for others you might get optimized to LAN speeds by accident.

The other part is that LAN is just an example. The optimizations work on every level. For example if you're one of the lucky ones to be hampered by the Great Firewall of China and the bandwidth limitations / VPN requirements that come with it, you can get more efficient downloads staying within the national network.

- Didn't look if IPFS runs on separate port from HTTP, etc, but given the number of enterprise users behind tight firewalls, etc, this would likely just cause further problems for them.

The defaults are TCP 4001, 5001 and 8080 for peer-to-peer traffic, daemon control API and the HTTP gateway resp.
Indeed to connect outside restrictive networks your best bet is to fall back on a public HTTP gateway.
The only advantages for them would be higher availability (as the gateway will have the extra availability of the IPFS network to back it, unlike a mirror) and the immutable content addressing.

- Relying upon independent HTTP mirrors for downloads is used as-is now because I don't have enough server resources / bandwidth to host all of the download files to the public myself.

Understandably so. Your hosting requirements wouldn't increase, it's exactly the point to have PTS users help out in the peer-to-peer network.
The hosting requirements of the mirrors could decrease this way as well. IPFS would compliment the existing mirrors.

The requirements would fall onto the PTS software, bundling with the IPFS software, adding IPFS addresses to the list of mirrors, etc.
In it's simplest form Somebody(tm) provides the seeding, adds the link to the test via http://localhost:8080 to use a local IPFS daemon and https://gateway.ipfs.io to have the gateway as a fallback.
An advanced setup would be to use the mirrors as source, have PTS clients publish the hashes automatically, etc. Though that may be better suited for the Phoromatic servers?

[Michael]

Not familiar with IPFS but anyhow:

- When broken download links are reported, they are corrected.

- The LAN 'feature' for test distribution is not relevant since PTS/Phoromatic already address this for being able to have all files cached locally and can setup your own download cache or Phoromatic Server.

- Didn't look if IPFS runs on separate port from HTTP, etc, but given the number of enterprise users behind tight firewalls, etc, this would likely just cause further problems for them.

- Relying upon independent HTTP mirrors for downloads is used as-is now because I don't have enough server resources / bandwidth to host all of the download files to the public myself.

[RobinIT]

Short intro. When I used PTS for the first time, I noticed a lot of download mirrors are long since gone.
(I would have liked legacy Enemy Territory benchmarks, played that game to death.)
And for those that are present downloads can be slow, large or both.

So I thought, wouldn't it be better to use IPFS as the primary distribution method?

https://ipfs.io/

IPFS is pretty cool and I'd suggest hitting up their site to get the full details, but short summary why it'd apply here:
- Uses a peer to peer network for data distribution (think a giant bittorent swarm).
- Uses content addressing (use cryptographic hashes of the file contents as it's address).
- The cross-platform daemon provides HTTP gateways and mounting for easy integration.

The torrent-like network advantages are obvious for PTS. If you're a heavy user of the suite, you can get your downloads over LAN from the test bench next door, rather than going out to a slow mirror on the other side of the planet. And bandwidth requirements for the mirrors will be lessened when PTS users share the data amongst themselves.

Additionally, with the content addressing you don't have to maintain these mirror URLs. It's only a matter of having "seeders" and the download will be very reproducible compared to relying on HTTP mirrors. Also, this may increase availability when a test is really old and mirrors stop offering the files.

---

The downside would of course be, you'll use system and network resources to seed for the network when you're done getting your files.
Some users may not like this for privacy or bandwidth reasons. And for some tests it may impact results to have that extra process running.

One way to relieve those issues is to use a public HTTP gateway, for example https://gateway.ipfs.io/ or one could run on openbenchmarking.org.
That would pretty much defeat the purpose of the peer to peer network though and be no different from being another mirror.

--

So what do you reckon, should this be implemented?
Would you be OK with running an IPFS daemon to share the benchmark data?
Do you think it should be the default or opt-in?
Should public gateways be added as mirrors?