X.Org/FreeDesktop.org Is Looking For Sponsors Or May Have To Cut Continuous Integration Hosting

[phoronix]

Phoronix: X.Org/FreeDesktop.org Is Looking For Sponsors Or May Have To Cut Continuous Integration Hosting

The cost of cloud hosting -- or in particular hosting their own GitLab instance and running continuous integration (CI) support for FreeDesktop.org projects -- is putting financial strain on the X.Org Foundation...

http://www.phoronix.com/scan.php?page=news_item&px=X.Org-Cloud-CI-Cost-Sponsors

[peterdk]

Isn't gitlab.com free for opensource projects? Including CI? Why would you want to host that yourself as a opensource project?

Just checked, yes it's fully free for opensource project:

For organizations interested in free Gold features for groups, we also offer free Gold and Ultimate to educational institutions and open source projects.

[mroche]

peterdk I don't think this is an issue of cost of GitLab, it's the cost of hosting and managing the instance/storage/runners on GCP. Cloud costs can be killer. They're using the Community Edition, which is already free. https://gitlab.freedesktop.org/help

Some reasons for self-hosting this are you aren't affected by downtime or service mistakes on GitLab's behalf, you don't compete with other projects for runners (though you can still attach your own on the .com instance), and it can be faster (in my experience) in terms of interactions (depending on what you host it on). X.Org/FreeDesktop clearly put value in holding their code and services on self-managed infrastructure, if they didn't it probably would be GitHub/GitLab.com.

Cheers,
Mike

[paupav]

I guess people prefer donating to end products such as elementaryos, ubuntu huh and they are not sponsoring things they depend on?
Or is the problem with money management in Xorg?

[lumks]

Maybe it would be an Idea to have one gitlab or whatever with CI for all the SPI members.

[PlanetVaster]

As peterdk said, using GitLab's hosted service (gitlab.com) is free for open source projects, including CI. There's no reason to spend 90K a year on hosting it themselves.

[Ironmask]

IBM/Red Hat basically own Xorg now, they're the only ones maintaining it. Why don't they just make it official and host it on their own equipment?

Corporate culture is so strange sometimes. I feel like a small child trying to understand their parent's divorce.

[mroche]

So the issue is primarily about bandwidth that CI exacerbates when it comes to costs. Matt Turner made the following reply two messages later[0]. There's also a page documenting the f.do infrastructure[1].

Matt Turner:

Some clarification I got from Daniel in a private conversation, since I was confused about what the money was paying for exactly:

We're paying 75K USD for the bandwidth to transfer data from the GitLab cloud instance. i.e., for viewing the https site, for cloning/updating git repos, and for downloading CI artifacts/images to the testing machines (AFAIU).

I was not aware that we were being charged for anything wrt GitLab hosting yet (and neither was anyone on my team at Intel that I've asked). This... kind of needs to be communicated.

A consistent concern put forth when we were discussing switching to GitLab and building CI was... how do we pay for it. It felt like that concern was always handwaved away. I heard many times that if we needed more runners that we could just ask Google to spin up a few more. If we needed testing machines they'd be donated. No one mentioned that all the while we were paying for bandwidth... Perhaps people building the CI would make different decisions about its structure if they knew it was going to wipe out the bank account.

What percentage of the bandwidth is consumed by transferring CI images, etc? Wouldn't 75K USD would be enough to buy all the testing machines we need and host them within Google or wherever so we don't need to pay for huge amounts of bandwidth?

I understand that self-hosting was attractive so that we didn't find ourselves on the SourceForge-equivalent hosting platform of 2022, but is that risk real enough to justify spending 75K+ per year? If we were hosted on gitlab.com or github.com, we wouldn't be paying for transferring CI images to CI test machines, etc, would we?

So what do we do now? Have we painted ourselves into a corner?

Cheers, Mike

[0] https://lists.freedesktop.org/archiv...ry/041234.html
[1] https://www.freedesktop.org/wiki/Infrastructure/

[Teggs]

I expect the general population to screw up when it comes to understanding the costs of 'Putting It In The Cloud!' vs. just owning your own equipment. I didn't expect it here, especially as multiple people must have known about this.

I hope the responsible parties have the integrity to own their work. And I hope they aren't employed by Google...

[grigi]

GCloud/AWS/Azure are all scams, unfortunately. They promise you the world, give you estimated bills, then you realise they lied with a straight face, and hide behind enough technicalities and small print that it's still cheaper to just be extorted.

I have seen this happening over, and over, and over. My current company pays gcloud only about 30x what the estimated costs were. That's still 20x what we paid before at Hetzner.

So, please, don't ever trust these large cloud providers' promises.