X.Org/FreeDesktop.org Is Looking For Sponsors Or May Have To Cut Continuous Integration Hosting

[Anty]

GCloud/AWS/Azure are all scams, unfortunately. They promise you the world, give you estimated bills, then you realise they lied with a straight face, and hide behind enough technicalities and small print that it's still cheaper to just be extorted.

Dude - don't spread FUD.

First you can use cost calculator like this one in AWS https://calculator.s3.amazonaws.com/index.html or GCP https://cloud.google.com/products/calculator.

It is well known that you pay for egress in cloud. It happens for both AWS and GCP cost is similar - $89 and $85 per TB/month.

So in case of GCP $75000/year gives 882 TB/year -> 73 GB/month -> 2.4 TB/day -> 10 GB/hour

And out of $75000 only $30000 is egress cost (remaining is running other stuff).
So final number is just 4 GB/hour.

Seems high? 4 GB/hour is just 1.1 MB/s which is not impressive at all...

[starshipeleven]

Seems like capitalism to me.

Contrary to popular opinion, capitalism does not imply scamming customers.
Scamming people out of their money is a completely parallel thing to the economic structure and is prevalent in communism and socialism too.

[rastersoft]

It would be great a distributed system for CI, in the way of the old SETI@home...

[TemplarGR]

GCloud/AWS/Azure are all scams, unfortunately. They promise you the world, give you estimated bills, then you realise they lied with a straight face, and hide behind enough technicalities and small print that it's still cheaper to just be extorted.

I have seen this happening over, and over, and over. My current company pays gcloud only about 30x what the estimated costs were. That's still 20x what we paid before at Hetzner.

So, please, don't ever trust these large cloud providers' promises.

Seems like capitalism to me.

End users don't understand the concept of "upstream" of many applications or libs or infrastructure.

That's why financially supported end products should also financially support at least their core dependencies.

[starshipeleven]

Oh, wow. I guess some cost for bandwidth is to be expected, but 75k seems excessive. That's $200 per day.

They are talking of CI (Continuous integration), that means the cost is running the servers that compile and run the tests for all applications they serve.

[starshipeleven]

I guess people prefer donating to end products such as elementaryos, ubuntu huh and they are not sponsoring things they depend on?

End users don't understand the concept of "upstream" of many applications or libs or infrastructure.

[aaahaaap]

Seems like whoever set this up doesn't really know what they are doing, from a quick look at the issues simple things like making all traffic doesn't go via the outside (which means egress costs) isn't even done https://gitlab.freedesktop.org/freed...top/issues/206

And it would be useful to see a cost breakdown, it's very unclear what exactly is causing these high costs.

peterdk I don't think this is an issue of cost of GitLab, it's the cost of hosting and managing the instance/storage/runners on GCP. Cloud costs can be killer. They're using the Community Edition, which is already free. https://gitlab.freedesktop.org/help

Some reasons for self-hosting this are you aren't affected by downtime or service mistakes on GitLab's behalf, you don't compete with other projects for runners (though you can still attach your own on the .com instance), and it can be faster (in my experience) in terms of interactions (depending on what you host it on). X.Org/FreeDesktop clearly put value in holding their code and services on self-managed infrastructure, if they didn't it probably would be GitHub/GitLab.com.

Cheers,
Mike

Same issues caused on GitLab's behalf can happen also in self-hosting environment,.. Plus, you need skilled people to maintain it correctly. GitLab (and other git hostings) already provide that. And, for open-source projects for free.

When using official/centralized git hostings, then you can also gain more contributors, as not everybody is interested to setup more accounts, because projects want to be self-hosted. Even google hosts golang on github: https://github.com/golang/

Competing for runners,... That's just bad terminology there,... It's about utilization of resources, and I don't see anything wrong in waiting a minute or few to make CI build happen. Anyway, CI builds are only post-checks, which usually should be green, as developers should be cautious with their changes, because probably not everything is covered by automated tests.

And, about holding code in own hands,.. and having safe copy,.. well, then just setup self-hosted server to mirror git repositories including copies of old branches when force-pushed. Anyway,.. everything self-hosted can be cracked (sooner or later), therefore any real backups are local, and geographically spread, on machines, which aren't directly reachable from internet (behind NAT and firewall). And, the best ones are stored on external drives,... So, just setup mirroring in home of each developer, and you have perfect backups.

Honestly,.. I don't see any real gain in not using official GitLab hosting, or GitHub,... cost for computing and human resources is higher, than is the price for these services,...

[Isedonde]

Oh, wow. I guess some cost for bandwidth is to be expected, but 75k seems excessive. That's $200 per day.

I guess it could be a lot cheaper if "internal" traffic between the main gitlab instance and runners was actually internal instead of going through the internet. Then runner bandwidth cost could be reduced to a bunch of ethernet cables.

I mean, yes, the service should be somewhat reliable. But maybe it doesn't need the reliability guarantees offered by cloud services. Instead, couldn't they buy a bunch of servers of their own and pay a comparably small amount for colocation? Or maybe companies would be willing to offer colocation for free? Then CI wouldn't need any internet bandwidth, so they'd only have to pay for bandwidth that is needed when developers pull/push their changes, or people interact with the gitlab instance in other ways. Hardware failure could cause some outages, unless they buy more servers and implement some kind of fail over strategy. But this is not facebook or google where 100% availability is required. If developers can't push their changes today, they start working on something else and push tomorrow.