FreeDesktop.org GitLab Service Restored

[phoronix]

Phoronix: FreeDesktop.org GitLab Service Restored

After battling Ceph storage issues and related problems for the past day after two solid-state drives failed, the FreeDesktop.org GitLab that is used for the centralized, coordination of the open-source Linux graphics driver development and other open-source software is back online...

https://www.phoronix.com/scan.php?page=news_item&px=FreeDesktop-GitLab-Back-Online

[birdie]

No matter the story, backups, backups, backups.

And not just backups, verify they actually match what you've got, verify you can unpack and use them properly. A few days ago I accidentally zero'ed all the files in my home directory, almost fainted. Luckily I had a backup, though not very fresh but still better than nothing.

[CochainComplex]

..A few days ago I accidentally zero'ed all the files in my home directory, almost fainted. Luckily I had a backup, though not very fresh but still better than nothing.

Happend to me as well month ago.

[zexelon]

No matter the story, backups, backups, backups.

And not just backups, verify they actually match what you've got, verify you can unpack and use them properly. A few days ago I accidentally zero'ed all the files in my home directory, almost fainted. Luckily I had a backup, though not very fresh but still better than nothing.

Random question, how? I dont want to make the same mistake! Personally every once in a while I mount my backups (done with Restic) to test that I can recover files.

[KrissN]

I've been running a small ceph cluster in the past and apparently multi-disk failures are not uncommon for these systems. Ceph tends to hammer storage pretty badly and care has to be taken to use server-grade equipment and - what is more important - use a mix of different vendors, as in such case you will in most cases experience only a single disk failure, which is easy to deal with.

Ours was a budget one using consumer HDDs and most of them were failing after around 6 months of use.

[CommunityMember]

Ours was a budget one using consumer HDDs and most of them were failing after around 6 months of use.

So, you got what you paid for?

Seriously, Enterprise HDDs and SSDs do cost quite a bit more, but their design point is so much worth it at scale if only to reduce the operational expense of maintaining the service.

[birdie]

Random question, how? I dont want to make the same mistake! Personally every once in a while I mount my backups (done with Restic) to test that I can recover files.

Playing with find, xargs and redirections. Thank God I only used my home directory for that and not the root directory. Only when I rebooted I realized all the files were lost. Sleep well and don't rile up. I've been in a state of major depression recently exacerbated by horrible loneliness and feats of anger (towards myself) - that can really make you type the things you don't actually mean.

[zexelon]

Playing with find, xargs and redirections. Thank God I only used my home directory for that and not the root directory. Only when I rebooted I realized all the files were lost. Sleep well and don't rile up. I've been in a state of major depression recently exacerbated by horrible loneliness and feats of anger (towards myself) - that can really make you type the things you don't actually mean.

Thanks! It was a legit question... data is getting mor important than ever and woumd rather learn, than make my own mistakes!

[birdie]

Thanks! It was a legit question... data is getting mor important than ever and woumd rather learn, than make my own mistakes!

I hope you made those mistakes intentionally

[zexelon]

I hope you made those mistakes intentionally

Nope! Thats what typing on a phone while trying to accomplish a grueling psych exercise gets you :P

I hate touch screens... and phone typing... should really stop doing that!