CoreOS Moves From Btrfs To EXT4 + OverlayFS

[olifre]

I haven't had time to figure out how to make it stop taking snapshots all the fucking time, so I just delete them for now.

At least for stable OpenSUSE I know they do this when you update. If they also do this for factory, I'd consider this a bug - doing snapshots every half a day sounds like a very bad idea... As it happened on two machines at the same time for you, and they probably got similar updated and had similar large root-fs, sounds like that's it.

Just the fact that I could fill up my drive and not being able to restore the space without adding another device to the pool is a pretty crucial flaw.

Totally agree. That's a known btrfs issue and I think it has high priority upstream. btrfs wiki is alerting users about this problem since years, no idea why distributions who choose btrfs by default do not alert users or provide pre-warning mechanics. At least since 3.18, at least the metadata is auto-rebalancing itself.

[Leinad]

XFS

Why not XFS? Maybe shrinking support is crucial. It's XFS's achilles heel.

Because XFS doesn't have any snapshot capabilities which is needed for CoreOS.

[brosis]

EncFS work as a layer over existing filesystem, adding encryption and checksumming. It can't correct bad sums though AFAIK, only report them on access.

[Ericg]

My OpenSUSE install picked btrfs by default so I thought I'd go with it.
A few weeks later I'm stuck trying to remove data from the drive since it filled up in no-time with tons and tons of fucking snapshots.
Had to get a hold of a usb stick and format that to btrfs and add it so that I could REMOVE DATA (since removing files doesn't actually free up space, you need to rebalance the metadata manually )

Why would a rolling distro use this by default, when users have to manually watch out for their drive filling up and manually run rebalancing and deletion of snapshots every now and then? All through a few dozen terminal commands.

If you are administrating larger systems I'm sure btrfs could be really useful, but as the default for a desktop? No fucking way.

Just a guess, but OpenSUSE may have zypper create a btrfs snapshot before every transaction so if you do a lot of package install/remove/update youre gonna flood your system with snapshots.. I'm on Fedora and Arch and they DONT create snapshots for each transaction and not getting any of that. So it may not be a matter of a "rolling distro" doing it, it may just be OpenSUSE doing it.

[Nobu]

Even then it shouldn't be that bad, because /usr is static for the most part and /etc shouldn't change too much either. Is it snapshotting /home on every package install/upgrade/removal? That sounds like bad design/bug. /home (and similar) should be snapshotted on a time-basis, unless you have some specific need to snapshot it on some other basis (for example, before rolling out a distro upgrade, or server software upgrade that affects a specific user), since the data in there will be changing more frequently.

[xeekei]

Because XFS doesn't have any snapshot capabilities which is needed for CoreOS.

Neither does Ext4 last time I checked. But maybe that's what this OverlayFS is supposed to do, and maybe it only works with Ext4. I am unsure what it is.

[darkbasic]

Neither does Ext4 last time I checked. But maybe that's what this OverlayFS is supposed to do, and maybe it only works with Ext4. I am unsure what it is.

I use LVM2 for snapshots...

[xeekei]

I use LVM2 for snapshots...

Yes, but XFS works with LVM2 too. I guess I just failed to see that person's point.

[blackiwid]

Even then it shouldn't be that bad, because /usr is static for the most part and /etc shouldn't change too much either. Is it snapshotting /home on every package install/upgrade/removal? That sounds like bad design/bug. /home (and similar) should be snapshotted on a time-basis, unless you have some specific need to snapshot it on some other basis (for example, before rolling out a distro upgrade, or server software upgrade that affects a specific user), since the data in there will be changing more frequently.

run into similar problems because I installed in fedora something similiar to zipper or whats the name of this snapshot thing... and yes it also did make a backup of home on every installation of a package that was most likely what filled up the disk so fast, but that is no bug in btrfs. What could be done to help this problem on the btrfs site or df site, get df ready to give real numbers. So at least you find out fast what the problem is. new users asume that df dont lie to you even a DONT-KNOW from df would be better than this false numbers.

BUT I think you overdue it here a bit, in general deletion of files at least with one disk gives free the space. only with snapshots activated in some case maybe not.

But you said something about rebalancing so I guess you use some kind of raid mechanism. Its stable in the way that if you do not horrible commands it will most likely not eat any of your data, thats the most important thing right? sounds like they adress many issues right now so 3.19 should be interesting.

But yes I think I will not use btrfs on ohter installations for 1-2 years from now on. If all would work perfect it would be funny that still dont know 90% of the people still use ext3/4 right?

Rebalancing is doable just put it in your crontab or is there a problem with that?

[chrisb]

Just the fact that I could fill up my drive and not being able to restore the space without adding another device to the pool is a pretty crucial flaw.

What do you expect it to do? When your drive is full, the only options are to add more space or delete stuff. Obviously it can't just automatically delete your data... So if you fill the drive, you will need to either add more space or manually delete stuff.

Many of the complaints about btrfs come down to users not understanding that snapshots take up space, particularly snapshots of data that often change. Confusion with df doesn't help. But being able to immediately roll back the root filesystem after some failed upgrade is a feature that most admins would appreciate.