CoreOS Moves From Btrfs To EXT4 + OverlayFS

[Micket]

What do you expect it to do? When your drive is full, the only options are to add more space or delete stuff. Obviously it can't just automatically delete your data... So if you fill the drive, you will need to either add more space or manually delete stuff.

Many of the complaints about btrfs come down to users not understanding that snapshots take up space, particularly snapshots of data that often change. Confusion with df doesn't help. But being able to immediately roll back the root filesystem after some failed upgrade is a feature that most admins would appreciate.

Actually, you don't seem to know what happens with BTRFS. "Deleting stuff" doesn't free up space. That's why I said "not beeing able to restore the space".
To actually free up space, you need to do a long, slow, rebalancing of metadata. To rebalance metadata you need free space, which you don't have. You're fucked.
It seems to me that btrfs should have reserved space for this critical operation.

Instead, I had to get some external storage *hardware*, format that device as btrfs, add it to the btrfs pool, rebalance the metadata for like an hour, then remove it from the pool. There, finally a working drive again. 3-4 hours of work and and a few dozens terminal commands.
Had I not been a linux user for +10 years I very much doubt I would have ever solved the problem, short of a format+reinstall.

OpenSUSE Factory was wrong to make all these fricking snapshots I never asked for, but btrfs allows itself to be deadlocked to easily.

[Nobu]

Actually, you don't seem to know what happens with BTRFS. "Deleting stuff" doesn't free up space. That's why I said "not beeing able to restore the space".
To actually free up space, you need to do a long, slow, rebalancing of metadata. To rebalance metadata you need free space, which you don't have. You're fucked.
It seems to me that btrfs should have reserved space for this critical operation.

From the btrfs wiki:

A btrfs balance operation rewrites things at the level of chunks. It runs through all chunks on the filesystem, and writes them out again, discarding and freeing up the original chunks when the new copies have been written. This has the effect that if data is missing a replication copy (e.g. from a missing/dead disk), new replicas are created. The balance operation also has the effect of re-running the allocation process for all of the data on the filesystem, thus more effectively "balancing" the data allocation over the underlying disk storage.

So, a balance operation must be able to write data back to the disk. Also, btrfs is a COW filesystem, meaning the origional data cannot be destroyed until the copy is done being written. That means you need enough space for two chunks of data, plus two more if your disk is getting full (the original + copy, and the data you're overwriting + it's copy). Let's see, how big is a chunk? "As the filesystem needs storage to hold file data, or filesystem metadata, it allocates chunks of this raw storage, typically in 1GiB lumps, for use by the higher levels of the filesystem," so we'd need 2GiB minimum, up to 4GiB, of extra space in order to perform a balance operation. Doesn't seem like an efficient way of clearing space (not to mention all of the load it puts on the disks).

Now, snapshot subvolumes are (after changing information on them) basically a delta of the previous snapshot. When you create a snapshot, it takes up virtually no space on the filesystem. When you make changes to that snapshot, it starts to increase in size, up until you create another snapshot and switch to it. If you want to free up space, the logical thing to do would be to remove unnecessary snapshots, since that could free up GiB's of space (especially when the system is taking a snapshot after every package transaction). Now, iirc, it doesn't immediately free up space, as it has to incrementally merge the deltas back, but it should eventually result in less used space. Doing a balance will effectively write-out any changes pending, because what better time to do it than when everything's being re-written anyway, but it shouldn't be the primary means.

If you want to be sure that all the subvolume's information has been removed from the filesystem, you can use the 'btrfs subvolume sync' command (btrfs-subvolume man page: "Wait until given subvolume(s) are completely removed from the filesystem after deletion. If no subvolume id is given, wait until all ongoing deletion requests are complete. This may take long if new deleted subvolumes appear during the sleep interval."). If you want to force btrfs to write changes to disk, you can use the 'btrfs filesystem sync' command (similar to the 'sync' command; btrfs-filesystem man page: "Force a sync for the filesystem identified by <path>.").

[blackiwid]

Many of the complaints about btrfs come down to users not understanding that snapshots take up space, particularly snapshots of data that often change. Confusion with df doesn't help. But being able to immediately roll back the root filesystem after some failed upgrade is a feature that most admins would appreciate.

Again its not only the users fault, if the normal df would give right results and the distro-specific automatic snapshot on installing stuff tools would not make totaly retarded each time also a snapshot of the home volume and there is not even a option to disable that. Way less users would run into that problem.

[lovenemesis]

My OpenSUSE install picked btrfs by default so I thought I'd go with it.
A few weeks later I'm stuck trying to remove data from the drive since it filled up in no-time with tons and tons of fucking snapshots.
Had to get a hold of a usb stick and format that to btrfs and add it so that I could REMOVE DATA (since removing files doesn't actually free up space, you need to rebalance the metadata manually )

Why would a rolling distro use this by default, when users have to manually watch out for their drive filling up and manually run rebalancing and deletion of snapshots every now and then? All through a few dozen terminal commands.

If you are administrating larger systems I'm sure btrfs could be really useful, but as the default for a desktop? No fucking way.

Maybe you need to pay a bit more attention to your distro news?

The snapshot tool is the ultimate replenishing mechanism for Linux. The tool is enabled by default and snapshots are automatically created in regular intervals as well as before and after each software installation.

From openSUSE news

Anyway, Fedora has a yum plugins doing similar things but it was not enabled by default.
Maybe openSUSE developers need to communicate the default choice better to its end users.

[edoantonioco]

Phoronix has proved than EXT4 is the best filesystem than exist at least for HDD. I mean, in some specific test XFS and other filesystems win over this but in general ext4 is still the best.

[Kano]

I use btrfs with lzo compression for some months now, in the beginning i had some unmountable partitions after a while, but btrfs restore was often still possible. I did not get why check was impossible however. Kanotix provides a simple way to create new partitions inside the installer with lzo but oft course you can convert it in the fly with a changed fstab entry and use the filesystem defragment tool with -c option. You can expect about 10% save with Steam games (random selection). My SSD is always full and that helps a little bit.

[nanonyme]

Maybe you need to pay a bit more attention to your distro news?



From openSUSE news

Anyway, Fedora has a yum plugins doing similar things but it was not enabled by default.
Maybe openSUSE developers need to communicate the default choice better to its end users.

1) The snapshotting isn't enabled by default in Fedora
2) In Fedora you can tell it which subvolumes to snapshot

[nanonyme]

It seems that all new open source tech is very unstable. Maybe they shouldn't allow other projects to use it until it's done. They need to find a way to enforce that even though you have access to the source, you can't actually use it in a distro or something. I'm not sure that open source is a good development model because of this. Everything is alpha quality.

Btrfs most definitely is good beta quality. All new tech whether open or not is unstable. I'd keep in mind not many distros made btrfs default, only selectable. If you use a distro making bad judgement calls, who do you think is to blame?

[mitcoes]

Why not give XFS a try?

I use ext4, btrfs and xfs partitions.

I am using xfs for the partitions with more Input/output and it has been more stable than ext4 or btrfs when heavy taks plus hot weather comes.

My high I/O partitions usually fail at summer btrfs performed and repaired better than ext4, but xfs did not fail (finger crossed for next year)

[pal666]

I'm very happy it's been working for you, but a quick glance at the mailing list should (rightly) scare you about some of the potential bugs that are still lurking in btrfs. (It's a lot better than it was, but there are still regular bug reports.)

Hopefully Josef's "introduce random failures" testing code will enable more rapid stabilization.

Don't get me wrong, I really want btrfs's features (esp. pervasive checksumming), but right now it doesn't look stable enough.

if it is stable enough for facebook, then it is stable enough for you. there are millions of btrfs filesystems outthere and regular bug reports just mean than your harddrive have bigger probability to explode than btrfs. use backups.