Fedora Btrfs Activity Continues - New Options To Control Discard, Compression

[RahulSundaram]

Yeah, my point was just that knowing that they are using btrfs for something that is slightly more important than a swarm of webservers behind a load balancer would actually mean something.

Where are you seeing that they are only using it for webservers?

They do use it for a lot more than that

https://lwn.net/Articles/824855/

[starshipeleven]

Where are you seeing that they are only using it for webservers?

They do use it for a lot more than that

https://lwn.net/Articles/824855/

more non-critical stuff (in the sense that reloading a backup or offloading to the next one still alive is instantaneous), containers, build bots and developer VM workstations.

While to store data, like user message cache when the user is offline they are using XFS

[RahulSundaram]

more non-critical stuff (in the sense that reloading a backup or offloading to the next one still alive is instantaneous), containers, build bots and developer VM workstations.

While to store data, like user message cache when the user is offline they are using XFS

They have backups and high availability, so in that sense, you can call any individual node within a cluster as non-critical by itself but when everything is production is on containers, calling containers as non-critical doesn't make sense and as the article covers and threads in fedora devel go into even more detail, they are using Btrfs increasingly and XFS where the COW features are not required or even a filesystem at all might be redundant like say db servers. This is nowhere near the same as just using it for webservers.

[starshipeleven]

They have backups and high availability, so in that sense, you can call any individual node within a cluster as non-critical by itself

yes, that's my point.

but when everything is production is on containers, calling containers as non-critical doesn't make sense

Each single container is non-critical because its only one of a swarm of thousands. That's what I meant with "offloading to another that is still alive" above.

and as the article covers and threads in fedora devel go into even more detail, they are using Btrfs increasingly

but not for data storage. All their stuff that uses btrfs is carrying "volatile" data (in the sense that it can be restored or respun from a golden image) or work data.

THis is my main point, they are using it to do "big company things", but not for "normal end user" things.

[Charlie68]

In short, if it is used by large companies, it is not good for the end user, but for the end user it is said that it is not reliable, but if it is used in a business environment it means that it is reliable.
In the Linux distribution field it has been used for years in openSUSE and SUSE, I'm not a fanatic of btrfs, but it seems to me that there are many fanatics of this or that file system.
Each of us can choose on any distribution which file system to use, this denigrating btrfs makes me think that even on file systems there are all kinds of fan boys.

[starshipeleven]

In short, if it is used by large companies, it is not good for the end user, but for the end user it is said that it is not reliable, but if it is used in a business environment it means that it is reliable.

No. Business use isn't a monolithic thing, not all data in your infrastructure is important, actually in many cases it's better to split up stuff so that you have all important data stored on its own and all processing done somewhere else, as that allows more flexibility on how you allocate resources to your IT stuff.

From what I'm seeing, btrfs in "production" is still limited to "volatile" data or containers or other crap where none will cry if a filesystem goes read-only and would require a multi-week process of rescue guided by a developer from btrfs mailing list, if shit hits the fan they just nuke the filesystem and copy over the data from a backup, or a master copy for VMs

Is this how the average end user uses his home server? I really don't think so.

If we look at places where companies store their data, we see NTFS (on Windows), ext4 (most embedded NAS devices and Debian/derivatives), XFS (server grade distros like RHEL or SUSE), ZFS (various Unix-y like OSes).

In the Linux distribution field it has been used for years in openSUSE and SUSE

It is not used to store user data, at least by default. ANd the support for managing btrfs RAID in Yast is new and kind of basic and mdadm-like for now (you cannot add or remove drives from an array, which can be easily done from console commands)

By default they are using it only for root filesystem, while home partition is split off. There is no way to install it in a RAID1 configuration either.
Again, more of the "businness use", they are using it for its snapshot capability and CoW features for system files, but data isn't stored there.

this denigrating btrfs

I'm being pretty fair and factual. I've been using btrfs for a couple years now on OpenSUSE Tumbleweed as both a PC and a KVM server and a ARMv7 NAS, and I'm really not enthusiastic about it. It's not unreliable (as long as you limit yourself to RAID1/RAID10 and don't use fancy features), but it's not the best thing since sliced bread either.

If ZFS had better integration with GUI tools and DEs (formatting and mounting drives formatted with zfs), I would drop btrfs in a snap.

[RahulSundaram]

THis is my main point, they are using it to do "big company things", but not for "normal end user" things.

Sure because unlike end users, large organizations have backups and high availability. You do agree now that this isn't just webservers, correct? It doesn't make them not real servers just because there is no single point of failure. Quite the opposite

You are also aware that large organizations have a lot more data and therefore can track when failures happen and that statistical information on failure rates+ additional real world testing maps isn't irrelevant for desktop systems. It is just not a 1:1 mapping and that isn't a particularly insightful observation

[Charlie68]

No. Business use isn't a monolithic thing, not all data in your infrastructure is important, actually in many cases it's better to split up stuff so that you have all important data stored on its own and all processing done somewhere else, as that allows more flexibility on how you allocate resources to your IT stuff.

From what I'm seeing, btrfs in "production" is still limited to "volatile" data or containers or other crap where none will cry if a filesystem goes read-only and would require a multi-week process of rescue guided by a developer from btrfs mailing list, if shit hits the fan they just nuke the filesystem and copy over the data from a backup, or a master copy for VMs

Is this how the average end user uses his home server? I really don't think so.

If we look at places where companies store their data, we see NTFS (on Windows), ext4 (most embedded NAS devices and Debian/derivatives), XFS (server grade distros like RHEL or SUSE), ZFS (various Unix-y like OSes).


It is not used to store user data, at least by default. ANd the support for managing btrfs RAID in Yast is new and kind of basic and mdadm-like for now (you cannot add or remove drives from an array, which can be easily done from console commands)

By default they are using it only for root filesystem, while home partition is split off. There is no way to install it in a RAID1 configuration either.
Again, more of the "businness use", they are using it for its snapshot capability and CoW features for system files, but data isn't stored there.


I'm being pretty fair and factual. I've been using btrfs for a couple years now on OpenSUSE Tumbleweed as both a PC and a KVM server and a ARMv7 NAS, and I'm really not enthusiastic about it. It's not unreliable (as long as you limit yourself to RAID1/RAID10 and don't use fancy features), but it's not the best thing since sliced bread either.

If ZFS had better integration with GUI tools and DEs (formatting and mounting drives formatted with zfs), I would drop btrfs in a snap.

When I talk about fanboys I mean just that ..., distinguish between enterprise, server, workstations.
Here, we are not talking about servers, Fedora uses Btrfs on workstations, but every time we talk about btrfs, we must always start from a distance, from reliability, from the enterprise and not from the context, which is in the desktop environment! Now I don't know anyone who uses desktop raids, surely there will be but they are a laughable minority.
It also doesn't make any sense to talk about ZFS every time, it will never be adopted by various distributions, until it is integrated into the kernel. You want to use it, but you won't find many distributions that will officially support it, aside from Ubuntu.

[aht0]

Now I don't know anyone who uses desktop raids, surely there will be but they are a laughable minority.

More of such people than you think. Why do you think motherboard manufacturers often add software raid option to the consumer motherboards? My aunts husband for example who is not IT person has RAID setup because of his photography hobby..

[starshipeleven]

Here, we are not talking about servers, Fedora uses Btrfs on workstations,

RAID1 is standard in serious workstations for obvious reasons (hard drives die eventually)

we must always start from a distance, from reliability, from the enterprise and not from the context, which is in the desktop environment!

We start from reliability because "desktop environment" is a situation where you have a single disk and probably have no backups. Therefore you need best reliability possible.

Even SUSE/OpenSUSE that has btrfs as default for root isn't using it for default in the home partition (where user data is).

Now I don't know anyone who uses desktop raids, surely there will be but they are a laughable minority.

Anyone that works with media (photos and video) or has collections of digital content (usually downloaded illegally), or is using a serious workstation is using RAID of some form.

It also doesn't make any sense to talk about ZFS every time

yes it does

it will never be adopted by various distributions, until it is integrated into the kernel. You want to use it, but you won't find many distributions that will officially support it,

"official support" is meaningless for a free product. I don't get any support if I have issues even for "officially supported" features.