Btrfs To Ship Multiple Performance Improvements In The Next Linux Kernel

[ssokolow]

Congratulations! You're the third guy to answer and completely fail to understand the meaning of AUTOMATIC. Automation is the complete opposite of having to manage everything by hand. OP did not ask how to disable COW, he asked how to make it automatic. Implying he already knew how to do it manually.


That's just retarded. Obviously there's no sense patching every single app that works with files to support a filesystem-level feature. It has to be a system service at a bare minimum if it cannot be built into the kernel.

I don't see how that could be accomplished without some kind of heuristic (ie. fallible) algorithm to guess at which files should and shouldn't be COWed.

[jacob]

I don't see how that could be accomplished without some kind of heuristic (ie. fallible) algorithm to guess at which files should and shouldn't be COWed.

IIRC ZFS has something along those lines. IMO it's a horrible idea. To think that the FS driver would decide on its own and without intervention to disable its main integrity features based on some internal heuristics is just abominable. The switch must be manual, because it's only acceptable if the application that uses such files has itself some integrity mechanism built-in, and that's something only the user knows.

[Spazturtle]

That's just retarded. Obviously there's no sense patching every single app that works with files to support a filesystem-level feature. It has to be a system service at a bare minimum if it cannot be built into the kernel.

Not every app no, only ones that don't work well with COW. Only the application itself can know if it doesn't work well with COW, and only the application knows if it is safe to disable COW.

[garegin]

Not every app no, only ones that don't work well with COW. Only the application itself can know if it doesn't work well with COW, and only the application knows if it is safe to disable COW.

That’s what i had in mind.
i wonder how APFS or ReFS does it

[Michael_S]

(2)Bunch of random incidents aided Linux equally or more than it's license. Linux took off in popularity when FreeBSD was implementing SMP and did at first shitty job. Then FreeBSD's users (it was used far more than Linux back then) migrated to Linux because it happened to be ready and accessible alternative (no OpenSolaris yet). Later times, additional factors aiding Linux were Oracle closing OpenSolaris after buying Sun and Google opting to use Linux kernel for it's new embedded OS Android. Without all of it, FreeBSD or OpenSolaris could easily be in the same position Linux has nowadays. Just mostly luck IMHO.

Where GPL in fact aided and served it's purpose was with Linksys court case - suddenly people could have access to sources for their routers - would not have been possible with BSD license.

Also, define success - Linux has a few percents market on desktop and still less far smaller market share in servers than Windows, except for web servers where it indeed rules the roost.

I disagree on the license part. Many of the important contributions to Linux came from corporate-backed developers even in the late 1990s and early 2000s. It was popular then, but at the time it didn't dominate servers like it does today. The only reason many of those contributions went upstream was GPLv2. FreeBSD will never catch up because companies like Apple and Sony that make extensive use of it have no interest in pushing back key in-house enhancements that might allow it to compete with their own products.

Copyleft is free-as-in-freedom, BSD/MIT/Apache is free-as-in-labor. And of course today companies are violating the terms of the GPL or using firmware and associated tricks to avoid releasing their code while still using Linux.

And Android explains why Linux is popular as the kernel on mobile operating systems. But even without Android, Linux conquered the server space.

[waxhead]

Congratulations! You're the third guy to answer and completely fail to understand the meaning of AUTOMATIC. Automation is the complete opposite of having to manage everything by hand. OP did not ask how to disable COW, he asked how to make it automatic. Implying he already knew how to do it manually.

Yup, thanks for noticing. All good things comes in threes I guess. My mistake, and I hereby nominate myself to the ignorant-ego-bastard-of-the-week award. I only read the first page so there you go

Anyway , doing this automatic has to be done on the application level. And even then it is sort of "against" what the user expect of his/her/it's? filesystem.

The best "automatic" method is to create a separate subvolume, set that nocow and store your things in there. Ergo , a isolated part of the filesystem having nocow attribute set. On the other hand you *might* want to consider other filesystems for your use case , but then again you loose the flexibility of BTRFS anyway. Or another option is to enable the autodefrag option that sort of does the trick for some stuff.

[kreijack]

Not sure what you mean by "checksums for parity against bitrot"? Btrfs has had checksumming for reliable data and metadata for years now.

parity was not checksummed in the past

BTRFS doesn't have parity checksummed. From a reliability point of view, there is no gain to have the parity checksummed. Even in the case that both the data and the parity are corrupted, the current BTRFS checksum infrastructure is good enough to detect the problem.
The only advantage having a parity checksummed is the speed to detect a possible corruption. However, again, the parity checksummed is not needed from a reliability point of view.

[Nille]

kreijack lets think about this scenario. Raid 6, 5 Disks and one Disk failed. Parity on one disk is invalid. How does btrfs fix this problem now? How does it know which parity is invalid? Does the restore check it that the file was invalid with parity A and test again with parity B?

This is one of my concerns.

[caligula]

This is great and all, but I don't think most desktop users notice much difference in filesystems, I certainly don't. The difference between a HDD and an SSD is night and day, but once you've made that leap I don't think there's much in it. I recently went from a SATA 3 SSD to an NVME m.2 drive for reasons unrelated to performance, I didn't notice anything. I use BTRFS, I doubt I'd notice any difference if I went with EXT4, or whatever the performance leader is.

Still, that's just my use case, I'm sure under different workloads, especially in the server space, this is particularly useful. And it's nice that it's getting development attention.

If your workload actually stresses the disk, your NVMe drive might throttle after few seconds due to overheating which closes the gap between SATA SSD and NVMe. This is quite likely if the drive is powerful and doesn't have a heatsink installed (it won't unless you bought one). Now, why a typical Linux user won't even notice this is the basic file operations might make extensive use of Linux disk cache, which works really nicely both for writes and reads. For example, my system typically uses around 2-3 gigabytes of RAM as a disk cache after starting up the desktop, browser, and other basic applications. A large part of my whole distribution is now loaded in RAM. It would work fast even off of an USB 1.0 drive. When I do some work, the tools don't read whole files, they use random access and seek only relevant parts. The difference is more obvious when actually doing something terribly I/O bound like having a big ass server or moving 500+ gigabytes of stuff around. When you operate the system manually, I'd argue that in many cases the tasks just don't generate enough IOPS to saturate the SATA SSD.

[GreenReaper]

And, of course, Facebook uses BTRFS.

...largely in single mode, as I understand it. Also Facebook probably has node-level redundancy going on. It might be more more suitable for systems with hardware RAID if you're looking for redundancy.