Btrfs Gets Talked Up, Googler Encourages You To Try Btrfs

[nils_]

Marc and Chris were discussing this after the talk actually - are you using Laptop Mode Tools and/or aggressive power saving tweaks? I think Marc said he put in a bug report for it or was going to. I've encountered similar issues, but I thought I had done something to break it and so didn't dig very deep into it.

Nothing out of the ordinary, I think I'll just have to try that thing again on a test filesystem and then probably file a bug report. The only thing that could be considered unusual is that the underlying LVM volume is encrypted.

[JX8p]

As for changing license: Linux was here way before CDDL/ZFS and Sun probably intentionally made it incompatible with GPL in hope to reduce competition from Linux to their Solaris. But now we can see: where is Sun and their Solaris? And where is Linux? Sun plan status: FAIL. And Linux is strong enough to have its own way rather than resorting to being some footpad. Its especially interesting to take a look on FreeBSD guys who ditched GCC in favor of clang due to GPL, but then they do not mind using ZFS under CDDL, while CDDL is weak copyleft (what about traditional BSD mumblings about perfect freedom, huh?), which somewhat similar to GPL in overall idea but totally Sun-inclined and incompatible with GPL. What a double standards. Should I say FreeBSD people are looking like bunch of pathetic losers after such moves?

And disabling CoW is ONLY meant for cases when program on top of filesystem does journalling/cow on its own. This usually means something like database (probably primary usecase, since btrfs has been architected at Oracle) and VMs (they just do CoW on their own to be able to do snapshots). Disabling CoW for usual data == shooting your own legs (no journalling, means inconsistent data/metadata and many strange things resulting from data corruption).

First of all, Solaris' engineers specifically asked that ZFS be made GPL incompatible to prevent Linux from siphoning it up and turning it into a part of the Linux kernel where it would rot and fester in-tree, becoming directly attached and unportable. As for Solaris - the Oracle takeover killed it.

With regards to FreeBSD - 'perfect freedom' doesn't come into it, and 'FreeBSD people' is a meaningless term - your whining about 'pathetic losers' tells us a lot more about you than it does of them. CDDL is closer to LGPL than GPL. And FreeBSD was fine with GCC until the GPLv3 came, which the commercial consumers of FreeBSD found disgusting and unacceptable.

Well, either you don't include it in a distribution, and the user has to download it separately as a kernel module, which is not very practical, especially for, say, a default filesystem,
or you use it as a userspace component, which results in huge losses in performance for such a low level component as a filesystem.

It can be bundled as a kernel module, it just can't be linked into the kernel itself or stored in the same source tree as the kernel. I consider that a good thing - monolithicity is dangerous for code quality and abstraction.

[0xBADCODE]

Let's be fair here, ZFS on FreeBSD has been an option since 2008 with the release of version 7, however FreeBSD still defaults to UFS still 6 years after the fact.

But:
1) It has been unstable for few years regardless of bold claims. Most notable "stable" filesystem in FBSD lacked correct sendfile() syscall support, causing heck a lot of deadlocks when some program trying to use that and noticeable outcry about lockups.
2) ZFS wasn't written from scratch - its huge chunk of Sun's code. Btrfs on other hand has been writter from scratch, and some decisions were made in design which addressed some annoying ZFS shortcoming. Of course it takes longer to implement and debug than just take someon'e s code and glue it onto your design.

Btrfs is better in dealing with free/busy space and can, say, remove drive from pool as long as remaining data can fit remaining drives. Btrfs haves backrefs, it allow it to move data away from particular drive fast. ZFS design never assumed drive could be removed. So from technical side there is no easy and fast ways to move data out of particular drive on ZFS. Ha-ha. Then ZFS never took databases and VM disks into account. Once application does journalling logic on its own, its really bad idea to do CoW in filesystem as well. Btrfs has been desined to deal with it so it haves settings to disable CoW for particular files, making it okay for DBs and VMs (ofc no cow == no spanshots, but it's not like if you want snapshot of DB or VM via filesystem: it would break many assumption and wiil cause side effects). Then, Btrfs can technically assign various raid levels to various objects. Its oversimplified assumption each and every file is equally valuable. So technically there is nothing wrong to have mirror raid for valuable files and stripe raid for not-so-valuable files. Giving both good protection to valuable data and high-speed access to less valuable things out of one storage. This is not yet fully implemented but internal design allows it. While ZFS haves some interesting tricks to offer in raid area, it never meant to be so damn flexible. Then we can see, every time CoW does write, fragment is created. So CoW means filesystem is full of fragments. Brtfs devs acknowledged that prob and added defrag into GC logic. After all its so damn logical to try to defragment data when doing garbage collection - once you're about to (re)move stuff here and there, its logical to try to put it together, at least for mechanical drives. ZFS devs are mumbling some markerting bullshit instead that it "does not needs" defrag (BLATANT LIE for CoW designs). Sun marketing ppl are just offering to add new drives in timely manner to prevent pool usage over 80% or so (which would explode number of fragments quickly without ways to reduce fragmentation). Funny, especially since IIRC there is still no easy ways to remove drive from pool so far. Isn't it cool to replace technical solutions of complicated problems with loud marketing bullshit? That's what Sun did. IMO Sun's approach suxx. Then in ZFS we have some really moron memory management not really integrated with kernel memory management. Its okay on file server as long as it does not runs anything else. But can give a nasty surprise on big memory allocations otherwise. Ofc you can limit ZFS cache size but then it would underuse RAM and since underlying design appears to be SLOW unless backed by dozen of gigz of RAM, you'll get SLOW I/O. Not to mention its just wrong to turn user into cache manager. Btrfs on other hand meets best expectation in this regard, being integrated with kernel. So it not anyhow different from any other Linux filesystem and does not requires special attention in this regard unlike ZFS - it can use almost all RAM for cache but if someone would need that memory, kernel can free that memory and give it to other programs.

And no, you can't use ZFS as default. Its like selling everyone old Boeing 737 instead of car. Its slow to get it flying and assumes some overhead. If you need to drive some few miles, its really wrong idea. UFS is like a car. Old and crappy car from XX century. Its technical design can be compared to something like EXT2. It even lacks proper extents (as well as ZFS, lol), making it slow. UFS is IMO some ancient design which is not even worth of mentioning it when it comes to disk storage techs. Just like EXT2. It is outdated to the hell if we'll compare it vs techs used in other filesystems. But since FBSD mostly lacks other working filesystems, there is not much choice. Btrfs on other hand is like new ultra-modern car which can transform into suborbital jet, should you need to travel large distance. Sure, it made it more complicated than car and it sometimes exposes strange behavior, especially if you do not run latest kernels. But it design mostly finished and it already "just works" in many cases. Sure, it can do weird things in corner cases but it being ironed out at impressive speed each and every kernel release.

From practical view, btrfs serves me on some non-critical machines already and I haven't got any major troubles so far.

Further they waited quite a while until clang was finally in a good enough state for their purposes before they made the switch to it.

If we'll look on it from practical side, there was heck a lot of efforts put into garbage bin. A lot of work ... without user-visible results. It haven't improved anything from user standpoint. And hell, I can even use modern GCC 4.9 to build totally proprietary crap and it wouldn't violate any license. So I can't gain absolutely anything at all. Then gcc offers better optimisations and supports more platforms. What I can gain from such change? Some headache? That's what I call bad project management.

Also the only CoW filesystem that is permissively licensed at this point is HAMMER and its successor HAMMER2 which they'll probably adopt when it's ready.

1) I do not really care about freedom of few proprietary f...ks to close source and behave like a bunch of parasites. IMO its counterproductive approach.
2) BSDs never bothered self to sync efforts - each and every BSD fork has been totally incompatible. So I do not think they'll do it in some foreseeable future.
3) And its not like if I live forever - I can't wait these "permissively inclined" nuts indefinitely. At the end of day I have full hand of various tasks HERE and NOW. Sometimes I can wait for some time. Maybe year. Or two. But not more than that and it is much better when stuff works right when I need it, not 10 years later.

First of all, Solaris' engineers specifically asked that ZFS be made GPL incompatible to prevent Linux from siphoning it up and turning it into a part of the Linux kernel where it would rot and fester in-tree, becoming directly attached and unportable.

So we can see alive Linux kernel. It runs, it kicks. And then we can see ZFS. It gone into few incompatible flavours (Oracle flavour and community flavour are not compatible AFAIK). UFS story repeats itself one extra time? Some people are unable to learn from past mistakes, aren't they?

As for Solaris - the Oracle takeover killed it.

Its dependence on single vendor and lack of capable community what killed it. And its dependence on single vendor and being proprietary is what caused lack of capable devs around project. You see, you can't takeover Linus this way. It just not going to work. So its major disadvantage of project if it can't defend itself due to lack of those who are willing to work on it outside of Oracle. And it is a complex project management issue. Basically Sun did everything to make it happen this way. So its really wrong to complain about it. That's a fate of proprietary projects which lost their traction. And opensource is not about throwing source away when you're half-dead. Its about efficient collaboration without unnecessary borders and complications. Powerful process takes some time to spin-up, then it kicks the ass. Sun eventually got idea terrain is ahead, but even full engine throttle was not able to save them since it takes time to spin up. They were so slow with it they had no time, so they crashed their project and sold leftovers to Oracle. Sad but predictable.

CDDL is closer to LGPL than GPL.

It is. But it is still copyleft. I smell double standards in the air...

And FreeBSD was fine with GCC until the GPLv3 came, which the commercial consumers of FreeBSD found disgusting and unacceptable.

Well, it is acceptable for heck a lot of Linux customers. So I would assume its these commercial customers what is actually "disgusting" and "unacceptable" since they want to be parasites who closes source and never gives adequate payback to upstream. And taking care about wellbeing of parasites is really odd thing to do. IMO it hardly counts as "good deed" at all.

It can be bundled as a kernel module, it just can't be linked into the kernel itself or stored in the same source tree as the kernel. I consider that a good thing - monolithicity is dangerous for code quality and abstraction.

Its basically risky from legal point of view and takes extra efforts. And then there is btrfs which just works out of the box with mainline kernel and takes ZERO EFFORTS to get there. So yeah, theoretically I can fly to Moon tomorrow, there is nothing which prevents it in completely fatal ways. Practically it is highly unlikely to happen due to various issues. Same with ZFS - I do not think it would be warmly welcome in Linux. Parasites should go eat some DDT.

[jacob]

So we can see alive Linux kernel. It runs, it kicks. And then we can see ZFS. It gone into few incompatible flavours (Oracle flavour and community flavour are not compatible AFAIK). UFS story repeats itself one extra time? Some people are unable to learn from past mistakes, aren't they?

Actually the community flavour is fragmented into several sub-flavours. For example Linux ZFS is incompatible with other implementations if you enable SA xattrs, which is basically a requirement if you plan to use xattrs (or anything that relies on them, like ACLs...) at all.

Honestly I don't get these anti-GPL rants. It's not like the GPL takes away anything from anyone in the FOSS world, it just prevents unnecessary forking and fragmentation. It seems the BSD folks are really keen to give Apple and MS a free ride on their work for nothing. Well if that makes them happy...

[thelongdivider]

BTRFS is awesome

I recommend BTRFS to anyone at this point. On my main system, I have a 1tb ssd as my boot drive with a single filesystem with 4 subvolumes. Built in compression=lzo, CoW, snapshots, ssd optimizations, etc. No complaints, never unstable, and high multithreaded performance.

I also have a raid 5 across 5 3 tb drives, with the compress=zlib option and all the other usual options. It works quite well at this stage in the game, and I have hit no bugs over 6 months of pretty heavy use.

I'm all for the evolution of Ext4 to btrfs, and all of the ZFS competitive features.

[jimbohale]

Actually the community flavour is fragmented into several sub-flavours. For example Linux ZFS is incompatible with other implementations if you enable SA xattrs, which is basically a requirement if you plan to use xattrs (or anything that relies on them, like ACLs...) at all.

Honestly I don't get these anti-GPL rants. It's not like the GPL takes away anything from anyone in the FOSS world, it just prevents unnecessary forking and fragmentation. It seems the BSD folks are really keen to give Apple and MS a free ride on their work for nothing. Well if that makes them happy...

The GPL CAUSES (not the GPL directly but open source software in general) people to want to fork over any small little disagreement which fragments the SHIT out of the community, and if the two forks are both relatively popular and changing random software that depends on one thing is expected to be able to depend on the other, and it's just annoying. Now, I understand why the GPL exists and I'm not saying it's a bad thing, it's entirely up to the software manufacturer and I base my decisions on quality instead of licensing.

Also, don't say Linux ZFS is incompatible with other implementations if you enable some random feature you shouldn't be using on a ZFS filesystem. That's stating the obvious. It is perfectly compatible if you don't break it. In a lot of use-cases (quite a lot, I don't use them at all for anything other than a server that runs PHP which is cancer in itself) it's just completely unnecessary and certainly the instances which ZFS is meant for it's unnecessary. ZFS is great for reliability and throughput. Btrfs is great for losing my data on a power failure and being unable to be repaired. If btrfs improves enough to use RAM (via caching, and don't you DARE suggest having some bullshit block-level caching, it's far more effective to have it somewhere where it knows what it's going to cache rather than just cache random blocks on a COW FS) and not lose my data, then I will use it. It does not do much caching, and it loses my data, so I am not using it right now. ZFS does not lose my data, and it has been through many power failures and random drive disconnects. ZFS uses a lot of RAM, which is why when you use it you get a lot of RAM. I have a lot of RAM, and what else am I going to do with that RAM but use it. Seems silly to not. I am a ZFS *fan*, but I am NOT a fanboy. If Linux had something that was as good for my use-cases, I would use it.

I'm going to explain how this whole licensing thing works for you, since you don't seem to understand:

If the BSDs were GPL'd, Apple would not use it. Apple would instead hire people to create their own OS with the insane amount of money they had (or alternatively have purchased BeOS instead of NeXT). Apple would likely still have one of the more popular desktop OS's, while the BSDs would have gotten nothing from that exchange. Instead, Apple HAS used *BSD and they've given them quite a number of improvements and Clang (which is a godsend for programmers, if you say clang is of bad quality I'm sorry but I don't believe you're a programmer). I don't use GCC during development, and it's not that I care in the slightest about the fact that it's GPL'd, I don't use it because it is slow and it gives the worst error messages I have ever seen. When reading the code, it appears to be of low quality, and I do not have the time to write my own so rather than complain I just use clang. Anyways, going back to the topic *BSD has not lost anything from Apple using their code, it's a matter of they wouldn't have used it had it been GPL'd.

[Luke_Wolf]

The GPL CAUSES (not the GPL directly but open source software in general) people to want to fork over any small little disagreement which fragments the SHIT out of the community, and if the two forks are both relatively popular and changing random software that depends on one thing is expected to be able to depend on the other, and it's just annoying. Now, I understand why the GPL exists and I'm not saying it's a bad thing, it's entirely up to the software manufacturer and I base my decisions on quality instead of licensing.

While I more or less agree with the rest of your comment (not with all of it but most of the idea), I have to disagree with this part, Open Source Software doesn't lead people to forking over every little thing, the only times they're really created is when there's an issue with project management, either a group doesn't like the people running it or they're fouling things up or whatever. See for example libva vs ffmpeg, or LibreOffice vs OpenOffice (or for that matter before that Go-OO vs OOO), libreSSL vs openSSL, etc. Most projects are run well enough that forks, and particularly the situation of two or more popular competing forks are exceedingly rare and never happen over a small disagreement, and aren't really the fault of being opensourced.

On the other hand re-implementing the same idea a bunch of different times by different people is very common but being more open and permissive is the answer to this not something that generates a problem. If something is under a proprietary license then you're very likely to have a billion and one re-implementations that nobody knows about, if it's released under a strong copyleft license then it's much less likely to be re-implemented in the open source community but still will be re-implemented in commercial ventures that want to sell software licenses. If something is under a permissive or weak copyleft license then all developers are much more likely to just make use of that implementation rather than creating their own and thereby fragmenting effort. That said that's only one piece of the picture, code quality, design, project management and such also matter.

[Lennie]

Have only played with it, but...

im using ext4 for everything on a home server with some raid-0's and some raid 1's and other fss on ssd's. Ive never had a crash or any other issue, ever. Because i use it for family photos videos etc and important docs, i dont ever want to hear "not stable".

I messed around with brfs 5-6 years ago and thought it was da shit. Really, I love it to bits man, and cant wait to start using it on my 6 drives, but i aint moving jack squat till someone says that it is rock solid. period. if they get most of the cool features to rock solid stage then ill migrate even if they have some features that i dont need that are not fully stable... as long as they are crystal clear on which exact features are not 100% solid. if i see any comments anywhere about "minor corruption" it'll just put me off again. When was the last time you heard that about ext4?

If I feel like this about my own server, how can anyone consider moving to btrfs in an enterprise for a production server? What has Oracle done to make it production worthy on Oracle Linux?? What the hell is Chris M up to? is he deep into feature creep without fixing the core stability issues? Facebook giving him to much stuff to do not fs related? (If he still works there even. I lose track). Im no developer, and i imagine its hard work, but its been many years since we all thought it was good enough to use for testing with minor problems, but it doesnt seem to have moved on from there.

I hear (Open)Suse includes btrfs and disables all the btrfs features that aren't production ready AND picks the right kernel for you.

Basically, feature wise this means: no multi-device support (thus no useful RAID-like features) and maybe even no compression (I'm not on Suse).

I think to many people think developing a new filesystem is an easy task or that getting such a filesystem stable takes only a year or so. You are wrong. Sun at the time had more engineers on ZFS and it took years to create it. The number of developers of btrfs is increasing though. So that is good.

So, yes, you can use it with single drive filesystems. And enospace errors have happened on certain kernels in specific cases. And even if that happened, in most of the cases you can fairly easily switch to a kernel that does work and get out of problems. Even if you can't you can copy the data out of the filesystem with btrfs you can use the restore tool: https://btrfs.wiki.kernel.org/index.php/Restore

That is what I've seen on the btrfs mailinglist.

But, hey what do I know, I'm not a user of btrfs. :-) But I've been following it closely to see when I can jump to it.

[Lennie]

I hear (Open)Suse includes btrfs and disables all the btrfs features that aren't production ready AND picks the right kernel for you.

Basically, feature wise this means: no multi-device support (thus no useful RAID-like features) and maybe even no compression (I'm not on Suse).

I think to many people think developing a new filesystem is an easy task or that getting such a filesystem stable takes only a year or so. You are wrong. Sun at the time had more engineers on ZFS and it took years to create it. The number of developers of btrfs is increasing though. So that is good.

So, yes, you can use it with single drive filesystems. And enospace errors have happened on certain kernels in specific cases. And even if that happened, in most of the cases you can fairly easily switch to a kernel that does work and get out of problems. Even if you can't you can copy the data out of the filesystem with btrfs you can use the restore tool: https://btrfs.wiki.kernel.org/index.php/Restore

That is what I've seen on the btrfs mailinglist.

But, hey what do I know, I'm not a user of btrfs. :-) But I've been following it closely to see when I can jump to it.

Here is probably an even better description of how to recover from problems:

http://article.gmane.org/gmane.comp.file-systems.btrfs/37980

[jwilliams]

I think to many people think developing a new filesystem is an easy task or that getting such a filesystem stable takes only a year or so.

I doubt many people think it takes one year. But btrfs has been under development for 7 YEARS! ZFS had a production-ready release after 4 years (2001-2005). The development of btrfs should have been faster than ZFS since ZFS had already done some pioneering work, establishing some of the concepts of modern filesystems.

But the development of btrfs is just ridiculously slow, and it has been in permanent beta for years now. Chris Mason is just not cut out to be the lead developer for a project like btrfs.