OpenZFS Launches To Promote Open-Source ZFS

[ryao]

Not being able to shrink a pool seems like it makes the system less useful. Yes, you can get around it but it's something that you have to workaround. Moreover, with lots of small random writes, and not enough space zfs' fragmentation problems cause severe performance issues (http://blog.delphix.com/uday/2013/02/19/78/).
All this aside, can we not agree that zfs could've benefited from Ohad's, or other's, findings? In the end, what we are talking about is what comes after zfs. If you truly believe that zfs is the perfect filesystem then conversing with you about these things is pointless.

This is completely orthogonal but I thought you might find this interesting: http://sourceforge.net/p/snapraid/di...bc3413d5/#21b0
This guy apparently has a working quad and hex parity implementation.

I consider ZFS to be far better than anything else available by years, which is distinctly different than considering it to be perfect. You are the only one here suggesting that ZFS is perfect.

Anyway, those are real issues, but they have nothing to do with Valerie's post. Here is where things stand on each:

Shrinking: This requires block pointer rewrite. The only implementation of that was done years ago by Matthew Ahrens at Sun and never saw the light of day because it had performance issues. Oracle has a team of new people working on it in their fork, but it is safe to consider this to be stalled. Since ZFS is meant to deprecate partitioning, this is not a huge problem.

Small random writes: ZFS is already very good at this thanks to the write sequentialization provided by ZIL and SLOG devices can help here significantly. Etienne Dechamps wrote a FASTWRITE algorithm in ZFSOnLinux in 2011 that further improves things in the presence of multiple top level vdevs. More recently, Brian Behlendorf wrote patches that further improve this on mirrors. In late July/early August, I wrote a patch that implements a list of drives known to misreport their sector sizes in the Linux port. It will adjust ashift automatically when creating top level vdevs with drives in the database. This improves random IO performance on new pools that involve those drives when the system administrator did not automatically override the default when making the vdevs/pool.

Fragmentation as things reach capacity: All mainstream filesystems suffer from fragmentation and reduced performance when they fill to capacity. George Wilson at Delphix recently wrote a set of patches that help things degrade more gracefully.

As for higher raid levels, Jeff Bonwick's original plan for ZFS was to implement N-parity raidz, but Oracle's acquisition of Sun led to his resignation. He is now at a fairly secretive startup and this work has stalled. We have triple parity raidz, which should be sufficient for the foreseeable future.

[a user]

How can source code be put in a cage?

i think he means further developement and not the source code currently available. a licens like the BSD one allows to modify the current source and close it. it is a quite sick kind of interpretation to think such licenses are "free". it is like freedom to rob you and spit in your face.

sure, everybody how he wants it but i think there are good reasons from a sanity point of view to enforce a gpl like license. you need not to agree with that but ignoring the arguments again and again is getting borring.

at the end it reduces to the following question:

what is better or less bad: a license that allows companies to use that code, allowing them to close source all or a part to keep their secretes as they want (which is quite legit from their point of view) or a license that allow all to use that code but hinders you close and hide any further developement and modifications?

when i look at what consequences licenses like BSD, apache2 etc. had to the original open source projects - actually nothing came back - i think i prefer a license that ensure the code remains free.

own contributions of a company can still be held in own libs under a closed source license without issues.

an exagerated analogy: if the specified freedom allows some to enslave others it is not the right kind of freedom, right?

[liam]

Her points are not problems because they do not involve things that:

• Make the system crash.
• Make the system hang.
• Cause a build failure.
• Make the system less useful.
• Have a demonstratively negative impact on system performance.

If you cannot demonstrate that something clearly falls into one of those categories, it is not a problem and no one cares about it.

Not being able to shrink a pool seems like it makes the system less useful. Yes, you can get around it but it's something that you have to workaround. Moreover, with lots of small random writes, and not enough space zfs' fragmentation problems cause severe performance issues (http://blog.delphix.com/uday/2013/02/19/78/).
All this aside, can we not agree that zfs could've benefited from Ohad's, or other's, findings? In the end, what we are talking about is what comes after zfs. If you truly believe that zfs is the perfect filesystem then conversing with you about these things is pointless.

This is completely orthogonal but I thought you might find this interesting: http://sourceforge.net/p/snapraid/di...bc3413d5/#21b0
This guy apparently has a working quad and hex parity implementation.

[ryao]

If the problems Valerie specified aren't actual problems, I would be interested in learning actual reasons why they aren't.

Her points are not problems because they do not involve things that:

• Make the system crash.
• Make the system hang.
• Cause a build failure.
• Make the system less useful.
• Have a demonstratively negative impact on system performance.

If you cannot demonstrate that something clearly falls into one of those categories, it is not a problem and no one cares about it.

[liam]

You keep saying that there are problems, but you are not stating any concrete problems.

I've referred several times to the specific problems brought up in the article. They are concrete, and you have repeatedly failed to address Valerie's criticisms on their technical merit. Instead, you continue to dismiss them based on hearsay of her lack of involvement in ZFS as if one has to be a ZFS developer to have valid criticisms of the file system.

As for the "person I know", we are talking about Matthew Ahrens, who co-invented ZFS. I already told you what he told me. I suggest that you stop clinging to the idea that Valerie is a ZFS developer. She is no more a ZFS developer than Ted T'so. They are both entitled to opinions and their opinions do come from people with filesystem experience, but neither can claim authoritative knowledge over what is known as ZFS today.

An argument from authority is almost as bad as an ad hominem counter argument...

You are right that ZFS is not perfect, but you lack the knowledge to point out actual flaws. You have outsourced your own comprehension to an article whose conclusions happen to agree with what you wanted to conclude.

Ahem

I don't care about particular issues with zfs or btrfs since I've no great atttraction to either (neither are my pets)

Is that sentence difficult to comprehend? Or are you just making unsubstantiated assumptions?

For what it is worth, I and others are presently working on fixes for actual problems in the codebase. If you could point one out, I would readily admit it, but your current reasoning is unlikely to ever result in the identification of a real problem.

If the problems Valerie specified aren't actual problems, I would be interested in learning actual reasons why they aren't.

[ryao]

The problem is that, as she says, it wasn't known that b-trees and cow could exist at the time zfs was created and that's why the VM-like block pointer structures that zfs uses was chosen. The Rodeh paper wasn't released until 2006, IIRC. So, as she was presenting the info, extents would've been the preferred choice for allocation. Since she's been involved with the creation of a number of filesystems, and I've not heard anyone argue her facts about zfs, I'd say she knows what she's talking about.




What matters is her technical knowledge of the topic. Does this person you know who worked at sun around the 2002-2004 timeframe on the zfs project recall if she understood the codebase? Was she involved in the coding at all or was she a cheerleader? I think she was probably involved with the development since she's gone on to do other development work on fs. That aside, my point stands unharmed since what matters are her technical arugments against which I've not heard anyone argue. Again, perhaps your friend would care to submit an article explaining how she was wrong to lwn?
I don't care about particular issues with zfs or btrfs since I've no great atttraction to either (neither are my pets), so indicating problems with btrfs doesn't change the facts.
I think my points have been pretty plain. ZFS isn't a perfect filesystem. The fact that it has problems (as I've mentioned) even after all of its development (manpower which dwarfed that which attends btrfs I think you'll agree) suggests she probably had a point about zfs having inherent design problems.
Until you can understand where zfs went wrong (again, as it must've) you probably don't understand the code well enough.

You keep saying that there are problems, but you are not stating any concrete problems. The fact that you can use extents does not mean that you should. If extent backed files are an improvement, we are able to implement them in Open ZFS. So far, I have seen no compelling reason to do that.

As for the "person I know", we are talking about Matthew Ahrens, who co-invented ZFS. I already told you what he told me. I suggest that you stop clinging to the idea that Valerie is a ZFS developer. She is no more a ZFS developer than Ted T'so. They are both entitled to opinions and their opinions do come from people with filesystem experience, but neither can claim authoritative knowledge over what is known as ZFS today.

You are right that ZFS is not perfect, but you lack the knowledge to point out actual flaws. You have outsourced your own comprehension to an article whose conclusions happen to agree with what you wanted to conclude. You can do any reasoning you want about any subject by doing that, but such reasoning is invalid (affirmation of the consequent). If this is a topic that interests you, I suggest that you stop relying on what others write and go get some actual experience (e.g. osdev.org).

For what it is worth, I and others are presently working on fixes for actual problems in the codebase. If you could point one out, I would readily admit it, but your current reasoning is unlikely to ever result in the identification of a real problem.

[liam]

Valerie is the only person of which I am aware who has claimed that the use of gang blocks to be "hacky". The extent approach in btrfs could be considered the same concept applied to the entire file. All filesystems fragment files when there is insufficient contiguous free space.

With that said, real issues are things that affect userland. This is not one of them.

The problem is that, as she says, it wasn't known that b-trees and cow could exist at the time zfs was created and that's why the VM-like block pointer structures that zfs uses was chosen. The Rodeh paper wasn't released until 2006, IIRC. So, as she was presenting the info, extents would've been the preferred choice for allocation. Since she's been involved with the creation of a number of filesystems, and I've not heard anyone argue her facts about zfs, I'd say she knows what she's talking about.

As for Valerie, her article was linked specifically because she claimed a connection to ZFS (that implied an authoritative position) and she stated that she believed btrfs was better. I find it doubtful that it would have been linked if one of those two were not the case. It is therefore perfectly valid to question her connection to ZFS. Making some minor contributions to an early prototype that likely did not survive in what is now known as ZFS does not make one a ZFS developer. I can say with confidence that any claim that her opinions are that of a ZFS developer are false.

That being said, I cannot say that I am interested in doing the analysis of her article that you want. People who link it have already decided that they prefer btrfs and that is fine. Any attempt at analysis will predictably result in responses either nitpicking in the case that I do write one or stating that I cannot in the case that I do not. Given that this exercise is a complete waste of time, I will opt for the latter. In this case, that would not be entirely untrue. There exist people that are far better qualified to do this analysis than myself. Of course, this exercise would be a waste of time for them too.

I will leave you with one interesting fact. ZFS fills predictably such that writes will fail with ENOSPC only when df reports less space than the write (e.g. 0). btrfs can report ENOSPC without warning even when df reports substantial free space. A rebalance will fix things until the next ENOSPC, but that is not well suited for production environments, where space requirements need to be predictable. Few who read Valerie's claims of efficiency in comparison to ZFS would expect btrfs to have this problem when ZFS is free of it. Even fewer would expect btrfs to have pioneered it.

What matters is her technical knowledge of the topic. Does this person you know who worked at sun around the 2002-2004 timeframe on the zfs project recall if she understood the codebase? Was she involved in the coding at all or was she a cheerleader? I think she was probably involved with the development since she's gone on to do other development work on fs. That aside, my point stands unharmed since what matters are her technical arugments against which I've not heard anyone argue. Again, perhaps your friend would care to submit an article explaining how she was wrong to lwn?
I don't care about particular issues with zfs or btrfs since I've no great atttraction to either (neither are my pets), so indicating problems with btrfs doesn't change the facts.
I think my points have been pretty plain. ZFS isn't a perfect filesystem. The fact that it has problems (as I've mentioned) even after all of its development (manpower which dwarfed that which attends btrfs I think you'll agree) suggests she probably had a point about zfs having inherent design problems.
Until you can understand where zfs went wrong (again, as it must've) you probably don't understand the code well enough.

[liam]

"As for "insider information", the project is oss so I don't know what that even means."

This seems like you were implying that a concept like "insider information" wouldn't apply to a project that is open source. I was only commenting on this line, because it seems to give the impression that there's no such thing as insider information in open source. Or maybe, that you don't know what would be considered "insider information" when talking about open source software. If that's the case, the answer is simply, the same thing that is considered "insider information" when talking about any other kind of software: something only the "insiders" - ie. people involved with the development of the software - are privy to.

There's plenty of software that is open source, but is still developed behind closed doors, with only the people involved in the project knowing what is going on between releases. Like Android, or Ubuntu (to an extent). As such, "insider information" would be anything known only to the developers but not to the public.

I took his comment to mean he was relying on her knowledge of the codebase. I don't see why it would matter who did what, in this case, since I've heard no one argue her tecnical points (which I've reiterated above), thus anything outside the code should't matter.

[hasib39]

Laptop

I have no topics for this matter so pls Inform me anyone how can find laptop site
laptop

[ryao]

I can't help how you view arguments, but she has been a pretty influential fs developer according to her wikipedia page (her maiden name is Henson) since she's done more than just zfs.
As for "insider information", the project is oss so I don't know what that even means. The points she made about issues with zfs were fragmentation (related to a hacky, according to her, method to get around the preallocated blocksizes and zfs' requirement that an object not make use of blocks of different sizes) and inability to do true partition shrinking.
No one is saying that zfs is a bad fs. As you say it might very well be the best we have (though, as I've said elsewhere, amplidata's dss looks next-generational), but that doesn't mean it is perfect, or couldn't be improved. Btrfs looks to be able to fix some of these issues but it remains to be seen if it can match the rest of zfs' positive attributes.

Valerie is the only person of which I am aware who has claimed that the use of gang blocks to be "hacky". The extent approach in btrfs could be considered the same concept applied to the entire file. All filesystems fragment files when there is insufficient contiguous free space.

With that said, real issues are things that affect userland. This is not one of them.