Proposed Reflink Support Would Provide Big Space Savings For Wine

[oiaohm]

i don't really care how much they have to implement to make it work, the fact is they didn't do it and they said they can't. i.e. their words can't be used as proof of "zfs supports reflink" which was attempted in comment to which i replied

The problem here is the oracle version of ZFS does support reflink and does use the same disc format. Yes oracle version of ZFS that have at times made work with Linux did in fact have the syscall working.

The reality its just a openzfs and zfs for Linux has chosen not to.
https://www.ctrl.blog/entry/file-cloning.html It is noted here that orcale zfs supports it.

Use cases why exposing reflink is important is the cases where you need to know if there is one or more copies of a particular data on storage. Yes de-duplicating is good to a point but when you get to the point that something does in fact need 3 copies because it critical to deal with bit flip problems not so good.

https://lwn.net/Articles/332802/ Where does Linux kernel reflink implementation come from none other than Oracle.

https://docs.oracle.com/cd/E88353_01/html/E37843/reflink-3c.html

Yes the Linux implementation of reflink design is directly based on the Solaris design of reflink. Yes the Solaris reflink functions are only for use with ZFS at the start under Solaris.

The reality here ZFS as a file system support reflink and did from day one. The third party drivers implementing Zfs such as openzfs and zfs for Linux have chosen not to support this. Ok under BSD platforms that don't have a reflink syscall this does make sense. But on Linux that has a reflink syscall based on the solaris reflink syscall.... this is a different matter right.

[pal666]

That at straight up lie with the incapable bit. Zfs by design is capable of providing reflink. Oracle ZFS has the feature. The data structs of ZFS include the feature. The open source implementations just don't implement it. Having the function implemented we can see with Oracle ZFS that most of time it no faster and no slower than without reflink. But there are select usage cases where having reflink support ends up faster with the Oracle ZFS.

i don't really care how much they have to implement to make it work, the fact is they didn't do it and they said they can't. i.e. their words can't be used as proof of "zfs supports reflink" which was attempted in comment to which i replied

[oiaohm]

that's not what they said. they said zfs is incapable of providing reflink support, i.e. they could support syscall, but it will be as slow as without it

https://www.ctrl.blog/entry/file-cloning.html

That at straight up lie with the incapable bit. Zfs by design is capable of providing reflink. Oracle ZFS has the feature. The data structs of ZFS include the feature. The open source implementations just don't implement it. Having the function implemented we can see with Oracle ZFS that most of time it no faster and no slower than without reflink. But there are select usage cases where having reflink support ends up faster with the Oracle ZFS.

[pal666]

There are many people who prefer old style partitioning with /home and / on mounted on different partitions. Easier to do clean reinstall that way. Reflinks don't work across different filesystems.

btrfs supports superior way of partitioning: /home and / on different subvolumes of same partition, which will not require you to ask your crystal ball how to split available space between partitions. it will support same easy clean reinstalls and it will support reflinks(if you will not mount subvolume separately, i.e. just make subvolume at /home but do not mount it)

[pal666]

According to https://github.com/openzfs/zfs/discussions/4237, they just don’t bother to add it as it does not improve performance significantly.

that's not what they said. they said zfs is incapable of providing reflink support, i.e. they could support syscall, but it will be as slow as without it

[oiaohm]

I always thought Wine needed some kind of virtual file system or overlay. It could work like reflinks but also be used for stuff like winetricks where you would have to install painful stuff like dotnet (still broken) only once and to reuse it you just add an override on top of another bottle. It would need to go beyond the filesystem probably like it touches the registry and who knows what else.

This turns out to be a lot harder. Please be aware that people changing between a lot of different questionable software end up having to complete reinstall windows as windows users. The problem file system and registry for most programs. The problem comes what is truly global set and what per user. Yes people do find particular things like some programs if you have multi users on windows and use a different user and that user does not have access to the other users home/profile directory the software dies. Because in system wide setting the application put user particular directories or user particular preferences. Yes this could be in the programs folder in a configuration file or in registry.

Yes this behaviour can change with software updates. So reuse is a lot harder. Yes being able to install a bottle/prefix and duplicate is a feature wine supports. relink copy a wine prefix I have done before xfs. This starts showing you how often programs do horrible things because files installed by something like dotnet you see programs modify global setting(as in files installed by the likes of dotnet' to suit their needs. Yes this global modification explains why if windows users installs a mixture of random applications they roll snake eyes from time to time. Yes the wine install 1 application per prefix is a diagnostic path to work out there is software conflict.

In a ideal software world you should be able to install all windows software in one wine prefix that don't require overrides. The reality is this does not work even under windows. Wineprefix is a work around to wine limitations and a windows limitation.

AHOY --reflink=always and --reflink=auto of the cp command are very useful when on a relink supporting file system to copy wineprefix/bottle with to keep sizes down to a point but as wine prefixes update you will notice they will also split from each other and you will have to run a deduplication pass to merge them back to each other.

Other thing to consider a runtime virtual file system does come with a overhead cost. relink route with a de-duplication tool is very light in overhead because you are working with a feature the file system supports.

Yes wine virtual file system for faking up case name insensitivity is very expensive and faking up something like reflinks is even more expensive as it would result in scanning more and more directories and having to do copy operations. AHOY there are just cases like case-folding and keeping wine prefix de-duplicated as possible you really do need for best performance file system support as native file systems can do this in synced structures without horrible overhead. The problem here is between the time you last scanned the file system and now how do you know if you are still current. File system global state is the problem to replicate as a layer on top. Global state as the file system itself is really simple to implement.

Do note even if wine did implement own overlay/reflinks in userspace you would still have the problem of needing de-duplication tool to merge prefixes back to only have 1 copy of a file when a file was updated in 2 or more prefixes to the same file by different applications.

Yes using btrfs or xfs reflink feature has manually been usable with wine for quite some time to keep prefix size down. Supporting auto reflinking when file system supports it will be beneficial. Even so after auto reflinking you still need a de-duplicate tool to scan over wine prefixes to find files that have been duplicated that need to be merged back for absolute ideal disc usage.

https://www.ctrl.blog/entry/file-cloning.html

The annoying part about ZFS here is oracles ZFS does support reflink yet openzfs freebsd zfs and zfs for Linux don't. Yes ZFS on disc format has the structures to support reflinking.

EXT4 problem is it does not have the disc structures to effectively support reflinking a developer had a look at it again in 2014 and it was found you add reflink support to ext4 you have to add a wrapper layer to patch over the disc structures limitations that would result in lower performance. I am not talking small percentage drop here I am talking write operations being like 10-15% slower.

There is no ideal file system for wine under Linux. Case-folding limited file system support and reflink support limitations are both problems. Ideal file system for wine would support both.

[oiaohm]

What stops them from having globally accessible system/system32 directories for DLLs that can be overriden by files within prefixes? I mean look for a dll inside the prefix first and if it's not present there, look for it in Wine installation directory. It would do the trick and there wouldn't be anything filesystem-specific to it. Are there any reasons why it couldn't work like that?

Anti cheat and copy protection made by some vendors install/upgrade process. Wine dll load order has been doing what you are talking about for a while.

The problem patterns goes as follows.
1 ) Open file handle to DLL.
2) use this file handle to modify DLL.
3) expect this to have happened with no error.
4) when program loads without modified DLL destory program registration information due to being hacked.

Why did 2 fail simple system wide file has to be protected from modification. So a COW is not a optional feature here.

Anti-cheats/copy protection that contain their own dll loaders. This is also why wine dll fakes had to go from a place holder to really callable library place holder libraries as well. Yes these will want the file handle to the dll open and read the dll . So this means you have to provide a faked "system/system32" or have a file system that can reflink to share effectively.

[AHOY]

I always thought Wine needed some kind of virtual file system or overlay. It could work like reflinks but also be used for stuff like winetricks where you would have to install painful stuff like dotnet (still broken) only once and to reuse it you just add an override on top of another bottle. It would need to go beyond the filesystem probably like it touches the registry and who knows what else.

I'm curious to see how it will work out in practice since I use 10 different wine versions, proton, patches, and downgrade/upgrade them even in the same profile.

[a7v-user]

Does Btrfs and XFS support case-folded file names?

[bple2137]

What stops them from having globally accessible system/system32 directories for DLLs that can be overriden by files within prefixes? I mean look for a dll inside the prefix first and if it's not present there, look for it in Wine installation directory. It would do the trick and there wouldn't be anything filesystem-specific to it. Are there any reasons why it couldn't work like that?