Proposed Reflink Support Would Provide Big Space Savings For Wine

[geearf]

So how would this actually work? When I create a new prefix, where does it get the original files to ref-link? ~/.wine is the default Wine prefix, but not necessarily one created with the same Wine version used to create the new prefix... I just don't understand how this could work without a Wine manager that knows which profile are using same Wine version and which are not. Will they be ref-linked from the system-installed package? I guess that should be acceptable for those that only have 1 version installed at a time.

[oiaohm]

So how would this actually work? When I create a new prefix, where does it get the original files to ref-link? ~/.wine is the default Wine prefix, but not necessarily one created with the same Wine version used to create the new prefix... I just don't understand how this could work without a Wine manager that knows which profile are using same Wine version and which are not. Will they be ref-linked from the system-installed package? I guess that should be acceptable for those that only have 1 version installed at a time.

Yes this would be reflink to the system package/packages. /opt/wine-devel/lib64/wine/x86_64-windows/ and /opt/wine-devel/lib/wine/x86_64-windows/ as possible paths.

https://www.winehq.org/pipermail/wine-devel/2021-July/191267.html

The vast majority of these files are byte-for-byte identical to Wine's central DLL copies.

Do note when you run a different version of wine over a wine prefix and you see updating wine prefix most of these central dll are being replaced in the prefix. There is are reasons why its not recommend to use multi wine versions in the same wine prefix. Problem 1 is the dlls in the wine prefix are version particular and at times you will run into the event where the application loads the system wide and the system32 in wineprefix at the same time and if they don't match problem. Wine is fairly good at updating wineprefix directories very bad at downgrading wineprefix directories because libraries from newer versions of wine can be left in place with a old wine version down grade then do something the old wine version totally hates. Next is the differences in registry entries between wine versions its the same kind of problem is this unknown to old wine registry entry something a application added or is this something a new version of wine added this can become a very hard question to answer

Of course if your location where wine is system wide installed is on a different partition to your home directory this relink solution in it current form is not going to work.

The explosion in wine prefix size is directly linked to the move from wine pe/elf to pe Yes in the pe/elf day the dll files in the wine prefix were a lot smaller due to being fakes with as small as code as possible just to redirect to the pe/elf binary bad new copy protection stuff hated finding elf headers and dll stacking. What use to be under 5 meg per wine prefix elf/pe with fake dlls has exploded to 150-300 megs in the PE dll time frame yes reflinking brings this back to 1 meg. The old fakedlls were not exactly the most disc effective option.

I see this as first step. I do wonder if support of a wine user to reflink against on a users home directory when that home directory is a different partition may be a good thing but I see this as a future problem. Please note you really don't want to be reflinking masters from something in the users home directory that could have been damaged by a install of a program when attempting to diagnose issues. With the way reflink works you get a error wine a reflink cannot be done so it is possible to have multi master copies with 1 master copy per partition and systematically attempt reflink after reflink until one works and if non works drop back to copy.

https://www.ctrl.blog/entry/file-cloning.html

Please note this reflink support long term is not only need to solve the wineprefix size explosion caused by going PE. It is also need to support applications expecting refs2 and to use "DUPLICATE_EXTENTS_TO_FILE" feature under windows. Yes you need applications to know if a file has managed to be de-duplicated or not. Yes some applications will alter their operations when they find that X location will not de-duplicate straight away.

So reflink support by some means over time will come a more required feature.

[landeel]

I just mounted Windows NTFS/FAT partition under Linux and used it that way. Surprisingly this worked well with Witcher 3 on Windows.

I'm currently using NTFS, but it has a few problems:
1) Bad performance if you use the NTFS-3g fuse driver;
2) Unstability, data corruption if you use the upcomming "NTFS3" driver (from my experience, I hope it gets better);
3) Symlinks made with NTFS-3g and NTFS3 are not compatible with each other, and not compatible with Windows. Makes a mess! ;
4) Steam/Proton compatdata (wine prefixes) doesn't seem to work correctly on NTFS.

[intelfx]

What they say is that zfs is fast enough that the bottleneck is the disk itself, and adding this syscall merely helps with deduplicate blocks.

They are not saying that it is undoable, since CoW filesystem, by its nature, split files into blocks so that on write, only the blocks that are updated and the meta blocks need to be added to the disk as new blocks.

They could easily support reflink by simply copy over references to the old blocks.

BTRFS also uses the same way since it is also a CoW filesystem, and if BTRFS can do it, there’s nothing by design that ZFS cannot do.

That's a very simplistic view of the matter. There's enough differences between btrfs and zfs despite both being based on the copy-on-write principle.

And no, what they say is exactly what they say — that zfs does not (yet) support reflinks, not that zfs is "fast enough" or anything else.

[binarybanana]

Another good option for shared storage is UDF. It's meant for DVD originally, but it works well for all kinds of media, including USB, HDD and SSD. It supports links and UNIX permissions among other stuff. Technically even something like (frozen) snapshots should are possible. Full RW support is available on almost any notable OS. Probably the most widely supported FS by far. It's surprising how little it's used.

If you're really hardcore and need to access a LUKS/LVM/bcache/btrfs volume or something similarly convoluted on Windows a small Linux VM that mounts and exports that over NFS (or SMB) also works well. Performance with VBox and qemu sucks badly though, but VMware easily pushes hundreds of MB/s over the virtual network.

[pal666]

What they say is that zfs is fast enough that the bottleneck is the disk itself, and adding this syscall merely helps with deduplicate blocks.

no, they said they can only do implementation which will be not much faster than no implementation, i.e. not much faster than read of source file followed by write of destination file. they could skip only memory copy, but before doing memory copy you have to read from storage to memory. on serious filesystems reflink avoids reading and writing file blocks, it just adjust pointers.

They are not saying that it is undoable, since CoW filesystem, by its nature, split files into blocks so that on write, only the blocks that are updated and the meta blocks need to be added to the disk as new blocks.

it was already said this has nothing to do with cow. besides btrfs target users of this syscall are xfs and nfs

there’s nothing by design that ZFS cannot do.

lol. you should organize zfs sect. there was an lwn article from ex-zfs dev in 2009 which i posted here explaining its design deficiencies.

[pal666]

Sure adding a feature adds overhead, but I think using a relink filesystem like btrfs over ext4 is the bigger slowdown

btrfs is faster than ext4 when you measure important things

[oiaohm]

4) Steam/Proton compatdata (wine prefixes) doesn't seem to work correctly on NTFS.

https://wiki.winehq.org/FAQ#Does_Wine_run_on_all_Unix_filesystems.3F

That one is answered in the wine FAQ. NTFS-3g is missing a feature lot steam hosted applications/games expect to work. This does not matter if it steam proton or the game installed in wine directly with some of these feature either. Yes using ext4 as a loop back file on a NTFS to run wine games can stupidly work out better than running straight on ntfs-3g.

[oiaohm]

Another good option for shared storage is UDF. It's meant for DVD originally, but it works well for all kinds of media, including USB, HDD and SSD. It supports links and UNIX permissions among other stuff. Technically even something like (frozen) snapshots should are possible. Full RW support is available on almost any notable OS. Probably the most widely supported FS by far. It's surprising how little it's used.

https://en.wikipedia.org/wiki/Universal_Disk_Format#Compatibility

Not really there is a nice little catch here. Yes Linux can read 2.50 and 2.60 UDF but cannot write it. Windows can auto update UDF 2.0x to 2.50/2.60. This leads to the fun you were able to read write 2.0x in Linux you boot into windows copy a file and now its read only when you return to Linux. This is another case driver support is problem. Please note mac os also says stuff you if windows upgraded UDF to 2.60

Using a file server like SMB/NFS is way less of odd ball stuff happening when sharing files between OS..

[oiaohm]

And no, what they say is exactly what they say — that zfs does not (yet) support reflinks, not that zfs is "fast enough" or anything else.

Orcale ZFS does support reflinks and SUN version of ZFS for Solaris also supported reflinks. This is before openzfs and zfs for Linux and zfs driver in freebsd existed. This is more the other implementations of ZFS have not implemented a feature the on disc format of ZFS in fact supports. Linux kernel support for reflinks is part based on the Solaris stuff.