Wine Reflink Support Continues To Be Worked On For Significant Space Savings

[oiaohm]

I'm really not that familiar with MacOS and exact FS that is mostly used on that OS, but if there's such feeature on the majority of MacOS systems, even better.

macOS High Sierra (version 10.13) September 25, 2017 and newer APFS has the support that matches up to reflink. Yes its the default file system for those MacOS versions clean installed.

If there's any valid reason why that isn't implemented in EXT4, I assume there is, but, I don't know even close enough about it to even discuss it.

I gave you why its not implemented. 2006-2009 from that time frame the lead developer of EXT4 says the same thing its a stop gap this is why features like reflink are not followed on with.

Google has picked up and implemented some features that the lead developer 2006-2009 said would not ever be implemented in ext4 and has implemented and got them merged. Just the reflink feature has not got any major parties attention.

https://lwn.net/Articles/789038/

Not having reflink in ext4 is leading into some horrible problems. This is why I don't see that it wine job to fix the reflink problem. Its something that should be fixed in ext4 it self that it could be or in overlayfs snapshots. Remember file system ext4 and the overlayfs can hook into core kernel structures to help with problem that wine should not touch.

[NobodyXu]

That would be fine, but I do not have a need to change FS, I do not need any of those "modern features", and making application work with specific FS is a very bad idea.

I think the WINE team probably treats this as an optimisation rather than a feature.

Just like cp has option `--reflink=auto`, which is nice to have if available and doesn't affect `cp` if not available.

[leipero]

This is missing a few key points still.

https://arstechnica.com/information-technology/2009/04/linux-collaboration-summit-the-kernel-panel/

Ext4 was a stop gap file system. This is why features like reflink were left out because ext4 was not being designed to last. Yes that T'so was the lead developer of Ext4 yes plan was by now you would not be using ext4 but would be using Btrfs. This has not happened.

Yes there was also since 2009 meant to be no features added to ext4 we have broken one many times over.

xfs is a older design file system than ext file system it had reflink retro fitted. ext3cow prototype demoed that reflink could be retro fitted to ext file systems this include ext4. Its just having the developer will todo it.

MacOS APFS does kind of support reflink. Its a more horrible implementation that you request to have a cow copy of a file made if that cannot be done then it with error and your application has to copy the file yourself. So not quite a linux reflink but close enough the Linux reflink if a cow copy cannot be made a normal copy will happen if that is what is selected yes the Linux kernel reflink syscall has two options
1 to reflink if possible and if not just copy
2 to reflink if possible and error if not.
MacOS APFS cow copies only have option 2 but the cow copies of files on APFS on Macos behave after created exactly like a Linux reflink created files zero difference in operation past that point.

Freebsd does not have reflink equal at this stage. But wine has trouble getting enough people to operate a built farm for freebsd.

Few things to note before we move on. That article is from 2009, meanwhile, google (and not only them, just taking example) invested a lot in EXT4, there is/was a reason why EXT4 is used, but that is not really relevant for this discussion, we can argue why BTRFS or other file systems are not as widely used, and why the argument "It's X year technology" is a poor argument, but all of that doesn't really matter, what matters is that is how things are, aknowledging that fact alone is important when you want to develop some functionality in your application that will depend on something that isn't present on vast majority of the user base of that application.

I'm really not that familiar with MacOS and exact FS that is mostly used on that OS, but if there's such feeature on the majority of MacOS systems, even better.

If there's any valid reason why that isn't implemented in EXT4, I assume there is, but, I don't know even close enough about it to even discuss it.

[oiaohm]

oiaohm All of that makes sense, still, if you have a file system as widely used as EXT4, with users mostly not interested in reflink feature, it's not a smart decision to make application functionality dependable on that feature. I don't really know if there's any valid reason not to include such feature in EXT4, my issue here is, it looks to me like "lazy solution", also, note that other OS's use WINE as well, I'm not sure if they have FS with a reflink feature (such as MacOS for example, whatever FS it uses).

This is missing a few key points still.

https://arstechnica.com/information-technology/2009/04/linux-collaboration-summit-the-kernel-panel/

Despite the fact that Ext4 adds a number of compelling features to the filesystem, T'so doesn't see it as a major step forward. He dismisses it as a rehash of outdated "1970s technology" and describes it as a conservative short-term solution. He believes that the way forward is Oracle's open source Btrfs filesystem, which is designed to deliver significant improvements in scalability, reliability, and ease of management.

Ext4 was a stop gap file system. This is why features like reflink were left out because ext4 was not being designed to last. Yes that T'so was the lead developer of Ext4 yes plan was by now you would not be using ext4 but would be using Btrfs. This has not happened.

Yes there was also since 2009 meant to be no features added to ext4 we have broken one many times over.

xfs is a older design file system than ext file system it had reflink retro fitted. ext3cow prototype demoed that reflink could be retro fitted to ext file systems this include ext4. Its just having the developer will todo it.

MacOS APFS does kind of support reflink. Its a more horrible implementation that you request to have a cow copy of a file made if that cannot be done then it with error and your application has to copy the file yourself. So not quite a linux reflink but close enough the Linux reflink if a cow copy cannot be made a normal copy will happen if that is what is selected yes the Linux kernel reflink syscall has two options
1 to reflink if possible and if not just copy
2 to reflink if possible and error if not.
MacOS APFS cow copies only have option 2 but the cow copies of files on APFS on Macos behave after created exactly like a Linux reflink created files zero difference in operation past that point.

Freebsd does not have reflink equal at this stage. But wine has trouble getting enough people to operate a built farm for freebsd.

[leipero]

oiaohm All of that makes sense, still, if you have a file system as widely used as EXT4, with users mostly not interested in reflink feature, it's not a smart decision to make application functionality dependable on that feature. I don't really know if there's any valid reason not to include such feature in EXT4, my issue here is, it looks to me like "lazy solution", also, note that other OS's use WINE as well, I'm not sure if they have FS with a reflink feature (such as MacOS for example, whatever FS it uses).

[oiaohm]

Thanks for the explanation. Still, WINE is buggy as hell the way it is..., the whole approach in dealing with dll's should be changed IMO, now, maybe an argument that developers are not up to the task is a valid one, however, the solution with reflinks is not really a solution, if they were some rich corporation you wouldn't be wronged to say it's lazyness, but because they do live on donations, it's not so easy to play a "blame game", either way, at some point, you have to wonder about some things.

The reality this is a three to tango. Wine use to use a system that use pe/elf .so files this kept the system provided dll out the wine prefix directory. Wine has moved to built PE files because game anti-cheat and commercial DRM on application(not games) goes and checks what in memory to what is on disc. Yes both anti-cheat and commercial DRM are not smart enough to open the system dll files as read only instead opens the file read/write as a high privilege user yes this is valid todo under windows no matter how stupidly dangerous this is. Yes when a anti-cheat or a drm goes wrong under windows due to having files open read/write that are core files this does result in windows being not boot able. Yes this is some of the cause of why windows fails so much on some users as it the poor quality DRM/anti-cheat destroying the OS because Microsoft allows applications that should not to have write access to core files.. Do note as your normal Windows administrator user you cannot harm the files the DRM/anti-cheat can nicely make blank internally if they wish to(yes reduce to a 0 size file.

Yes this is the first two wine and commercial windows software they have had over 25 year issue here. Yes this is where windows software does horrible things and wine has to do equally horrible things to support them.

Third is the Linux kernel itself particularly the ext developers. Question you never asked your self has there ever been a form of the ext file system that has supported reflink the answer yes. That would be ext3cow that failed to get mainline.

Yes when ext4 was done it was decide that reflink was not required same with copy on write features. Yes it was decided back then that people wanting those features would migrate their Linux installs to btrfs this is why btrfs has a transparent upgrade from ext4 to btrfs. Yes this transparnt upgrade of ext4 to btrfs is done in the same kinds of ways ext3 to ext4 upgrade is done.

Reality here leipero you would not be having a issue with reflink if ext4 had decided to implement relink right instead of making this a forced upgrade to btrfs to get feature. Please remember xfs is older than ext4 and what was done to implement reflink on xfs could be done on ext4.

So we have a 3 way tango here causing the problem. Symlinks and hardlinks are not options for wine in a lot of cases due to the behaviours windows applications demand.

[leipero]

You can consider switching to XFS.

Nowadays it is almost as robust as ext4, while being more modern and supports reflink.

That would be fine, but I do not have a need to change FS, I do not need any of those "modern features", and making application work with specific FS is a very bad idea.

[leipero]

Because most of the file WINE is running didn’t have symlink support in mind.

They assumes that the open give them the file, not a symlink.
After opening the file, they would just update it without replacing that symlink with an actual file.

If you would like to add symlink handling support to prefix, then you would have to patch the interception of syscall/library call in WINE, to detect if the app is trying to open the prefix to be writeable, and update it according.
It’s likely it will introduce more bugs into WINE…

One easy alternative way is to use overlayfs.
It can be run over any filesystem to give it “copy on write” semantics, though generally it copies the whole file on write.
With some more complex runtime setup (need to mount overlayfs for every prefix for each application, requires sudo unless using fuse overlayfs, which have worse performance than in-kernel implementation), you can get it to act like reflink.

The downside, is if you are using in-kernel implementation of overlayfs, then kernel version >= 3.18 is needed and WINE needs to run as root to mount the overlayfs.

If you are using the fuse implementation (userpspace implementation), then you won’t need kernel version >= 3.18 or WINE to run as root, but fuse implementation is going to be slower on opening the file, though reading/writing the file is on par with native filesystem.

Thanks for the explanation. Still, WINE is buggy as hell the way it is..., the whole approach in dealing with dll's should be changed IMO, now, maybe an argument that developers are not up to the task is a valid one, however, the solution with reflinks is not really a solution, if they were some rich corporation you wouldn't be wronged to say it's lazyness, but because they do live on donations, it's not so easy to play a "blame game", either way, at some point, you have to wonder about some things.

[leipero]

This is the point. Instead of having to have a copy of mostly the same DLL's in each prefix, you reflink them. If you had used symlinks and you need to change or add some DLL due to the game or whatnot, then it will overwrite the files shared by other prefixes.

Well, here is the thing, it will write files to that specific prefix, so it will overwrite the symlink not the file itself, basically you should get "broken symlink" (in fact, no symlink because it is overwriten with real file). Once override is removed, symlink is copied from the source replacing the overriden file (automatically removing any extra dll's).

[NobodyXu]

Well, I can't use reflink anyway on EXT4. I don't really understand what you mean by avoiding the risk of mixing up different versions of dll's? If you use dll override in WINE, you are already "mixing versions" because built in WINE solution isn't really any version to begin with, and of you mean for prefixes, well, every prefix uses different location, so as far as I can see, it's a non-issue because "base" is always the same = WINE versions.
If you are talking about multi WINE versioning, that's another thing, but still, I can't see the problem there either, plus, that should be a task of prefix manager such as Lutris or POL etc.

You can consider switching to XFS.

Nowadays it is almost as robust as ext4, while being more modern and supports reflink.