EXT4 Case Insensitive Support Sent In For The Linux 5.2 Kernel

[k1e0x]

I guess this is good. Please god Linux don't make this the default (Looking at you Redhat.. I see you over there.. stop it.)

Also ZFS can already do this, so your windows datasets or wine games can just be setup case insensitive now.

[xorbe]

I don't know how I feel about this. Probably SystemD will suddenly depend on it one day, and force change all distros.

[k1e0x]

I don't know how I feel about this. Probably SystemD will suddenly depend on it one day, and force change all distros.

Haha, nice.

[Grogan]

Like other people have said in previous threads about this subject, it's absolutely lovely how some proponents of the "pro choice" Linux movement seem to absolutely despise when those choices aren't made by them. Is it opt-in? Yes. Does it hamper performance and/or security? Nope. So please, do us all a favor and be silent, guys. That said, I'm curious as to why exactly they're not happy about this support. Is it something more serious than "we don't want M$ crap polluting our sacred kernel"?

Well... it's not as simple as you make it out to be. Do you have any idea the invasive changes to EXT4 that have to take place to accommodate this? They essentially have to add language encodings to the filesystem now because there is no such thing as "case" at that level. Only byte values interpreted in user space as characters.

Do you know what's going to happen when this is enabled in the superblock and the filesystem is mounted with an older kernel? You might surmise that they'll be smart about that, but have you tested the implementation?

So unless you have used this, benchmarked it and audited the code, you really have no business admonishing anyone else to be silent. It is changing long existing fundamentals.

[rene]

The simple answer is that it cannot just be at the VFS layer. All the issues wine runs into implementing different userspace solutions gives you what you will hit implementing at the VFS layer.

https://wiki.winehq.org/Case_Insensitive_Filenames

"Using a case-insensitive FUSE filesystem" here is the same as implementing at the VFS level. Wine project has tried stacks of different things to solve this beast.

VFS has been modified to a point by this process so that case sensitivity can be detected though it but the case sensitive/instinctive has to be file system.

Performing case insensitive that will not require doing horrible cpu consuming tasks or hit impossible issues need to be a file system level. Think this so that you cannot end up with Test and tEST in a case insensitive directory. Think how do you decide what one to give to the application wine third party developers tried all kinds of things with fuse to attempt to solve this only to find out there is no correct solution.

Yes there might appear to be many different solutions but when you implement them you can see all bar 1 fall in a heap of fail. Wine and Samba have tried the lot.

Results of Samba/Wine Case Insensitive solutions
1) Userspace solution falls in a heap of I cannot solve this and performance killing fail.
2) Fuse base solutions fall into a heap of I cannot solve this and performance killing fail.
3) VFS base solutions would fall into a heap of I cannot solve this and performance killing fail.
4) File system level works ideal with some alterations to VFS so you can find out what directories are and are not case insensitive..

Really I would like to hear if you know of some solution that is not covered by what Samba/Wine have tried as I am fairly sure they tried everything.

Please note point 4 File system works Vfat driver that does like fat32 has been doing it for ages and not hitting I cannot solve this or performance killing.

for what does someone even want case insensitive, except running this stone-age WinDOS applications. And why do we implement this un-feature in 2019?

[slalomsk8er]

I'm curious as to why exactly they're not happy about this support. Is it something more serious than "we don't want M$ crap polluting our sacred kernel"?

I can tell you why I don't like the concept of case insensitive file systems:

If you port a LAMP stack to WAMP ist just works (as long as the PHP versions are the same) - from WAMP to LAMP you need to fix the wrong cases in the DB strings and the PHP includes

With this new feature I don't need to fix this crap any more - same with the case insensitive option in MySQL/MariaDB but I don't like the sloppiness of wrong cases and I bet performance takes a hit.

The fear is, that this encourages the same sloppiness, that happens on Windows, under Linux and that the existing sloppy code no longer gets fixed.
The first point is invalid as it is not on by default but the second one even tempts me

[oiaohm]

for what does someone even want case insensitive, except running this stone-age WinDOS applications. And why do we implement this un-feature in 2019?

Its not a un-feature. Its need because of a unicode disaster and some users will be very happy to have it. This will normally be people who are skilled in multi languages particularly the ones with horrible overlapped keyboard maps in appearance but not unicode values behind keys.
.

https://linuxplumbersconf.org/event/2/contributions/242/attachments/140/173/case-insensitive-lookups.pdf

On this presentation you got to page 3 the following.

15631222 -rw-r--r-- 1 root root 0 Nov 1 01:47 coração
15631218 -rw-r--r-- 1 root root 0 Nov 1 01:46 coração
15631217 -rw-r--r-- 1 root root 0 Nov 1 01:46 coração
15631215 -rw-r--r-- 1 root root 0 Nov 1 01:46 coração

You can in fact end up in this mess by changing you keyboard language. Because we have chars that look basically the same with different unicode numbers.

The feature is casefold so its not just plain case insensitive. Casefold include making chars on different keyboard layout that look basically the same successfully match instead of being unique unicode values as well.

For some users this will save a hell load of hair pulling.

ftp://ftp.unicode.org/Public/UNIDATA/CaseFolding.txt
Casefold is part of the Unicode standard because they worked out they had a major mess as well.

Us in english when we here case insensitive we are thinking of just upper and lower case. Unicode Casefold is more diverse its not just upper and lower case its sideways across languages keyboard maps so the example would not happen. Yes it would be possible to Casefold without doing the upper and lower case but the implementation does not include that option at this stage.

[rene]

Its not a un-feature. Its need because of a unicode disaster and some users will be very happy to have it. This will normally be people who are skilled in multi languages particularly the ones with horrible overlapped keyboard maps in appearance but not unicode values behind keys.
.

https://linuxplumbersconf.org/event/2/contributions/242/attachments/140/173/case-insensitive-lookups.pdf

On this presentation you got to page 3 the following.



You can in fact end up in this mess by changing you keyboard language. Because we have chars that look basically the same with different unicode numbers.

The feature is casefold so its not just plain case insensitive. Casefold include making chars on different keyboard layout that look basically the same successfully match instead of being unique unicode values as well.

For some users this will save a hell load of hair pulling.

ftp://ftp.unicode.org/Public/UNIDATA/CaseFolding.txt
Casefold is part of the Unicode standard because they worked out they had a major mess as well.

Us in english when we here case insensitive we are thinking of just upper and lower case. Unicode Casefold is more diverse its not just upper and lower case its sideways across languages keyboard maps so the example would not happen. Yes it would be possible to Casefold without doing the upper and lower case but the implementation does not include that option at this stage.

well, while unicode folding makes a little more sense, I hope this does not lead to complexity like TrueType font parsing CVE's on the Windows kernel side. Maybe sometimes it is better to just keep things stupid simple, instead of over engineering solutions for maybe not that important "problems".

[oiaohm]

well, while unicode folding makes a little more sense, I hope this does not lead to complexity like TrueType font parsing CVE's on the Windows kernel side. Maybe sometimes it is better to just keep things stupid simple, instead of over engineering solutions for maybe not that important "problems".

Please note they tried to solve this problem for over 10 years without needing to be in kernel space. And every attempt failed some appears to work for a while.

If you are talking stupid simple. Stupid simple way of doing casefold that works is implement in the filesystem so that you make conversion once and search for a file match and not run into conflicts. Turns out any other way is more complex leading to race conditions and conflict events.

Like due to code quality things this could cause I don't exactly like this casefold but there is no other way to have the feature.

Android attempts to use a fuse file system to give the feature and has been jammed exploitable security and stability faults caused by the conflict events in file hosting and multi pass processing(eating the cpu). Yes the android fuse option was looking like success until it failed and they worked out there is no way to properly fix it as a layer above file system.

Samba servers have had DOS attacks because the casefold system ate server out of ram and cpu time.
So we already have CVE from avoiding doing casefold in the file system drivers.

Wine has had complete programs die because of casefold issues.

The reality is if you don't want casefold in kernel space only option is go microkernel and have the complete file system driver userspace. This is just the way the issue is.

I do wish there was another solution to these problems. Willing to look into any ideas but I really do suspect that any suggest solution other than make it part of the file system driver has been tried and is a failure. Sometimes a really horrible failure causing DOS attacks and other security issues.

[rene]

Please note they tried to solve this problem for over 10 years without needing to be in kernel space. And every attempt failed some appears to work for a while.

With keep things stupid simple (in kernel space) I mean not to try to solve this problem at all. IMHO it does not sound like that much of a deal to create this esoteric Unicode surrogates or whatever in filenames. Dealing with international files and email attachments, with Chinese, Russian e.g. unicode glyphs I experienced such a problem zero times so far. And even if I had I would probably not care at all for this in-kernel complexity. Of course I realize my 2¢ comment will not stop anyone from further innovating this feature into the kernel space ;-) Oh, wait, did you say micro kernel? ;-) https://www.youtube.com/watch?v=g85yri1kfJo