EXT4 Getting Faster Case-Insensitive Performance

[starshipeleven]

Invalid point. Nobody forces you to use it or enable it on the directory.

I'm arguing against a guy that wants it enabled across the board because he can't deal with case-insensitive file search or something in his own applications.

[oiaohm]

Invalid point. Nobody forces you to use it or enable it on the directory.

You want a simple argument for it? Wine performance.

It not just for wine. NFS, Samba and Wine have usage cases for case insensitive. Problem is all three also have usage cases for case sensitive.

So really it a case we need both. One or the other does not cut it.

There has been a lot of history stubbornness over case insensitive and case sensitive with the stupid logic that you have to choose 1 or the other absolutely when correct answer is create a system that provide both solutions.

Both windows 10 and Linux are going the per folder route for choosing if you are case insensitive or case sensitive it seams like the right option.

[starshipeleven]

Not true.

Still true, even if it's "more efficient" for your own usecase it's still not as efficient as case-sensitive and applications that don't need that (the vast majority) are getting hit for no reason.

While it can be done in a couple of lines, the actual code for this is at least 10 times bigger than the normal "file open", and prone to errors.

If you are rewriting the same code over and over from scratch it's your own issue. It's a long-solved problem.

The real problem is that such a code is extremely slow.

I didn't notice that on any GUI or CLI application doing it.

Only applications actually having issues with that are Samba and Wine stuff. But there is no reason to do that with a new application.

I didn't say that. Nothing even close. I am arguing that "case-insensitive" handling is better. I didn't say anything about forcing it on anybody. The solution with per-directory preference is quite acceptable to me.

You aren't making much sense then, is your application requiring the user to set this filesystem/folder option on all folders he keeps his documents and stuff just because your application needs to find his stuff? That's very convenient for end-user experience.

Reminds me of some shit Windows applications that require the user to set Windows system folder to world writable to work at all.

1. Not true for Wine and similar apps.

That's legacy unmaintainable crap. (the applications you run with Wine, not Wine itself) You can just work around it and it's fine.

2. Irrelevant argument. We are arguing what's better, not whether there will be an immediate performance impact.

Performance impact is part of the assessment about what is "better".

No, you are not.

Yes I am, but you are just choosing to ignore what does not fit your worldview.

Of course you never said it. You just implied it.

More of the above, you are seeing only what you want to see.

I implied that if all current applications can do it you can do it too for your specific usecase and it will be fine.

[oiaohm]

1. FS Driver has additional information and privileges that user-space apps and libraries don't have, therefore enabling for a more efficient implementation.

Not the cause of the advantage. The cause of the performance advantage to doing it kernel driver side is filename hashing. File names are first searched for by hash.

Filling this hash table with casefolded make a case-insensitive search more effective at the file system level and the hash table is shared between processes so does not have to be started every time and some file systems allow you to save the hash table in the file system metadata between runs.

Remember casefolding the look-up request is not free so case-insensitive always has more overhead than case-sensitive.

2. It is easier to turn a case-insensitive file search into a case-sensitive than vice-versa.

Also not true. Both kill you if you attempt todo it from userspace.

https://lwn.net/ml/linux-fsdevel/[email protected]/

About time you look at the patches. To turn case-insensitive file search into a case-sensitive you basically have to reverse the above patch.

The problem is the filename lookup hash what is is filled with.

A pure case-sensitive search is always faster than a case folded case-insensitive search. Turning a case-insensitive search into a case-sensitive one adds extra processing.

Basically you are better to start with a case-sensitive search and extend it to include case-insensitive than attempt to convert a case-insensitive search into a case-sensitive one.

Please also note a directory really cannot be both case-sensitive and case-insensitive at the same time as this leads to conflict issues on what is and is not allowed.

[carewolf]

Another guy seeing thigs that are only in his own mind.

This is a filesystem feature that is needed by one of the possible usecases. Like Samba shares, or Wine application folders.

It's not on by default and it's specifically designed to be enabled on a folder basis.

It isn't needed. It could be extra API.

Ironically Windows went the other way. They have have Linux APIs that makes it possible to store files that otherwise would name clash with normal Windows APIs.. NTFS have always been able to handle both, it is just different APIs.

[starshipeleven]

It isn't needed. It could be extra API.

Afaik there is no real requirement for an extra API (beyond internet trolls posing as software developers). This is just a quick workaround for some problematic applications, that's it.

[oiaohm]

Case-sensitive comparison is equally fast as case insensitive (more precisely, the difference is negligible when used in file-system operations.

https://lwn.net/ml/linux-fsdevel/201...collabora.com/
No case-sensitive comparison at file system driver level is in fact slower than case-sensitive you have a longer code path. If what you are doing is like a Linux kernel build having the file system folder where it is as case insensitive does hurt by taking longer.

It is not fine (to emulate case-insensitive file-open by retrieving a list of all files in a directory) because:
- it is slower
- it requires extra code and testing

This is true at kernel level as well. Just not as much as processing full list in userspace.

- requires code duplication

Kernel level avoid code duplication this is the advantage of doing in kernel space is you don't have to duplicate up the hash table and this table can be cached between runs.

- requires extra error checking, memory allocation and other things which experts know very well about, but amateur's don't
- requires extra case handling (when two synonymous files exist in the same dir).

These is true at kernel level as well. Casefold at kernel level requires extra memory allocation and extra checking.
Bugs in file system means two synonymous files may exist so true at kernel level.

- etc, etc...

You did not mention these. I would suspect most of them effect kernel level as well. There is very little difference in issues between implementing in kernel space and implementing in userspace. Duplicate processing issue/Duplicated code is the fairly much the total performance hit difference between the user space and kernel implementation. All the other performance/memory hits from being case insensitive other than the Duplicate processing issue/Duplicated code stay the same when you move the processing of case insensitive from user space to kernel space.

In short, it is not fine because it is not an elegant solution. An elegant solution is to provide this functionality on the kernel level.

Yes it might be more elegant for the kernel to provide case insensitive over userspace. But case insensitive is not a free no matter where you implement it. Cheapest in performance cost is always case sensitive.

The advantage of implementing case insensitive in kernel space is avoiding some duplication and race condition problems. The avoid some duplication increases performance but it does not change the fact that case insensitive processing is still slower than case sensitive processing even when in kernel space.

Case insensitive search is always a longer code path than a case sensitive one. The reason why Linux kernel developers resisted adding case insensitive support for so long is the fact it adds extra code into file system layer to be debugged and harms performance. Of course those emulating case insensitive in user-space being wine and samba were being hit worse.