Microsoft Still Working To Squeeze More I/O Performance Out Of WSL / Bash For Windows

[F.Ultra]

Don't forget that Windows is also case-insensitive, which is quite a pain to do for Unicode, not to mention NTFS can store two such files case-sensitively and so conflicts still exist (it's undefined which one gets picked by Windows). I guess looking up case-insensitively is a lot more work if you do millions of open() file operations (case sensitive on the other hand, like Linux, requires zero extra processing).

What I'm saying is, I doubt they (Microsoft) can make it (native NTFS, not WSL) much faster, and I doubt WSL will ever be faster than native (unless it directly uses special kernel calls, but I seriously doubt it, it's probably just a wrapper so it cannot be faster). Unless, unless, they wrote some shit unoptimized code in there, but I really doubt it after so many years.

It was a bad design from the beginning, and I heard macOS is also case-insensitive, except HFS+ is much, much worse of a mess than NTFS is. Always makes me facepalm. It's one thing where Unix and Linux got it right completely, full respect.

Couldn't agree more. However also raw file transfer is slower so the problem goes deeper than just lookups. As you say however open() is dog slow on Windows, I'm sure you also have run your fair share of automake scripts in msys2 and wanted to throw the machine out a window in frustration (e.g configuring and compiling OpenSSL on Windows is not recommended for people without patience).

[F.Ultra]

They really need to find a fix for directory traversal as well. "/path/to/softlink-dir/../" goes to the parent of the directory that softlink-dir points to, rather than to the parent directory of the softlink-dir. Has to do with how underlying NTFS deals with directory junctions.

WTF? Why do they even enter NTFS at that stage, cleaning paths like that to "/path/to/" should be done in userspace long before the filesystem is even touched.

[duby229]

I doubt that context switching overhead would be so great that it could explain such a big performance penalty. I think that it's a combination of NTFS being just slow (I mean the internal structure of NTFS is way over engineered with multiple stream support and so forth) and that Windows kind of does something to keep the IO subsystem busy 24x7.

Now to be honest it was over 10 years since I saw a rack mounted Windows server, but back then when you walked into a server room at night when the servers where idling the activity leds for disk and network on the Linux and BSD servers where dark while the Windows machines light up like it was Christmas. If I'm not mistaken Windows have a tendency to prefer to put memory of running applications to swap as soon as possible which also would stress the IO subsystem more.

It's exactly context switching. Just profile an IO benchmark or any IO sensitive load you know of. There you will see the pipeline flushing and refilling repeatedly and meanwhile it'll be doing no real processing at all. I'm serious, you really should profile it.

EDIT: Linux has an actual real scheduler, Windows does -not-. Linux prioritizes IO and guarantees that priority, Windows simply just allocates IO into time slices and then kinda like hands out those slices with a context switch in between ever single one of them. On top of them if the IO process isn't ready in time for the time slice then it issues wait cycles until it is, which is just a serial string of context switches, and that also means Windows can't guarantee IO priority... (It's why Linux has the concept of "niceness" which it guarantees and Windows doesn't, and also the reason why IO makes Windows feel unresponsive, and also the reason why you see these performance differences between native Linux and WSL)

[oleid]

Oh god I wish there was a "hate" button too....

no worries, i merely forgot my irony tags.

[Zan Lynx]

Yeah no shit, Windows is slow as fuck at I/O operations compared to Linux & their Download speeds are utter shit too vs Linux.

I have seen slower I/O operations in Windows, but download speeds? Maybe in Windows XP before service pack 2?

But since then Microsoft gave Windows TCP/IP window scaling and SACK support, and some pretty decent auto-tuning. I haven't seen any problems downloading. My Windows 10 machines get the full 180 Mbps off the cable modem and 980 Mbps off the LAN from my Linux file server.

And from friends of mine who work with Windows, I know that Windows supports some very slick high speed RDMA / SMB Direct file sharing interfaces. For Hyper-V machines that makes a network share perform close to local NVMe.

[F.Ultra]

It's exactly context switching. Just profile an IO benchmark or any IO sensitive load you know of. There you will see the pipeline flushing and refilling repeatedly and meanwhile it'll be doing no real processing at all. I'm serious, you really should profile it.

EDIT: Linux has an actual real scheduler, Windows does -not-. Linux prioritizes IO and guarantees that priority, Windows simply just allocates IO into time slices and then kinda like hands out those slices with a context switch in between ever single one of them. On top of them if the IO process isn't ready in time for the time slice then it issues wait cycles until it is, which is just a serial string of context switches, and that also means Windows can't guarantee IO priority... (It's why Linux has the concept of "niceness" which it guarantees and Windows doesn't, and also the reason why IO makes Windows feel unresponsive, and also the reason why you see these performance differences between native Linux and WSL)

Ah, yes if they divide up IO into such slices then I can see how context switches would come into play.

[brrrrttttt]

WTF? Why do they even enter NTFS at that stage, cleaning paths like that to "/path/to/" should be done in userspace long before the filesystem is even touched.

Junctions make it much more complicated than that.

[linuxgeex]

Oh god I wish there was a "hate" button too....

Or at least a giggle button.