New readfile() System Call Under Review For Reading Small~Medium Files Faster

[markg85]

I hope you do realize that there exist far better [research] methods of how to increase performance of kernel-userspace transitions, by a factor of 10 at least compared to what Linux is currently doing.

Enlighten me
As far as i know, io_uring is the best there is going to be. And it won't even come close to a 10x improvement (more like 0.2x) the main difference being fully async in nature.
So i'm really curious to know your 10x claim. Do back it up by an article that shows the performance supremacy.

[markg85]

Just for fun.
I did an strace -c on a folder opened in dolphin. The folder only contains 2048 jpg files (wallpaper sizes of full hd resolution)

Without thumbnail presentation (they are generated before this run)
28114 syscalls!
Look here for the full output: https://p.sc2.nl/TWKfB

With thumbnail presentation (note that dolphin does that for only those that are in your view, so this is even optimized)
39984 syscalls!
Look here for the full output: https://p.sc2.nl/un8r5

Keep in mind that this is Dolphin. Thus it uses KIO. That means you don't see the actual opening of the thumbnail files in strace as that's done in IO slave handlers in other processes. Having said that, now i'm REALLY curious what those statx calls, read and writes do and where they come from. It's also interesting to see so many errors! I think i have to investigate a bit there to figure out what's going on.

Note the high time percentage of futex too. There's room for improvements i think. But does it matter? This many stat calls and dolphin still starts in mere milliseconds and you're off browsing the files. In other terms, reducing stat calls here to the bare minimum is likely not going to result in any perceived speedup. But it will likely result in the CPU being done (much) faster thus for energy savings this is beneficial. If it's worth it is another matter entirely

[markg85]

Ryzen 3000 desktop. Mitigations disabled.

then your results are worth nothing, sorry
While i agree with disabling mitigations. Everyone should do that! The security paranoia went too far on that front. The super majority will just update their system and don't bother adding boot flags to disable them. Distributions also likely won't add them..

You're also comparing roughly the slowest medium for these days (spinning drives). Lots of people nowadays have an NVMe as their daily driver. That will significantly change your results. Still a syscall is likely going to be a very minor part of the whole operation that was executed. So for one call that won't give any notable performance difference i suppose. But things might change if you multiply it by tens of thousands.. Remember, a simple action like showing a thumbnail probably executes a handful of syscalls now (stat it, read, open, close, extended attributes, ....).

[coder]

Just a note: Reducing 3 syscalls into 1 most likely won't increase uncached IOPS: HDD has about 100 random IOPS, SSD is about 10000 random IOPS (except for 3D XPoint), a getpid() syscall is about 100 nanoseconds and 1/100ns = 10 million no-operation "IOPS". 10e6 is 1000 times smaller than 10e3.

Good point.

On what CPU were those measurements taken? Did you have any mitigations disabled? Not that it would change the overall impact on IOPS (except perhaps with Optane), but it'd be good to know.

[markg85]

There is no size limit, the mentioning of "small and medium" is because for smaller files the overhead of 3 syscalls is way larger than the actual time it takes to read the contents of the file and that is where this new syscall gives a performance boost, if it's convenience you are after then it's a boost regardless of file size.

Ohh, that changes things! Thank you for that reply!
That means that any file you'd previously read into memory could now use this syscall. Or rather, should! As it only give you a net win.

Now i kinda hope that toolkits (say Qt/GTK) are going to provide support for this too!

Those places that already did open/read themselves with smarter memory usage (like streaming) should just stay as is i think?

[nanonyme]

There is no size limit, the mentioning of "small and medium" is because for smaller files the overhead of 3 syscalls is way larger than the actual time it takes to read the contents of the file and that is where this new syscall gives a performance boost, if it's convenience you are after then it's a boost regardless of file size.

Also there's the thing that you probably don't want to use this for gigabyte-sized files anyway as you might want to handle those in streamed way rather than reading everything into memory.

[F.Ultra]

Can anyone tell me the maximum filesize for this syscall?
Yes, i did read the code, and the new manpage. Nothing seems to indicate what qualifies as "small". The post also mentions "small and medium sizes".

It even has testcases:

+ test_filesize(0x10);
+ test_filesize(0x100);
+ test_filesize(0x1000);
+ test_filesize(0x10000);
+ test_filesize(0x100000);
+ test_filesize(0x1000000);

Where that largest testcase is 16 MiB.

I'm asking because i want to know if this syscall can be used to, for example, load icons and config files. Now config files are likely sub megabyte ones. But icons can be multiple megabytes (think about ico files that contain multiple images). But besides that, think about reading thumbnails for display in, say, dolphin or gwenview etc...

Now i'm not betting on this to be "substantially faster" then 3 syscalls (open/read/close) as the syscalls themselves are probably just a tiny percentage of actually handling the file data to do something with it (say jpeg/png/svg/webp decoder). But still, if you expand this over large folders it might become a notable difference.

Lastly, and this one is specifically for phoronix. Greg is, in the next version of this patch, going to post benchmarks too: https://lore.kernel.org/lkml/[email protected]/

There is no size limit, the mentioning of "small and medium" is because for smaller files the overhead of 3 syscalls is way larger than the actual time it takes to read the contents of the file and that is where this new syscall gives a performance boost, if it's convenience you are after then it's a boost regardless of file size.

[coder]

Without the ability to read multiple small files using a single system call, it is impossible to increase IOPS (unless using multiple threads).

IOPS is Io Operations Per Second. You can increase them either by overlapping more operations or decreasing the time between successive operations. This addresses only the latter, though still improves performance (which, again, is their reason for doing it)

If you also want the former, then you can simply use it with io_uring to queue up multiple readfile() operations.

So, you can have your cake and eat it!

[cl333r]

Is there a real-world scenario in which open/read/close of sysfs and procfs files is a bottleneck?

Yes, there is.

[jaskij]

markg85 nowadays I would. But I was a noob who got thrown into a middle of a project. The hardware was done, the client had stupid requirements and I was a noob trying to make the best out of it.

These days I'd probably tell them to just slap a hardware PWM or an MCU in there. But if you're doing small runs anything software-defined is usually not worth the added R&D costs if you can pay extra 5$/device in hardware.

Also, I stand by my position that for cheap stuff with a 500 MHz Cortex-A7 I'll take all the savings I can get.

P.S.
Sorry for not quoting, Phoronix forums are iffy on mobile.