~5 Minutes Of Coding Yields A 6%+ Boost To Linux I/O Performance

[rene]

Clickbait. Even Jens Axboe needed longer for that. More like 25 minutes plus performance testing. So more like an hour or two ;-)!

[caligula]

Please, give a concrete example of such awful and unjustifiable resource consumption with an easy mitigation. I'll wait.

And no, just deleting functionality does not count.

This is not KDE or Gnome, but an excellent example is Kodi. Just fucking closing the application may take one minute. Yes the system has 5950X, 64 gigs of RAM, M.2 SSD with ~7 GB/s I/O throughput. It's a vanilla install, no extra plugins.

Another great example is Kdenlive. The more you have clips, the slower it becomes (import & export). Try it. The read speed of the project file is few kilobytes per second. Yes, it can take two minutes to read a project with total video content spanning 60 minutes. Yes, again using the same "workstation" as above.

I'd also want to know why KDE or Gnome uses 1,1 GB of RAM after launching. I have more than 8 GB of video RAM. Does it store the video buffer in system RAM? For example Gnome doesn't seem to be running that many heavy background services. Where does it all go?

[coder]

This is not KDE or Gnome, but an excellent example is Kodi. Just fucking closing the application may take one minute. Yes the system has 5950X, 64 gigs of RAM, M.2 SSD with ~7 GB/s I/O throughput. It's a vanilla install, no extra plugins.

Sounds like there are probably some polling loops and other bad design practices lurking in that app. There's absolutely no good reason an app can't quit almost instantly.

Another great example is Kdenlive. The more you have clips, the slower it becomes (import & export). Try it. The read speed of the project file is few kilobytes per second. Yes, it can take two minutes to read a project with total video content spanning 60 minutes. Yes, again using the same "workstation" as above.

Sounds like more bad userspace code. I wouldn't blame the OS, unless/until I saw evidence it wasn't the app's fault.

There's just a lot of sloppy code in the world. I've seen more than my share. Overwhelmingly, it's in the app and not the OS, compiler, or runtime libraries.

[caligula]

KMail is such an example. To start it one have to start whole akonadi with all the services (not only e-mail) enabled. So to read a simple e-mail on my machine I would have to give that resource hog like 800 MB of RAM from the beginning.

For reference, starting Claws Mail (non KDE app in KDE DE) needs like 20 MB of RAM from the beginning.

Claws mail is actually one of those lightweight apps still available. Unfortunately many think they're obsolete. Kmail used to be quite light during the KDE 3.5.x era. All of that metadata index crap is just adding bloat.

[Nth_man]

CPU:AMD Ryzen 7 3700X
GPU: AMD RX 560
RAM: 32 GB in dual-channel, 3200 MHz, forgot which brand, probably Corsair
SSD: M.2 Sandisk 3.2Gen1 taken from a laptop, which seems to not work in NVME mode, but in SATA mode as benchmarks don't seem to get over 600 MB/s.

I think that the bottleneck is the storage, but is still a SSD, even though it runs just in SATA mode and FFS, should not matter that much.
I mean with 32 GB of RAM you can put the entire DE and whatever dependencies and other crapware that it needs totally in RAM and access the storage only if really needed and even in that case, do it in async mode, without blocking everything because you wait for it.
I don't know what the fuck these so called modern DEs do, but they are not modern at all and I don't give a crap about using the least amount of RAM when I have so much and the whole DE and Firefox or whatever program I have open stutters or freezes because of some background activity.

Ok. That doesn't happen to me since I bought a new computer (using Kubuntu 23.10), although sometimes it happened to me using the old computer (under Kubuntu 23.04), so in that PC I started using the `schedtool -D -n 19 -e ionice -c 3` way that RealNC wrote about (upvoted 25 times at least) on https://www.phoronix.com/forums/foru...76#post1377776

(we can also see https://www.mail-archive.com/gentoo-...msg181789.html)

[caligula]

I got 32GiB RAM on Fedora 39 amd64, Plasma 5, open apps - Firefox (6 tabs), Telegram, Visual Studio Code, terminal, audio player - only 5.3GiB are used.
Seems pretty good to me.

But it's not. You listed the DE, browser + 6 tabs, IM, IDE, terminal, audio player. On my system
- XFCE DE uses around 350 MB (includes also all headless background services, fwiw)
- Firefox - depends on pages, but more than 300 MB per page seems quite a lot
- don't know about Telegram, but Pidgin uses less than 100 MB
- terminals (konsole, gnome-terminal, xfce-terminal etc.) only use few megabytes of RAM
- audio player like Audacious uses less than 100 MB.

I don't use VS Code, but I suspect it's the worst offender here, maybe followed by the browser.

[sophisticles]

Yes, there are cases where the performance of certain block devices sucks. I expect where DE's run afoul of this is by not sufficiently decoupling the IO from the UI threads. That's probably because it's not a problem for most devices, but if you use one of the bad ones, you feel it.

It doesn't help to over-generalize or catastrophize, as you seem to enjoy doing.

It's not that i enjoy it or that i am trolling, it annoys me that people act like there are no problems with Linux use and that the things i experience and see with my own eyes are not actual issues.

I'll give you a perfect example, I am a firm believer in multiple backups, I have had too many instances over the years of devices going bad and the backup being inaccessible so that now I usually have 3 backups.

Assume you have an encrypted external drive with ext4/LUKS, 1tb, lots of video, pdf, doc, jpg, python scripts, excel sheets, and mp3.

You want to make a backup to a different encrypted external drive with ext4/LUKS.

Try this on a distro, Ubuntu, Fedora, any of them, that is running Mate.

You start the copy from one drive to another and the GUI is now frozen, nothing works. If you wait long enough, typically hours, eventually the GUI unfreezes after the copy has finished but in the meantime the system is unusable.

KDE/Plasma is much better in this regard, there is some performance impact but at least the GUi is nit completely frozen, but still not something you really want to do.

Win 10/11 with BitLocker are way better, they stay responsive and you can keep working.

This is why I use my guinea pig system for backups.

In all fairness I think you are probably correct, I do not think this is a Linux kernel issue, as evidenced by the fact that different DE experience different levels of performance degradation and that in a headless server deployment these issues do not seem to pop up.

[sophisticles]

I'd also want to know why KDE or Gnome uses 1,1 GB of RAM after launching. I have more than 8 GB of video RAM. Does it store the video buffer in system RAM? For example Gnome doesn't seem to be running that many heavy background services. Where does it all go?

I can't find an article that address this, but years ago i took a class on computer architecture and I remember we were taught that on all systems, regardless of OS, a portion of system ram equal to the amount of vram available is set aside by the OS.

The reason for this was because back then GPU's were not capable of reading and writing to and from storage devices and data needed to be copied to and from system memory and vram.

This was one of the major complaints about CUDA when it first came out, that the performance gains of running code on the GPU were being eaten up by the need to copy to and from system memory.

This is no longer true, Nvidia introduced the ability for a GPU to access a storage device such as an SSD directly and avoid the costly back and forth between system ram and vram.

I do believe however that OSes set aside a portion of system ram proportional to vram, so the more vram you have the more system ram is set aside.

It does beg the question of what would happen if someone decided to do something crazy and put a card with 24gb vram into a system with 4gb system ram?

[sophisticles]

There's just a lot of sloppy code in the world. I've seen more than my share. Overwhelmingly, it's in the app and not the OS, compiler, or runtime libraries.

This may be the case but when the apps are open source designed to run on one specific OS, Linux in this case, it gives the OS a bad name because people are experiencing it while using that OS.

The same in the Windows or Mac Os world, if it's Windows or Mac OS specific apps that behave a certain way, MS and Apple get the blame.

The fish stinks from the head, if a team is having a horrible season who gets blamed? The QB, the coach, ownership, if the economy sucks the President gets the blame, if the economy is good the President gets the credit, everyone ignores that's a very complicated system with numerous parts.

It's always the CEO sucks, the President suck, the coach sucks, the owner sucks, the leader sucks, it's just the way it is.

[Xake]

This is not KDE or Gnome, but an excellent example is Kodi. Just fucking closing the application may take one minute. Yes the system has 5950X, 64 gigs of RAM, M.2 SSD with ~7 GB/s I/O throughput. It's a vanilla install, no extra plugins.

Kodi does that for me, but more or less only if there is a background scan going on that looks for metadata of new files. Otherwise it still takes a couple of seconds, but not to bad.
I guess it also needs to close all those database connections/files. Because it has lots of databases. One for video, one for pictures, one for music, one for textures, one for.... yeah, it goes on.
Normally an application has one database with loads of tables, which means fewer files/connections to sync/close at the end.

But this behavior is also true for Kodi on windows last time I used it, so I guess that is not Linux-specific.