Originally posted by Xake
View Post
Announcement
Collapse
No announcement yet.
~5 Minutes Of Coding Yields A 6%+ Boost To Linux I/O Performance
Collapse
X
-
Originally posted by sophisticles View PostI can't find an article that address this, but years ago i took a class on computer architecture and I remember we were taught that on all systems, regardless of OS, a portion of system ram equal to the amount of vram available is set aside by the OS.
Comment
-
Originally posted by Pajn View Post
Ah, makes sense. I was gonna quote the paper I've read but couldn't quite remember the title, and a quick google search on GC overhead made it obvious it was gonna take a lot of time to find it again without a more of the exact title.
Comment
-
Originally posted by caligula View Post
My Kodi installation only contains a database of movies and TV series. No pictures or music. There are tons of videos though. Also it doesn't use mysql or any real db, I think the db files are built with sqlite.
Kodi uses SQLite, an open source light-weight SQL database-engine, to store all the library, add-on, EPG, textures and PVR related data using a number of specific databases.
And you will have:
* an add-on database telling kodi that is empty because you have no add-ons, but it will still be opened together with Kodi, and closed with Kodi, ready if you want to install an add-on
* The same goes for picture and music, however these three should not pose a larger problem, only adding at most seconds to close time
* "textures" database which contains links to all posters, thumbnails and other graphics shown in you videos/music lists
If you have a large set with many episodes and movies this will be huge
The same goes for you "MyVideo"-database that contains all info about the video files themselves, the scraped info about what the video-file contains, how it is related to other videofiles (movie collections, series), actors/actresses, cross-references for these, watch-lists and more.
Also, SQLite is good at pretty small databases. Large ones? Not so much. That also means that every time you close Kodi it needs to:
* Finish all its current queries
* close the SQLite "connections" correctly, so that all locks and alike things in the SQLite-files is handled
* Sync the files to filesystem
For large databases with large datasets this can take a while, and I am unsure if Kodi does this async or sync (that is closes databases in parallel or one at a time).
Adding to this and related size: Kodi is _NOT_ good at cleaning out the database. This by design to not remove stuff just because you for example lost your NAS-connection.
My database contains references to loads of files removed ages ago, even if the movies does not show up in the lists, because they where cleared away by a "clean library". These references are only removed if I before removing/moving the file from the context menu remove the reference in the library.
For example of this:
Comment
-
Originally posted by Xake View PostFor SQLite that means one file per database.
And you will have:
* an add-on database telling kodi that is empty because you have no add-ons, but it will still be opened together with Kodi, and closed with Kodi, ready if you want to install an add-on
* The same goes for picture and music, however these three should not pose a larger problem, only adding at most seconds to close time
* "textures" database which contains links to all posters, thumbnails and other graphics shown in you videos/music lists
As someone who regularly deals with very resource-intensive processes with lots of open connections and files (up to 16k), I'm telling you: there's no good reason it should take more than a couple seconds to shut down, if that.
I'm sure the bulk of the problem is some garbage code around their threading and synchronization. This stuff is hard to get right, if you don't know what you're doing or have a pretty clear plan from the outset.
- Likes 2
Comment
-
Originally posted by coder View PostAs someone who regularly deals with very resource-intensive processes with lots of open connections and files (up to 16k), I'm telling you: there's no good reason it should take more than a couple seconds to shut down, if that.
As someone working with resource-intensive processes against MySQL-clusters I can say that the finishing and disconnect of the MySQL-server is fast.
Closing mysqld directly after before it has "settled" and synced everything? Not so much.
I actually do not work that much with SQLite, but my limited experience is that it makes the application slower at shutdown then working against a mysql-server, as the application in itself becomes the SQL-server having to handle closing, cleanup of locks and the other stuff that for example MySQL defers til its own shutdown.
I guess you have to do a fast perf of Kodi to actually find what takes time. My instance (running MyVideo/MyMusic on a remote mysql server) takes <5 seconds as long as it does not have a scan in progress it needs to settle.
Comment
-
Originally posted by skeevy420 View Post
[...] My entire KDE desktop will freeze up occasionally when I'm copying RAWs from my camera's SD card to my PC for editing. That happened with my previous system, too. KDE and a high IO load don't always mix.
GNOME has a hard time even reaching 60hz on low end hardware. [...]
Comment
-
Originally posted by caligula View Post
But it's not. You listed the DE, browser + 6 tabs, IM, IDE, terminal, audio player. On my system
- XFCE DE uses around 350 MB (includes also all headless background services, fwiw)
- Firefox - depends on pages, but more than 300 MB per page seems quite a lot
- don't know about Telegram, but Pidgin uses less than 100 MB
- terminals (konsole, gnome-terminal, xfce-terminal etc.) only use few megabytes of RAM
- audio player like Audacious uses less than 100 MB.
I don't use VS Code, but I suspect it's the worst offender here, maybe followed by the browser.
The only thing raising eyebrows to me is FF, but then again.. meh, got plenty of memory anyway.
Comment
-
Originally posted by Xake View PostIs that against a remote mysql-server or local SQLite-file?
Closing mysqld directly after before it has "settled" and synced everything? Not so much.
Originally posted by Xake View Postas the application in itself becomes the SQL-server having to handle closing, cleanup of locks and the other stuff that for example MySQL defers til its own shutdown.
Originally posted by Xake View PostI guess you have to do a fast perf of Kodi to actually find what takes time.
- Likes 2
Comment
-
Originally posted by coder View PostYeah, probably because it has lots of pending writes and other bookkeeping to update. Unike this Kodi example we're discussing, which should be mostly just reading its databases.
Originally posted by coder View PostClosing files and deleting locks are both trivial operations.
Originally posted by coder View PostI'd try getting a stack dump immediately after telling it to exit. Then, look at what all the threads are doing. I'll bet you'll either find deadlocks or some background threads/processes with long polling intervals.
Me myself has a load of videos os well, and my old laptop with a i7-5500U, 8gb memory and an old SATA-SSD with webbrowser and many tabs open in the background having MyVideos and MyMusic in a remote MySQL reached over n-wifi take about 3 seconds to exit. It takes shorter time on the HTPC which has even more oddities but newer CPU. So if yours takes longer something seems odd with your installation or setup.
Comment
Comment