Endless OS Switching To The BFQ I/O Scheduler For More Responsive Linux Desktop

[NotMine999]

From the article:

I think you're benchmarking speed while I/O schedulers for us are all about responsiveness. Yesterday when I was doing a file copy operation in the background the youtube videos in Firefox almost stopped playing (Core i5 - 4 cores, 6GB memory, GT 1030 2GB, 2TB HDD) - that's pathetic for a desktop OS! I don't expect BFQ to give me *speed*, in fact I expect it to slow things down a bit but it's a great tradeoff.

Your post lacks a certain amount of detail regarding the amount of data that you are trying to copy. At least you noted that you are using a HDD and that could be some of the issue. Or did you mean SSD? So your issue sounds like it probably a "local problem".

Is the HDD even operating SATA1 speeds? SATA2 speeds? SATA3 speeds? Some drives and controllers will "downshift" if they encounter too many errors on the SATA link. A quick look at "dmesg" output will reveal if the drive/controller are downshifting the SATA speeds.

Does the HDD have any bad sectors? What does "smartctl" report?

Perhaps the HDD is a 5400 or 5900 rpm device.

Perhaps the HDD has small onboard buffers. Drives with smaller onboard buffers can have issues writing out data at full speed when other applications are trying to read data from the same HDD at the same time.

Huh?

Perhaps Youtube uses your local storage to buffer it's content while you watch, especially if the content is not live? Think about it. Your PC can download the content faster than it is being played back; downloads can be 10s of megabits or more per second while playback might be a few megabits per second even with 4k video from Youtube. Youtube can pull down the video to your local storage and play it back from there 9rather than pulling it over the Internet), making your viewing experience that much better.

Did you know that LIVE Youtube streams that have a "rewind" feature buffer that "rewind" content to your local storage and that data has to be constantly updated as content older than the "rewind time limit" is aged out? There's no other reason for my drive LEDs to constantly blink when the PC really isn't doing anything else than watching Youtube, not to mention my system monitoring program shows a steady low-level volume of drive writes when I watch LIVE videos with a "rewind" feature and those drive writes stop when I stop Youtube. If I "rewind" the LIVE video stream I see an increased volume of drive reads take place. QED?

I have a similar PC; i5-4460, 8GB RAM, 120GB SSD, Firefox-ESR, Debian Linux. I routinely play 2 or even 3 different Youtube LIVE streams in 1080p (all have the "rewind" feature) at the same time (different screens) with full audio (each monitor has cheepy speakers) and still copy files to and from the SSD filesystem without any errors seen or heard in the video streams.

It pays to "snoop on the hidden life" of your PC when it's doing something!

[Zan Lynx]

For what task/use-case Kyber is optimized for?

If I remember correctly, Facebook built it because they have SSD / NVMe backed data servers. When the database is flushing a long stream of writes to storage, the read latency suffered a lot when using the Linux standard NVMe scheduler of "none".

Kyber fixes that. It's designed for leveling out latency for users of NVMe drives under heavy load.

[polarathene]

Is there any Kyber or BFQ ppa so I can install it easily? Currently there's no BFQ in Ubuntu 18.10.

It should have BFQ and Kyber available, but you need to use blk-mq schedulers not the single queue ones like CFQ. If you want single queue BFQ that's a patched kernel, no kyber variant for single queue either.

[hax0r]

If you want MuQSS/BFS and BFQ in Ubuntu install Liquorix kernel, https://liquorix.net/, it is based on zen patchset, very good stuff for linux desktop users.

sudo add-apt-repository ppa:damentz/liquorix && sudo apt-get update
sudo apt-get install linux-image-liquorix-amd64 linux-headers-liquorix-amd64

check current i/o scheduler for given device, it can be changed on the fly (example):

cat /sys/block/<device>/queue/scheduler
echo noop > /sys/block/<device>/queue/scheduler OR append 'elevator=noop' to kernel boot line

[molecule-eye]

The situation right now for me is unbearable. If a process eats up a lot of background I/O, everything slows down. I was in a multiplayer game today, and suddenly everything came so a grinding halt until I Sysrq+K(illed) my whole tty. I couldn't even ssh in. I suspect it was doo to baloo, which has been eating up my resource usage lately.

When baloo gets ugly, disable it (which now removes the index) and reenable it. Even better, disable searching of contents before you reenable. If you need that functionality, use recoll instead for that.

[cl333r]

If you want MuQSS/BFS and BFQ in Ubuntu install Liquorix kernel, https://liquorix.net/, it is based on zen patchset, very good stuff for linux desktop users.

sudo add-apt-repository ppa:damentz/liquorix && sudo apt-get update
sudo apt-get install linux-image-liquorix-amd64 linux-headers-liquorix-amd64

check current i/o scheduler for given device, it can be changed on the fly (example):

cat /sys/block/<device>/queue/scheduler
echo noop > /sys/block/<device>/queue/scheduler OR append 'elevator=noop' to kernel boot line

It worked! Will see how it behaves.
The output:
~$ cat /sys/block/sda/queue/scheduler
[bfq-mq] none

[Raka555]

Hi Guys

I notice that many complain about bad responsiveness under heavy IO.

This is caused IMO by pressure on the VM system. By the time it starts flushing, there are no free pages and the kernel needs to scan/free like crazy.

Try the following settings in your /etc/sysctl.conf (or recommended for your distro)
vm.dirty_bytes = 16777216
vm.dirty_background_bytes = 4194304

Do "sudo sysctl -p" afterwards.

The VM settings for the so called "laptop modes" are brain dead and will cause major stutter and even OOM kills.

I cant remember when was the last time any of my machines stuttered