BFS Scheduler Lost Some Charm With Linux 3.11

[a user]

I am sorry, but to those saying that scheduling is worse under Windows: You've clearly never tried to do any work under heavy disk usage in Linux.

i did. there is no system performing worse under heavy load than windows (counting only relevant OSs).

Here, try this use case:

Download photorec and let it restore deleted files from one partition to another. Then try to move your mouse, open some applications, type something in. The waiting times are unacceptable! On my system (12.04 LTS, Kernel 3.12) I can't even move the mouse properly! This never happens with Windows. The CPU and disk usage might be as high as they want, you will always be able to move your mouse and kill the offending application, if necessary.

i think i know what caused your trouble and i admit that there are some scenarios under which linux kernels with default configurations performed quite bad.
but this is not a representativ example to judge this issue currectly.

These issues with responsiveness are my number one problem with Linux.

the only time i ever had issue with responsiveness was going out of memory without swap.
but i can't count the times i had responsiveness issues on windows.

and i am not talking only as a private user but also as a over 6 years scientist executing various heavy work loads of different kinds on linux and windows systems.

I can live with bugs in the UI here and there but to see this core system issue stay unfixed for five years now is mind-boggling to me. Linus wants desktop Linux to succeed. Then why not put more resources into fixing one of Linux' most deal-breaking problems!? And if there's a trade-off between performance and responsiveness why not make sure that the kernel can be configured towards either end by distributions?

i would like to comment this but i can'T as i never experienced such issues.

[erendorn]

i am yet to experience this so called bug number one since i am running linux (>12 years).

I experienced it on the N900 (it could even trigger the watchdog when copying very big files), so it can happen.
But then, it had a 600MHz ARM proc and 256Mb of ram, so it was a tad more constrained that the desktops I'm used to running.

[movieman]

Then try to move your mouse, open some applications, type something in. The waiting times are unacceptable! On my system (12.04 LTS, Kernel 3.12) I can't even move the mouse properly! This never happens with Windows. The CPU and disk usage might be as high as they want, you will always be able to move your mouse and kill the offending application, if necessary.

Are you swapping? On Windows, our video drivers locked the mouse code in RAM so it couldn't be swapped out and could always run, whereas Linux probably doesn't.

Of course one reason I moved to Linux was because XP had the delightful habit of swapping out my web browser to make the disk cache bigger if I was copying a big file from one disk to another in the background... killing the offending process was hellish when most of the operating system had been swapped out to disk to make space for pointless disk caching.

I think Windows 7 may have fixed that, but I disabled swapping completely on the new Windows machine so it's hard to tell.

[lano1106]

BFS is not worth the effort to install at this point , heck even Liquorix performance kernel stopped using it because all of the bugs it caused and the non effect it has on desktop responsiveness.

BFS did make things smoother on the desktop a few years back I will admit, but lately its been noticeably slower and really buggy.

Kudos to Con for helping to move the kernel forward, and I hope he does find and fixes the bugs in BFS.

I strongly disagree with your statements. I have been using BFS for about a year now and I did report to Con very minor issues (sched features that no one uses in real life. posix process cpu clocks) and I did contribute few other minor improvements to it.

What I have seen is kernel race condition bugs that went unnoticed with CFS that did show up with BFS due to their different timing signatures.

[lano1106]

What? I was not the one who made a false statement about CFS.

Well, IMHO, this is nitpicking. In my mind O(1) is the same as O(C). If you say that CFS isn't O(1) and we can agree that it is O(C), I have no problem with that.

We are saying the same thing. It will scale with any number of processes that it has to schedule.

[tiredoffglrx]

Are you swapping? On Windows, our video drivers locked the mouse code in RAM so it couldn't be swapped out and could always run, whereas Linux probably doesn't.

Of course one reason I moved to Linux was because XP had the delightful habit of swapping out my web browser to make the disk cache bigger if I was copying a big file from one disk to another in the background... killing the offending process was hellish when most of the operating system had been swapped out to disk to make space for pointless disk caching.

I think Windows 7 may have fixed that, but I disabled swapping completely on the new Windows machine so it's hard to tell.

Yes, I think I mixed this up a bit. The mouse movement issue only occurs when the system starts swapping. The general issues with responsiveness, OTOH, always occur when the disk I/O is under heavy load.

It's interesting to see how Windows handles this. I must ask, though: Do you happen to know why Linux doesn't do the same? For me (a newbie to Linux, who has no idea of programming) this sounds simple enough to implement.

As for the issue with the disk cache, I have yet to experience anything like that on Windows. Again, it's rather Linux where I often see problems with responsiveness when copying large files or large amounts of data in general. As far as I know this has to do with aggressive caching by the kernel, where it will quickly load any file it currently handles to memory, agnostic of whether it displaces crucial desktop-components while doing so.

i did. there is no system performing worse under heavy load than windows (counting only relevant OSs).

i think i know what caused your trouble and i admit that there are some scenarios under which linux kernels with default configurations performed quite bad.
but this is not a representativ example to judge this issue currectly.


the only time i ever had issue with responsiveness was going out of memory without swap.
but i can't count the times i had responsiveness issues on windows.

and i am not talking only as a private user but also as a over 6 years scientist executing various heavy work loads of different kinds on linux and windows systems.


i would like to comment this but i can'T as i never experienced such issues.

I must admit that I am very new to the Linux world and can't look back at much experience. But this particular issue has been plaguing me right from the start and through different systems. It's one of the first problems I noticed after switching to Ubuntu and something I regularly run into in my everyday usage. And, most importantly, an issue that I was able to completely fix by installing a BFS/BFQ patched kernel (until I had to switch back that is).

So I am glad that you didn't have to experience the issue yourself, yet, but at the same time I must point out that this is a very real issue, as attested by the large response to the bug report I linked to previously.

[curaga]

There has been some work, see:

https://github.com/stiletto/angrymlocker

http://vignatti.com/2007/07/17/xorg-input-thread-summary-or-something-2/

[mark45]

It's not even a "pathetic behaviour", its by design.

With scheduler, you have a trade-off between performance and responsiveness.
(One of the simplest way to understand: + lots of bla bla bla.

This is one example of demagogy by pseudo-explanation like it the past when you had to compile your own driver the Linux fans tried to "explain" you how this is good for you, or that it's not a "problem" or can't be solved, or how Linux having a small market share is "good", or whatnot. It's like listening to liberal demagogy about the invisible hand of the market.

Long story short, it's a bug (more likely a collection of bugs), happens randomly, but it's serious, hasn't been (fully) fixed, it's a problem.

[lano1106]

It's interesting to see how Windows handles this. I must ask, though: Do you happen to know why Linux doesn't do the same? For me (a newbie to Linux, who has no idea of programming) this sounds simple enough to implement.

BTW, I have seen the mouse cursor blocking without swapping. My best explanation is a totally different design philosophy between Windows and Linux.

In Windows, the GUI subsystem is part of the OS. In Linux it is implemented as a user space process.

In Windows, the GUI is possibly managed totally outside the scheduler maybe in part in the interrupt handlers themselves.

In Linux, the GUI will be impacted by process scheduling. Possibly playing with the X server scheduling params could help with the mouse cursor issue.

[kertoxol]

I have the same problem as some of you guys, CFS behaving bad with heavy IO workload, and as said by an user before, tweaking CFS with sysctl.conf could lead to improvements. I think to know why some people aren't experiencing this problem, the cause could be a distribution custom sysctl.conf with such settings already applied.

So, why don't we compare our sysctl.conf files?

vm.swappiness = 1

Isn't doing nothing because the system is swapping even with 500M/1G of memory cached, over 4GB... Obviously swapping kills responsiveness a lot, even more than moving data from/to usb storage drives.