interactivity and preemptibility (desktop responsiveness) cumulative thread

[kernelOfTruth]

Seems like the linux-kernel recently really gets some love in terms of preemptibility for much lower latency which (might lead) leads to better responsiveness at the desktop / X, too:

[PATCH 00/20] mm: Preemptibility -v4


this first will get into -next when people feel like it's ready

[kernelOfTruth]

desktop linux really gets some love

[PATCH 0/10] Reduce latencies and improve overall reclaim efficiency v1

There have been numerous reports of stalls that pointed at the problem being
somewhere in the VM. There are multiple roots to the problems which means
dealing with any of the root problems in isolation is tricky to justify on
their own and they would still need integration testing. This patch series
gathers together three different patch sets which in combination should
tackle some of the root causes of latency problems being reported.

The first patch improves vmscan latency by tracking when pages get reclaimed
by shrink_inactive_list. For this series, the most important results is
being able to calculate the scanning/reclaim ratio as a measure of the
amount of work being done by page reclaim.

Patches 2 and 3 account for the time spent in congestion_wait() and avoids
calling going to sleep on congestion when it is unnecessary. This is expected
to reduce stalls in situations where the system is under memory pressure
but not due to congestion.

Patches 4-8 were originally developed by Kosaki Motohiro but reworked for
this series. It has been noted that lumpy reclaim is far too aggressive and
trashes the system somewhat. As SLUB uses high-order allocations, a large
cost incurred by lumpy reclaim will be noticeable. It was also reported
during transparent hugepage support testing that lumpy reclaim was trashing
the system and these patches should mitigate that problem without disabling
lumpy reclaim.

Patches 9-10 revisit avoiding filesystem writeback from direct reclaim. This has been
reported as being a potential cause of stack overflow but it can also result in poor IO
patterns increasing reclaim latencies.

There are patches similar to 9-10 already in mmotm but Andrew had concerns
about their impact. Hence, I revisisted them as the last part of this series
for re-evaluation.

Oddly, the nocongest patches took longer to complete the test but the
overall series reduces the test time by almost an hour or about in one
third of the time. I also looked at the latency figures when allocating
huge pages and got this

http://www.csn.ul.ie/~mel/postings/vmscanreduce-20100609/highalloc-interlatency-hydra-mean.ps

So it looks like the latencies in general are reduced. The full series
reduces latency by massive amounts but there is also a hint why nocongest
was slower overall. Its latencies were lower up to the point where 72%
of memory was allocated with huge pages. After the latencies were higher
but this problem is resolved later in the series.

[kernelOfTruth]

For those who want to try those out:

e.g. click on the link of [PATCH 0/9] Reduce latencies and improve overall reclaim efficiency v1

then click on "First message in thread" (at the left) - if it's not already the first mail posted concerning that topic

* this will usually contain a description of the patchset, the number of contained patches, etc.

at the left you'll see a sort of tree in which the patches/messages are ordered:

First message in thread
Mel Gorman
Mel Gorman
Mel Gorman
Mel Gorman
Mel Gorman
Mel Gorman
Mel Gorman
Mel Gorman
Mel Gorman
Mel Gorman
Mel Gorman
Mel Gorman

click on the next message with the author's name

in this example it'll be: "[PATCH 01/10] tracing, vmscan: Add trace events for LRU list shrinking"

conveniently lkml.org also shows us if a patch was attached by saying:

Patch in this message
Get diff 1

(or diff 2, diff 3 - if there are more)

click on "Get diff 1" and you'll see the patch in raw form

now repeat this for all the patches in that series and download them locally


make sure you have a recent enough linux-tree - then

change into that directory and first see whether it's worth it to apply those patches:

patch -p1 --dry-run < ..patch/to/patch/01_reduce\ latencies\ and\ improve\ overall\ reclaim\ efficiency\ v1.patch

to really apply it remove "--dry-run" from the patch-command

if there are some rejects ("patch failed" - files with e.g. vmscan.c.rej) then you'll need to fix those reject manually

in most cases they should be trivial to fix - if not - don't risk it unless you know what part of the kernel you're dealing with and the consequences ...

now: good luck & have fun !

[kernelOfTruth]

man I love the direction the kernel-development is currently going towards:

[PATCH 0/3][RFC] Improve load balancing when tasks have large weight differential

[kernelOfTruth]

in case anybody missed it for 2.6.35:

[PATCH] vmscan: raise the bar to PAGEOUT_IO_SYNC stalls <-- this to a great degree reduces hangs or jerkiness during heavy i/o

[kernelOfTruth]

in case anybody missed it for 2.6.35:

[PATCH] vmscan: raise the bar to PAGEOUT_IO_SYNC stalls <-- this to a great degree reduces hangs or jerkiness during heavy i/o