AMD Linux Engineers Introduce New "schedstat" Tool

Written by Michael Larabel in AMD on 8 May 2024 at 06:54 AM EDT. 3 Comments
AMD Linux engineers have introduced a new perf tool called "schedstat" that aims to be less resource intensive and convenient than the existing "perf sched" tool for profiling kernel scheduler behavior.

The schedstat tool conveniently reports the time elapsed in jiffies and other CPU scheduling statistics. The intent of the tool is to be lighter-weight and used by developers and others profiling Linux kernel scheduler changes.

AMD engineer Ravi Bangoria explained of the schedstat tool:
"Existing `perf sched` is quite exhaustive and provides lot of insights into scheduler behavior but it quickly becomes impractical to use for long running or scheduler intensive workload. For ex, `perf sched record` has ~7.77% overhead on hackbench (with 25 groups each running 700K loops on a 2-socket 128 Cores 256 Threads 3rd Generation EPYC Server), and it generates huge 56G for which perf takes ~137 mins to prepare and write it to disk.

Unlike `perf sched record`, which hooks onto set of scheduler tracepoints and generates samples on a tracepoint hit, `perf sched schedstat record` takes snapshot of the /proc/schedstat file before and after the workload, i.e. there is zero interference on workload run. Also, it takes very minimal time to parse /proc/schedstat, convert it into perf samples and save those samples into file. Result file is much smaller. So, overall `perf sched schedstat record` is much more light-weight compare to `perf sched record`.

We, internally at AMD, have been using this (a variant of this, known as "sched-scoreboard") and found it to be very useful to analyse impact of any scheduler code changes.

Please note that, this is not a replacement of perf sched record/report. The intended users of the new tool are scheduler developers, not regular users."

Schedstat sounds like a great fit and complementary tool to the common "perf sched" functionality and a nice evolution of their earlier sched-scoreboard tool.

AMD sched-scoreboard

AMD's schedstat tool was presented today on the Linux kernel mailing list as part of this patch series seeking comments from other developers on interest and upstreaming this code into the perf tools within the Linux kernel source tree.
Related News
About The Author
Michael Larabel

Michael Larabel is the principal author of and founded the site in 2004 with a focus on enriching the Linux hardware experience. Michael has written more than 20,000 articles covering the state of Linux hardware support, Linux performance, graphics drivers, and other topics. Michael is also the lead developer of the Phoronix Test Suite, Phoromatic, and automated benchmarking software. He can be followed via Twitter, LinkedIn, or contacted via

Popular News This Week