1. Computers
  2. Display Drivers
  3. Graphics Cards
  4. Memory
  5. Motherboards
  6. Processors
  7. Software
  8. Storage
  9. Operating Systems


Facebook RSS Twitter Twitter Google Plus


Phoronix Test Suite

OpenBenchmarking.org

SUSE Posts kGraft, Red Hat Posts Kpatch Patches

Linux Kernel

Published on 01 May 2014 12:22 PM EDT
Written by Michael Larabel in Linux Kernel
5 Comments

SUSE developers yesterday posted their sixteen patches for implementing their kGraft live kernel patching mechanism in the mainline Linux kernel as an alternative to Ksplice. Red Hat has immediately followed-up by posting their kernel patches to Kpatch, their new approach to live patching a running Linux kernel.

SUSE announced kGraft back in February as a new way of live-patching the Linux kernel to reduce downtime. In late March they then released the kGraft source code and reiterated their intention to mainline kGraft in the Linux kernel.

Red Hat in early March meanwhile announced Kpatch as their new approach to live kernel patching that was under development prior to knowing about kGraft.

Jiri Slaby of SUSE posted yesterday the kGraft kernel patches under a "request for comments" flag to seek feedback from Linux kernel developers on the work. The kGraft work is spread across 16 patches.

This morning, Red Hat's Josh Poimboeuf countered with their two kernel patches needed to implement Kpatch. Those two patches can be found via the LKML mailing list. Kpatch is served as a self-contained GPL kernel module that doesn't need any core kernel code changes. Red Hat wants to see it merged, or some combination with kGraft.

Josh Poimboeuf's explanation from the Red Hat perspective in comparing against kGraft came down to:
I think the biggest difference between kpatch and kGraft is how they ensure that the patch is applied atomically and safely.

kpatch checks the backtraces of all tasks in stop_machine() to ensure that no instances of the old function are running when the new function is applied. I think the biggest downside of this approach is that stop_machine() has to idle all other CPUs during the patching process, so it inserts a small amount of latency (a few ms on an idle system).

Instead, kGraft uses per-task consistency: each task either sees the old version or the new version of the function. This gives a consistent view with respect to functions, but _not_ data, because the old and new functions are allowed to run simultaneously and share data. This could be dangerous if a patch changes how a function uses a data structure. The new function could make a data change that the old function wasn't expecting.

With kpatch, that's not an issue because all the functions are patched at the same time. So kpatch is safer with respect to data interactions.

Other advantages of the kpatch stop_machine() approach:

- IMO, the kpatch code is much simpler than kGraft. The implementation is very straightforward and is completely self-contained. It requires zero changes to the kernel.

(However a new TAINT_KPATCH flag would be a good idea, and we do anticipate some minor changes to kprobes and ftrace for better compatibility.)

- The use of stop_machine() will enable an important not-yet-implemented feature to call a user-supplied callback function at loading time which can be used to atomically update data structures when applying a patch. I don't see how such a feature would be possible with the kGraft approach.

- kpatch applies patches immediately without having to send signals to sleeping processes, and without having to hope that those processes handle the signal appropriately.

- kpatch's patching behavior is more deterministic because stop_machine() ensures that all tasks are sleeping and interrupts are disabled when the patching occurs.

- kpatch already supports other cool features like:
- removing patches and rolling back to the original functions
- atomically replacing existing patches
- incremental patching
- loading multiple patch modules
We should see soon enough on the kernel mailing list whether kGraft or Kpatch will prevail, otherwise how well the two will work together.

About The Author
Michael Larabel is the principal author of Phoronix.com and founded the web-site in 2004 with a focus on enriching the Linux hardware experience and being the largest web-site devoted to Linux hardware reviews, particularly for products relevant to Linux gamers and enthusiasts but also commonly reviewing servers/workstations and embedded Linux devices. Michael has written more than 10,000 articles covering the state of Linux hardware support, Linux performance, graphics hardware drivers, and other topics. Michael is also the lead developer of the Phoronix Test Suite, Phoromatic, and OpenBenchmarking.org automated testing software. He can be followed via and or contacted via .
Latest Linux Hardware Reviews
  1. Intel Xeon E5-1680 v3 & E5-2687W v3 Compared To The Core i7 5960X On Linux
  2. Intel 120GB 530 Series SSD Linux Performance
  3. Btrfs/EXT4/XFS/F2FS RAID 0/1/5/6/10 Linux Benchmarks On Four SSDs
  4. AMD's Windows Catalyst Driver Remains Largely Faster Than Linux Drivers
Latest Linux Articles
  1. NVIDIA vs. Nouveau Drivers With Linux 3.18 + Mesa 10.4-devel
  2. Is The Open-Source NVIDIA Driver Fast Enough For Steam On Linux Gaming?
  3. Linux 3.18 File-System Performance Minimally Changed But Possible Regressions
  4. AMD Radeon Gallium3D Is Catching Up & Sometimes Beating Catalyst On Linux
Latest Linux News
  1. Linux 3.18 Kernel: Not Much Change With Intel Haswell Performance
  2. More File-System Tests Of The Linux 3.18 Kernel
  3. Using NVIDIA's NVENC On Linux With FFmpeg
  4. There's Talk Again About An "Open To The Core" Ubuntu Laptop
  5. PowerVR SGX Driver Code Gets Leaked
  6. V2 Of KDBUS Published For Linux Kernel Review
  7. VirtualBox 4.3.20 Arrives, Still No Sign Of VirtualBox 4.4
  8. Scientific Linux 6.6 vs. Scientific Linux 7.0 Benchmarks
  9. Qualcomm Looks To Get Into The ARM Server Business
  10. HHVM 3.4 Adds New Features, Support
Latest Forum Discussions
  1. Roadmap to Catalyst 14.10 ?
  2. Updated and Optimized Ubuntu Free Graphics Drivers
  3. Cant get working Kaveri APU - A10-7850k
  4. Debian Developer Resigns From The Systemd Maintainership Team
  5. Script for Fan Speed Control
  6. Debian Init System Coupling Vote Results
  7. The Slides Announcing The New "AMDGPU" Kernel Driver
  8. Ubuntu Developers Still Thinking What To Do About Adobe Flash Support