Virginia Tech's "Popcorn Linux" For Distributed Thread Execution Seeking Feedback, Possible Upstreaming
Written by Michael Larabel in Linux Kernel on 29 April 2020 at 06:51 PM EDT. Add A Comment
LINUX KERNEL --
Popcorn Linux has been a multi-year effort out of Virginia Tech's Software and Systems Research Group for distributed thread execution across systems and even potentially different ISAs/accelerators given today's heterogeneous hardware.

Popcorn Linux is principally based around the Popcorn Compiler, a modified version of LLVM, and modifications to the Linux kernel and a support run-time library to allow for execution with a shared code-base on distributed hosts.


According today's request for comments, "Popcorn allows applications to start execution on a particular host and
migrate, at run-time, to a remote host. Multi-threaded applications may migrate any particular thread to any remote host. Unlike userspace checkpoint-restart solutions (e.g., CRIU), Popcorn enables seamless and dynamic migration across hosts during execution (no user interaction), and ensures coherent virtual memory across hosts for concurrent thread execution.
"

As for the kernel changes for this Popcorn approach to distributed thread execution, "Popcorn Linux implements a software-based distributed shared memory by extending Linux's virtual memory subsystem and it enables processes on different machines to observe a common and coherent virtual address space. Coherency of virtual memory pages of different hosts is ensured using a reader-replicate/writer-invalidate, page-level consistency protocol."

While a "request for comments" on Popcorn Linux was sent out on Wednesday to the Linux Kernel Mailing List, it's still a long ways out from potentially being upstreamed: the current code is based on Linux 5.2 rather than Git head, right now only the x86 support is presented as part of the RFC, some security features like L1 Terminal Fault mitigations have to be disabled for the moment, and there are known security concerns to the point of recommending Popcorn Linux currently not be loaded onto public Internet-connected systems.

But in any case, the RFC is out there so we'll see if upstream developers would ultimately be interested in such a solution. Those wanting to learn more about Virginia Tech's research in this area can visit PopcornLinux.org and the GitHub repository.
Related News
About The Author
Author picture

Michael Larabel is the principal author of Phoronix.com and founded the site in 2004 with a focus on enriching the Linux hardware experience. Michael has written more than 20,000 articles covering the state of Linux hardware support, Linux performance, graphics drivers, and other topics. Michael is also the lead developer of the Phoronix Test Suite, Phoromatic, and OpenBenchmarking.org automated benchmarking software. He can be followed via Twitter or contacted via MichaelLarabel.com.

Popular News This Week