Dynamic Kernel Stacks Proposed For Linux With Big Memory Savings

A "request for comments" patch series was posted on Monday for a new dynamic kernel stacks feature for Linux. Early testing has shown the potential for significant memory savings.

Following a proposal from last year's LSF/MM summit, Google has implemented support for dynamic kernel stacks. For the past decade the default stack size of the upstream Linux kernel has been 16K up from the prior 8K size. Google had been carrying kernel patches to keep 8K stacks but over time encountered the need for larger stack sizes to which they in turn hit increased memory use with their hyperscale deployments. For mitigating the increased memory use, Google has been working on dynamic kernel stacks since many kernel threads can fit within 4K or 8K stacks.

The work-in-progress RFC patches for dynamic kernel stacks has so far been lightly tested and allows scaling from 4K to THREAD_SIZE limits. Google engineers found that dynamic kernel stacks can save on average 70~75% of the kernel stack memory.

The early results posted by Pasha Tatashin have been very promising but can depend upon the workload, virtualization, and other factors:

Tatashin further added, "Some workloads with that have millions of threads would can benefit significantly from this feature."

See the RFC patch series for those interested in this work for saving on the kernel stack memory.