Google Engineer Reworks Direct I/O In Linux Kernel
Posted by Michael Larabel on February 14, 2013
A Google engineer working on Linux, Kent Overstreet, has reworked the Linux DIO (Direct I/O) code so that it's "vastly simpler" while also being faster for at least some test cases.
On Monday was the original "work in progress" patch to improve the DIO code in the Linux kernel. As Overstreet wrote then, "The end result is _vastly_ simpler - direct-io.c is now ~700 lines, vs. ~1300 previously. dio_submit is almost gone, I'm down to 4 things left in it. It relies heavily on my block layer patches for efficient bio splitting, and making generic_make_request() take arbitrary size bios...It also gets rid of the various differences between async and sync requests - previously, for async reads it marked pages dirty before submitting the io (in process context), then on completion punts to worqueue to redirty the pages if any need to be. This now happens for sync reads, too."
Not only does it yield a net reduction in the number of lines of code for the Linux DIO code, but it's also yielding performance improvements with the most recent patch.
The Google engineer published on Wednesday, "Got it working and ran some benchmarks. On a high end SSD, doing 4k random reads with fio I got around a 30% increase in throughput...the decrease in compiled binary size is even more dramatic than the reduction in [lines of code]...It's only been lightly tested - I haven't run xfstests yet - but there shouldn't be anything broken excluding btrfs. There's a few more performance optimizations I may do, but aside from the btrfs issues I think it's essentially done. Due to the sheer number of hairy corner cases in the dio code, I'd really like to get as much review as possible. The new code should be vastly easier to review and understand, I think."
This Linux I/O improvement that's leaner yet higher-performing sounds exciting but hasn't been reviewed extensively yet by Linux kernel developers. If everything pans out, hopefully this work will be merged into a future Linux kernel release in the near-term.
On Monday was the original "work in progress" patch to improve the DIO code in the Linux kernel. As Overstreet wrote then, "The end result is _vastly_ simpler - direct-io.c is now ~700 lines, vs. ~1300 previously. dio_submit is almost gone, I'm down to 4 things left in it. It relies heavily on my block layer patches for efficient bio splitting, and making generic_make_request() take arbitrary size bios...It also gets rid of the various differences between async and sync requests - previously, for async reads it marked pages dirty before submitting the io (in process context), then on completion punts to worqueue to redirty the pages if any need to be. This now happens for sync reads, too."
Not only does it yield a net reduction in the number of lines of code for the Linux DIO code, but it's also yielding performance improvements with the most recent patch.
The Google engineer published on Wednesday, "Got it working and ran some benchmarks. On a high end SSD, doing 4k random reads with fio I got around a 30% increase in throughput...the decrease in compiled binary size is even more dramatic than the reduction in [lines of code]...It's only been lightly tested - I haven't run xfstests yet - but there shouldn't be anything broken excluding btrfs. There's a few more performance optimizations I may do, but aside from the btrfs issues I think it's essentially done. Due to the sheer number of hairy corner cases in the dio code, I'd really like to get as much review as possible. The new code should be vastly easier to review and understand, I think."
This Linux I/O improvement that's leaner yet higher-performing sounds exciting but hasn't been reviewed extensively yet by Linux kernel developers. If everything pans out, hopefully this work will be merged into a future Linux kernel release in the near-term.
Discuss this article in our forums, IRC channel, or email the author. You can also follow our content via RSS and on social networks like Facebook, Identi.ca, and Twitter (@Phoronix and @MichaelLarabel). Subscribe to Phoronix Premium to view our content without advertisements, view entire articles on a single page, and experience other benefits.
Copyright © 2004 - 2013 by Phoronix Media.
All trademarks used are properties of their respective owners. All rights reserved. Legal Disclaimer & Disclosure.
Powered by the Phoronix Content Management System (PHXCMS-7.4).
All trademarks used are properties of their respective owners. All rights reserved. Legal Disclaimer & Disclosure.
Powered by the Phoronix Content Management System (PHXCMS-7.4).