GCC "-fparallel-jobs" Sent Out For Compiling Individual Files In Parallel - Up To ~1.9x Speedup

For the past two summers student developer Giuliano Belinassi has been working under Google Summer of Code in working to address GCC parallelization bottlenecks and ultimately a goal of allowing single source files to be split up for compilation in parallel by GCC. In particular, being able to split the compilation of large source files across multiple CPU cores. The latest patches on this "-fparallel-jobs=" was sent out today as we approach the end of GSoC 2020.

The GCC parallelization effort has been a success and shown much promise even going back to the end of last year. This summer he's been working more on allowing more of the GCC work to happen in parallel for large source files and culminated this week with sending out the "-fparallel-jobs=" patches.

The patches parallelize intra-procedural optimizations using the GCC LTO engine. When -fparallel-jobs=N is set, the maximum number of jobs to use is specified. The -fparallel-jobs can also accept "jobserver" for communicating with the GNU Make job server for launching jobs.

The new code works by splitting the output of a given source file, searching for symbols that must be partitioned together, cc1 will fork itself for each partition to be compiled, and the driver will then partially link each partitioned file together into a single object file. Depending upon the source file it also has the logic to potentially serial compile the file if it deems a parallelized compile ineffective.

Speedups on a Core i7 mobile CPU range from 0.95x to 1.9x when testing with two large source files in GCC. Yes, in some cases at the moment there is the possibility of regression and hence why this parallelization is concealed by a compiler flag for the time being. Included with this work is a new partitioner for parallel compilation, creating a fork-based parallelism engine, and other related bits.

The patches in their current form amount to around twenty five hundred lines of new code. Hopefully it will be reviewed punctually and ideally we could finally see this new parallelization support within GCC in the not too distant future.

While on the subject of compiler build times, for those interested in the current build speed requirements of different CPUs, the new OpenBenchmarking.org has some easy displays of: Linux kernel build times for a wide assortment of CPUs, LLVM build times, GCC build times, and more.