How Open-Source Allowed Valve To Implement VULKAN Much Faster On The Source 2 Engine

For those that haven't read our earlier Vulkan articles or elsewhere, here's how LunarG describes it at length:

Mantle, Metal, DX12 are all proprietary API’s that enable the GPU to be programmed in a very explicit way. Vulkan, with its Mantle roots, is also a very explicit API, and it achieves this in a cross-OS, cross-GPU fashion. To better understand the real benefit of these explicit APIs, we need to go back to their inspiration: the game console. For years, the console game engineers have been developing their software directly on the hardware. The extended life-cycles of video game consoles allows these platforms to reach a large audience; and using a 3D API can get in the way of creating a better game engine. The engine developers took over the responsibilities of building command buffers, keeping them in flight and managing the GPU resources required to ensure the hardware worked properly and efficiently. Ultimately the console game engines were able to squeeze the maximum performance out of the GPU while still leaving plenty of CPU bandwidth available for the game logic itself.

Traditional PC and mobile hardware are in a different boat. With new and better hardware coming out very frequently from competing vendors, change comes quickly on these platforms. 3D APIs are necessary to insulate the engine developer from a proliferation of hardware traditionally at a cost to overall performance. The task of building command buffers, keeping them in flight and managing GPU resources falls primarily to the drivers. Efficiently getting the most performance out of PC Hardware has become more complicated than the console way of programming directly to the hardware. The previously mentioned explicit APIs recognize that challenge, and have embraced the type of programming traditionally done on consoles allowing game engine developers to write more directly to the hardware. Vulkan's explicit approach exposes the nature of how modern GPUs work while still managing to abstract away most of the hardware and OS differences. The result is a manageable number of special cases that can be written by engine developers to enable a broad range of hardware to work more effectively than ever.

The first Valve title to be ported to Vulkan was Dota 2, which they showcased back at the Game Developers' Conference when announcing Vulkan. The Valve developers involved in adding Vulkan support to Source 2 compared writing this Vulkan back-end to writing a hardware drivers themselves. In conjunction with the sample Intel Vulkan driver being developed by LunarG on contract for Valve, they brought it up in about four months (from November 2014 to February 2015). LunarG was developing this reference driver in-step with the Source2 Vulkan back-end with "more and more sophisticated features" being added over time. Jens Owen, the CEO of LunarG, explained, "The Source 2 Vulkan back-end would be enhanced to utilize these features with integration and debugging being done by driver and engine developers working closely together. This team had minimal GPU debugging tools, nominal validation layers and a Vulkan API that was being defined and refined simultaneously with development. Their success hinged on two key elements: their experience with the 3D graphics pipeline and full access to the Vulkan driver source code."

"Enabling Valve’s team of Source 2 developers full access to the source code in the Vulkan sample driver made this development possible. Simply giving them a black box to figure out how to use the evolving Vulkan API would not have yielded the resulting GDC demo in such a short period of time. It was encouraging to see the benefits of providing driver source code to engine developers. The engine developers do not know the driver and hardware architecture like the driver team does but being able to step across the Vulkan interface in the debugger enabled the teamto see what the driver was doing with the workload and more often than not isolate an issue in either their back-end, or the driver, or both. The driver and game engine teams worked to interpret challenges in their respective areas of expertise and communicate clearly to the other side what was needed. We attribute the productivity realized while tracking down the most difficult issues to having a strong team of 3D graphics experts with full visibility of the source code on either side of the Vulkan interface," Jens added.

In speaking to the success of open-source drivers, Jens continued, "After the initial bring up, the Source2 backend now works on black box Vulkan drivers as well." Presumably, these "black box" Vulkan drivers are the forthcoming binary blobs out of AMD and NVIDIA.

Valve funded the development of this reference Intel driver in understanding the benefit of having the driver source code available when bringing up a game on Vulkan for the first time. Once the Vulkan specification is out this year and the subsequent release of this open-source driver, they hope it will benefit others in porting their games/software to Vulkan.

If you appreciate all of the exclusive Linux/open-source articles and hardware reviews published on Phoronix over the past eleven years, please highly consider subscribing to Phoronix Premium or even a PayPal tip so that this work -- that's done near single-handedly -- can continue full-speed ahead. Thank you.