Nouveau Developer Working On OpenGL Extension To Help With Reverse-Engineering

Longtime open-source NVIDIA "Nouveau" driver developer Ilia Mirkin is drafting a new OpenGL extension proposal for helping out in driver reverse-engineering efforts.

The proposed GL_MESA_debug_operations extension is about making it easier to insert arbitrary commands into existing shaders to better understand these operations in different combinations. This extension is designed to help reverse engineering activities like Nouveau as they try to understand NVIDIA's hardware behavior but could have use-cases by other reverse-engineered OpenGL drivers like Freedreno, Panfrost / Lima, Etnaviv, and others. It's also possible this could be used as a sorts of testing/fuzzing different shader operations/behavior.

Ilia Mirkin describes the MESA_debug_operations proposal as:

Hardware often has operations which have poorly understood semantics, especially in combination with flags, modes, etc. As such, it can be desirable to insert those operations in the middle of an existing shader to see what they will do.

Options for doing this are few -- either a complete separate stack needs to be created in order to be able to submit this shader to the hardware and collect results, or an existing operation needs to be taken over. Doing this leads one to have to deal with all the existing limitations of that operation.

This extension introduces generic operations that may be used during reverse engineering activities. These operations are easily inserted into a shader and the driver may easily be modified to perform whatever operation is desired when seeing these new generic operations.

The extension hasn't yet been sent over to Khronos / OpenGL Working Group but on Sunday was announced on Mesa-dev for discussion by Mesa developers.