SEUS PTGI on radeonSI

[QuImUfu]

I thought this might be interesting for some people here, so i figured I'd make a post about it

SEUS PTGI is a minecraft shader (loaded via OptiFine, documentation) that does something called Path Traced Global Illumination to archive realistic lightning in Minecraft.
It is written using GLSL and is sadly currently behind a patreon "paywall" if you search for "SEUS PTGI E5" in your favourite search engine you could surely find a way to get it anyway.
It currently does AFAIK not work with any driver except Nvidias.

Now the part i found interesting, especially as i never looked in to graphics programming:
I tried running it on my RX460. Then i tried to analyse (and fix) all the errors that occurred.
The errors all occurred (AFAIK) because the shader pack does not conform to spec/is broken. Interestingly enough it does work on the Nvidia driver anyway:

Two errors occurred because the pack uses features that are not available in the declared #version. These are "fixed" by enabling force_glsl_extensions_warn in drirc for java:

gbuffers_water.vsh:
error:
no matching function for call to 'texture2DLod(sampler2D, vec2, int)'
problem:
uses glsl #version 120 should use #version 130.
see: https://bugs.freedesktop.org/show_bug.cgi?id=99477#c6

gbuffers_textures.fsh:
error:
initializer of const variable 'K_R' must be a constant expression
initializer of const variable 'K_M' must be a constant expression
problem:
the shader uses a function parameter to initialize those variables which requires #version 420. Because the rest of the file is written for #version 130 #version 420 compatibility is needed.
see: https://www.khronos.org/opengl/wiki/...ant_expression

Two errors seem to occur because there is faulty code that is never used and (i think) ignored by the Nvidia driver. Keep in mind i have only very basic understanding about how these things work, so i could be very wrong with that:

shadow.*sh:
error:
geometry shader input 'vrawNormal' has no matching output in the previous stage
problem:
vrawNormal is "in'ed" in shadow.gsh, but is not "out'ed" by shadow.vsh.
vrawNormal is being used to crate and "out" rawNormal which is in'ed by shadow.fsh but never used.
what i did to fix it:
remove vrawNormal and rawNormal and its usages from shadow.gsh and shadow.vsh

gbuffers_textured_lit.*sh:
error:
fragment shader input 'viewPos' has no matching output in the previous stage
problem:
viewPos is "varying'ed" in gbuffers_textured_lit.fsh but not "varying'ed" in gbuffers_textured_lit.vsh.
viewPos is used in gbuffers_textured_lit.fsh to calculate a value (in multiple steps) that is than not used because the line it would be used in is commented out.
what i did to fix it:
remove/comment every line containing viewPos in gbuffers_textured_lit.fsh

Another error does occur after fixing these, that i could not really understand how it does not happen with Nvidias driver.

shadow.*sh:
error:
fragment shader input 'invalid' has no matching output in the previous stage
problem:
invalid is "in'ed" in shadow.fsh, but is not "out'ed" by shadow.gsh.
it seems to get used.
what i did to "fix" it:
I removed it and it's usage. That might be the wrong thing to do.

After that all shaders compiled and linked successfully. This lead to the runtime error that i could not fix and that prevents the pack from working:

error:
Mesa: User error: GL_INVALID_OPERATION in glDrawArrays(mode=GL_QUADS vs geometry shader input GL_TRIANGLES)
Mesa: User error: GL_INVALID_OPERATION in glDrawArrays(mode=GL_QUADS vs geometry shader input GL_TRIANGLES)
Mesa: User error: GL_INVALID_OPERATION in glDrawArrays(mode=GL_QUADS vs geometry shader input GL_TRIANGLES)
Mesa: User error: GL_INVALID_OPERATION in glDrawArrays(mode=GL_QUADS vs geometry shader input GL_TRIANGLES)
Mesa: User error: GL_INVALID_OPERATION in glDrawArrays(mode=GL_QUADS vs geometry shader input GL_TRIANGLES)
[02:50:23] [main/ERROR] [net.optifine.shaders.SMCLog]: [Shaders] OpenGL error: 1282 (Invalid operation), program: shadow, at: shadow terrain solid
[02:50:23] [main/ERROR] [net.optifine.shaders.SMCLog]: [Shaders] OpenGL error: 1282 (Invalid operation), program: shadow, at: shadow terrain cutoutmipped
[02:50:23] [main/ERROR] [net.optifine.shaders.SMCLog]: [Shaders] OpenGL error: 1282 (Invalid operation), program: shadow, at: pre-useProgram
[02:50:23] [main/ERROR] [net.optifine.shaders.SMCLog]: [Shaders] OpenGL error: 1282 (Invalid operation), program: shadow, at: pre-useProgram
[02:50:23] [main/ERROR] [net.optifine.shaders.SMCLog]: [Shaders] OpenGL error: 1282 (Invalid operation), program: shadow, at: shadow translucent
problem (copied from the shader packs creators explanation):
Minecraft uses quads for geometry instead of tris/triangles, and geometry shaders do not allow quads as an input. Nvidia has an extension that fixes this, but unfortunately AMD does not. One way to allow AMD cards to function, would be if OptiFine added tessellation shaders, since those run before geometry shaders and always output tris. So the input to the geometry shader would be tris instead of quads. The developer who shared the information about tessellation shaders wasn't 100% sure that it would fix AMD's issue.

TLDR:
Looked at OpenGL shaders that work on Nvidias driver but not mesa. I came to these maybe wrong (i do not really have experience or knowledge about graphics programming) conclusions:
It's because mesa is more picky about what features are available on what version (without force_glsl_extensions_warn)
and it does not ignore errors that have no effect
and it does not support quads for geometry shaders.

[Dukenukemx]

You ever get this working? I'm trying it out on my RX 470 and I'm getting the same errors.

[QuImUfu]

No, the last issue can afaik only be fixed by modifying MC sourcecode (e.g. a Optifine update) or by a driver change that breaks the OpenGL spec and allows quads as inputs of a triangle geometry shader. (possibly converted to triangles?) Something like this is what i think Nvidias driver does.
i modified OptiFine's bytecode and put two tessellation stages in between, but that does not work as tessellation shaders require their input to be patches.
Maybe I'll try to modify minecraft to output triangles (or patches?) instead of quads.