Gallium3D Post-Processing, MLAA Nearly Ready

[HokTar]

The normal version doesn't seem to work right, but pp_jimenezmlaa_color should.

I didn't see any rendering issues with the mentioned command. But I couldn't really see an improvement, either.

[curaga]

The normal version doesn't seem to work right, but pp_jimenezmlaa_color should.

It's not broken, but certainly loses to the color one in many cases.

Sydney certainly has better curves with the depth version compared to without:

[droste]

I wonder if you could get a backtrace?

ATTENTION: default value of option pp_jimenezmlaa overridden by environment.
Initializing the post-processing queue.
Initializing program
mlaa: using 2 max search steps
[New Thread 0x7ffff3605700 (LWP 7719)]
[Thread 0x7ffff3605700 (LWP 7719) exited]
Queue successfully allocated. 1 filter(s).
Initializing FBOs, size 300x300
Requesting 1 temps and 2 inner temps
Running synchronized to the vertical refresh.  The framerate should be
approximately the same as the monitor refresh rate.

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff3e2ecac in r600_context_pipe_state_set_resource (ctx=0x792310, state=0x7fffec0622d8, block=0x85bd40) at r600_hw_context.c:1152
1152                            state->bo[0]->binding |= BO_BOUND_TEXTURE;
(gdb) bt
#0  0x00007ffff3e2ecac in r600_context_pipe_state_set_resource (ctx=0x792310, state=0x7fffec0622d8, block=0x85bd40) at r600_hw_context.c:1152
#1  0x00007ffff3e0fa23 in evergreen_set_ps_sampler_view (ctx=0x791fe0, count=1, views=0x90ff78) at evergreen_state.c:1068
#2  0x00007ffff4083a2d in pp_jimenezmlaa_run (ppq=0x791d20, in=0x7fffec022370, out=0x7fffec01ecf0, n=0, iscolor=false) at postprocess/pp_mlaa.c:128
#3  0x00007ffff3e28fca in dri2_flush_drawable (dPriv=0x7fffec0204b0) at dri2.c:56
#4  0x00007ffff76d62ee in dri2SwapBuffers (pdraw=<value optimized out>, target_msc=0, divisor=0, remainder=0) at dri2_glx.c:556
#5  0x00007ffff76ada46 in glXSwapBuffers (dpy=0x606010, drawable=130023426) at glxcmds.c:805
#6  0x00000000004033a5 in draw_frame (argc=<value optimized out>, argv=<value optimized out>) at glxgears.c:341
#7  event_loop (argc=<value optimized out>, argv=<value optimized out>) at glxgears.c:696
#8  main (argc=<value optimized out>, argv=<value optimized out>) at glxgears.c:776

with Mesa 7.12-devel (git-09b5f1f) on evergreen (juniper)

[curaga]

Can't reproduce on my e-350 (evergreen). The depth version works fine on it.

From the bt, it segfaults when trying to set the depth as a shader input. I believe there's no issues with this on nvc0, soft, llvm, or on my r600g; perhaps it's a r600g bug w/ dedicated memory?

[curaga]

Tested on the same revision.

[droste]

This could be a reason.

Here's a more detailed backtrace.

ATTENTION: default value of option pp_jimenezmlaa overridden by environment.
Initializing the post-processing queue.
Initializing program
mlaa: using 2 max search steps
[New Thread 0x7ffff3606700 (LWP 10840)]
[Thread 0x7ffff3606700 (LWP 10840) exited]
Queue successfully allocated. 1 filter(s).
Initializing FBOs, size 300x300
Requesting 1 temps and 2 inner temps
Running synchronized to the vertical refresh.  The framerate should be
approximately the same as the monitor refresh rate.

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff3e2fcac in r600_context_pipe_state_set_resource (ctx=0x792310, state=0x923788, block=0x85bd40) at r600_hw_context.c:1152
1152                            state->bo[0]->binding |= BO_BOUND_TEXTURE;
(gdb) bt full
#0  0x00007ffff3e2fcac in r600_context_pipe_state_set_resource (ctx=0x792310, state=0x923788, block=0x85bd40) at r600_hw_context.c:1152
        dirty = <value optimized out>
        is_vertex = 0 '\000'
#1  0x00007ffff3e10a23 in evergreen_set_ps_sampler_view (ctx=0x791fe0, count=1, views=0x90ff78) at evergreen_state.c:1068
        rctx = 0x791fe0
        resource = 0x90ff78
        i = <value optimized out>
        has_depth = 1
#2  0x00007ffff408456d in pp_jimenezmlaa_run (ppq=0x791d20, in=0x918a90, out=0x99e540, n=0, iscolor=false) at postprocess/pp_mlaa.c:128
        p = 0x791e30
        mstencil = {depth = {enabled = 0, writemask = 0, func = 0}, stencil = {{enabled = 1, func = 7, fail_op = 0, zpass_op = 2, zfail_op = 0, 
              valuemask = 255, writemask = 255}, {enabled = 0, func = 0, fail_op = 0, zpass_op = 0, zfail_op = 0, valuemask = 0, writemask = 0}}, alpha = {
            enabled = 0, func = 0, ref_value = 0}}
        v_tmp = {reference = {count = 0}, format = PIPE_FORMAT_NONE, texture = 0x6398c0, context = 0x943630, u = {tex = {first_layer = 16384, 
              last_layer = 1024, first_level = 0, last_level = 0}, buf = {first_element = 67125248, last_element = 0}}, swizzle_r = 0, swizzle_g = 0, 
          swizzle_b = 0, swizzle_a = 0}
        arr = {0x612a60, 0x0, 0x0}
        w = 300
        h = 300
        ref = {ref_value = "\001"}
#3  0x00007ffff3e29fca in dri2_flush_drawable (dPriv=0x62efd0) at dri2.c:56
        ctx = 0x62ee50
        drawable = 0x0
        ptex = <value optimized out>
#4  0x00007ffff76d62ee in dri2SwapBuffers (pdraw=<value optimized out>, target_msc=0, divisor=0, remainder=0) at dri2_glx.c:556
        priv = 0x62ef00
        dpyPriv = 0x612a60
        psc = 0x619780
        pdp = 0x614ee0
        ret = 0
#5  0x00007ffff76ada46 in glXSwapBuffers (dpy=0x606010, drawable=100663298) at glxcmds.c:805
        pdraw = 0x85bdb0
        gc = 0x62ec70
        tag = <value optimized out>
        opcode = <value optimized out>
        req = <value optimized out>
#6  0x00000000004033a5 in draw_frame (argc=<value optimized out>, argv=<value optimized out>) at glxgears.c:341
        dt = <value optimized out>
        frames = 1
        tRot0 = 1314545668.04561
        tRate0 = 1314545668.0345991
#7  event_loop (argc=<value optimized out>, argv=<value optimized out>) at glxgears.c:696
No locals.
#8  main (argc=<value optimized out>, argv=<value optimized out>) at glxgears.c:776
        winWidth = 300
        winHeight = 300
        x = 0
        y = 0
        dpy = 0x606010
        win = 100663298
        ctx = 0x62ec70
        dpyName = <value optimized out>
        printInfo = <value optimized out>
        i = <value optimized out>

(gdb) p state->bo[0]
$1 = (struct r600_bo *) 0x0

(gdb) up
#1  0x00007ffff3e10a23 in evergreen_set_ps_sampler_view (ctx=0x791fe0, count=1, views=0x90ff78) at evergreen_state.c:1068
1068                                    evergreen_context_pipe_state_set_ps_resource(&rctx->ctx, &resource[i]->state,

(gdb) p &((struct r600_pipe_sampler_view **)views[0])->state->bo[0]
$2 = (struct r600_bo **) 0x1500000052

(gdb) up
#2  0x00007ffff408456d in pp_jimenezmlaa_run (ppq=0x791d20, in=0x918a90, out=0x99e540, n=0, iscolor=false) at postprocess/pp_mlaa.c:128
128        cso_set_fragment_sampler_views(p->cso, 1, &p->view);

(gdb) p &((struct r600_pipe_sampler_view **)&p->view[0])->state->bo[0]
$3 = (struct r600_bo **) 0x1500000052

(gdb) p ((struct r600_pipe_sampler_view **)&p->view[0])->state->bo[0]
Cannot access memory at address 0x1500000052

I've updated mesa to Mesa 7.12-devel (git-099faee) and it still happens. But I've found out that it happens while postprocessing the second frame not the first frame (I don't know if that's important).

[curaga]

Do the other filters work properly for you?

[droste]

All the other (pp_nored, pp_noblue, pp_nogreen, pp_celshade and pp_jimenezmlaa_color) are working (working as in do not crash, I'm not sure what the result of pp_jimenezmlaa_color and pp_celshade shoud look like).

[oibaf]

These AA techniques are evolving very fast: about MLAA: http://www.iryoku.com/mlaa/

This technique is outdated, checkout our newer SMAA paper!

SMAA: http://www.iryoku.com/smaa/

[oibaf]

I tried the r300g v2 patch on my RV530 but it crashes with: http://paste.ubuntu.com/678739/ .