Cuda Error when setMaxTraceDepth is too small in OptiX 6.0

Hello everyone,

Additionally to one problem I have with turning off RTX mode (see other topic), I’m having an issue with RTX mode enabled.

Version is OptiX 6.0, Windows 10, GTX 1080 with the latest Studio Driver and Cuda 10.1.

I’m often getting the following error:

OptiX Error: 'Unknown error (Details: Function "_rtContextLaunch2D" caught exception: Encountered a CUDA error: cudaDriver().CuEventSynchronize( m_event ) returned (719): Launch failed)'

And tracked it down to setting a too small trace depth.

if( tutorial_number < 8 )
        context->setMaxTraceDepth( 5 );
    else
        context->setMaxTraceDepth( 5 );

In the tutorialSample, example 10, if I change the maxTraceDepth to 5 as shown, the program crashes with above error after throwing a couple of stack overflow exceptions:

...
Caught exception 0x3FC at launch index (622,45)
Caught exception 0x3FC at launch index (623,45)
Caught exception 0x3FC at launch index (621,46)
Caught exception 0x3FC at launch index (622,46)
Caught exception 0x3FC at launch index (623,46)
Caught exception 0x3FC at launch index (622,47)
Caught exception 0x3FC at launch index (623,47)
Caught exception 0x3FC at launch index (599,55)
Caught exception 0x3FC at launch index (600,55)
OptiX Error: 'Unknown error (Details: Function "_rtContextLaunch2D" caught exception: Encountered a CUDA error: cudaDriver().CuEventSynchronize( m_event ) returned (719): Launch failed)'

For a depth of e.g. 10, a lot of exceptions get thrown (as expected) but the actual program doesn’t crash.

This seems to be related to the previous topic by another user here:
https://devtalk.nvidia.com/default/topic/1048561/optix/problems-with-setmaxtracedepth/,
but sadly that never got resolved.

Is there any way to prevent this hard crash?

Thanks in advance,
David

Hi @david.gilbert,

The main way to prevent both stack overflow exceptions (0x3FC) as well as hard crashes is to ensure and guarantee your renderer uses fewer stack frames than the trace depth that you set.

If you are doing recursive ray tracing or path tracing, the most common approach we recommend is to not call rtTrace() from a closest-hit shader, and instead put secondary rtTrace() calls in your raygen shader. This way you can avoid needing extra trace depth even for longer ray paths.

Is that general approach helpful for you, or do you require a larger stack depth for other reasons?

–
David.

Hi!

Thank you very much for the quick answer. I kind of inherited the shader code from a previous master student, and was hoping that I wouldn’t have to rewrite them for OPtiX 6.0.
The whole program is a lens/laser simulation that renders both the view through the lenses and an actual “Laser” trace through them in VR. After digging a bit I suspect the crashes come from the rare case of total internal reflection, which probably pushes the trace calls over the limit.
I should be able to just keep track of the depth and stop the trace before the limit gets exceeded with a simple check.

Moving the whole thing into the raygen shader is an interesting approach I haven’t thought about, thanks a lot for that suggestion. I’ll look into that and see if it’s viable.