I am creating an NVENC session with shared direct3dsurface9’s which are registered then mapped.
If I create a single session with my code, all works fine. However if I create a second simultaneous session, using the same code, all creates successfully until I come to register the resources.
nvEncRegisterResource returns NV_ENC_SUCCESS but the returned pointer to the registeredResource is null.
Surely if nvEncRegisterResource fails then it should not return NV_ENC_SUCCESS? Because, unsurprisingly when I then try to map the null resource I then get an error.
I am not sure what else to try now to get it working with more than one session. Any ideas?
A bit more info
I have tried it on a Quadro K3000M and a GTX970 both with the same problem.
I am using the same IDirect3DDevice9Ex to create the Direct3dSurface9 surfaces I am passing the NVENC sessions and also pass this device to both sessions.
My code is a modified version of the H264NVEncoder example.
I’m facing exactly the same problem. I hope an engineer from NVidia can respond to this? It feels like a bug in their code.
In my case it is even not a second simultaneous session; I encode a video fine the first time, and then I destroy all objects. Running the same code the second time crashes. I only have a single session running.
I had the same problem when the source D3D9Ex surface came from NVFBC. By carefully comparing my code to the NVFBC SDK sample code I realized the problem was caused by capturing a frame to the D3D9Ex surface before asking NVENC to register the surface.
Perhaps the issue in other cases is the surface has already been rendered to, or otherwise used.
Update [2017-01-16]: The Capture SDK samples don’t set the NV_ENC_REGISTER_RESOURCE.bufferFormat member. By setting it explicitly to the matching format I avoid the above issue and can capture to the resource before NVENC registers the resource.