Making sense of timestamps in NVDEC

Hi,

I’m doing HEVC decoding and I’m trying to better understand the timestamps I’m providing and getting out of NVDEC.

I’m providing ascending PTS values at the input, which is what I’m getting out of FFMPEG. However, the values I’m getting at the output are all over the place. Since the input one is marked as PTS, I would expect the timestamps values at the output to be the same… If I do not provide any timestamps at the input, then I’m getting ordered ascending timestamps at the output.

Would it be possible to get more information as to what both timestamp values are supposed to mean and the relationship between the two ?

Thanks,

BasiC

Don’t forget about frame re-ordering for display. If you add timestamps for the input coded frames they will be re-ordered for display.

1 Like

What do you mean exactly? When I set a timestamp in CUVIDSOURCEDATAPACKET then the same value should be in the display callback in CUVIDPARSERDISPINFO for the decided frame of the same packet, right?

Sure but you won’t receive the display callbacks in the same order as the injected packets because coded order and display order are different.

1 Like

Clear, thanks!

Weird thing though. This might help someone too. I am sending packets in the parser with increasing timestamps but it seems like I got them reordered and even the values changed in the display callback. It might be caused by the fact that I was sending the packets in a different order than how they were encoded (yes I intended to do so because of an experiment). It seems like the timestamps in CUVIDSOURCEDATAPACKET can not override some internal timestamps (PTS, DTS?) in the packet data. The only way I managed to get the frames in the same order as they were sent to the decoder was to fetch them in the decode callback instead of the display one.

  1. If you do not send packets in coded order, then operation will be undefined and unpredictable.

  2. There is no PTS/DTS in the elementary stream data that you should be injecting. There are no “internal timestamps” that are returned.

  3. The timestamps you pass are returned transparently.

1 Like
  1. I made sure that there was a correct I-frame decoded before. I know this is not an ordinary use-case but I basically had two videos, both starting with the same I frame and settings but different following frames and I tried to send the packets from both into one common decoder.
  2. Thanks for clarification.
  3. That is strange, I clearly sent a few packets from both videos in the decoder with increasing timestamps in CUVIDSOURCEDATAPACKET and the resulting frames in display callback had the timestamps incorrect. In my test I tried to send I-frame packet A, 3 times the same frame packet from one video B and one packet from another video C. And I got the last frame B switched with C. So I sent this:

timestamp-packet
0-A
1-B
2-B
3-B
4-C

And got back in display callback:
0-A
1-B
2-B
3-C
4-B

But when I fetch the frames from decode callback directly, I get the right order. I think it might be as you said caused by the kind of undefined behavior when using the decoder like this. But it is more like parser-related or rather the display-logic issue.