About the timestamp of video encoder

Hi, I use the NvVideoEncoder of Tegra_Multimedia_API_R27.1.0, I find the timestamp output from encoder is incorrect.

I have modify 01_video_encode as follows,

In main function:

v4l2_buf.timestamp.tv_sec = timestamp++;
printf(“enc->output_plane.qBuffer timestamp(%d)\n”, v4l2_buf.timestamp.tv_sec);

ret = ctx.enc->output_plane.qBuffer(v4l2_buf, NULL);

In encoder_capture_plane_dq_callback function, I print the timestamp.

static bool
encoder_capture_plane_dq_callback(struct v4l2_buffer *v4l2_buf, NvBuffer * buffer,
NvBuffer * shared_buffer, void *arg)
{
printf(“%s Buffer timestamp(%d)\n”, func, v4l2_buf->timestamp.tv_sec);

...

}

The results are as follows:

vidia@tegra-ubuntu:/mnt/a/jetpack/tegra_multimedia_api/samples/01_video_encode$ ./video_encode /dev/shm/testa265.yuv 1920 1080 H264 out.264
Failed to query video capabilities: Inappropriate ioctl for device
NvMMLiteOpen : Block : BlockType = 4
===== MSENC =====
NvMMLiteBlockCreate : Block : BlockType = 4
875967048
842091865
enc->output_plane.qBuffer timestamp(100000)
===== MSENC blits (mode: 1) into tiled surfaces =====
enc->output_plane.qBuffer timestamp(100001)
encoder_capture_plane_dq_callback Buffer timestamp(0)
encoder_capture_plane_dq_callback Buffer timestamp(0)
enc->output_plane.qBuffer timestamp(100002)
enc->output_plane.qBuffer timestamp(100003)
encoder_capture_plane_dq_callback Buffer timestamp(0)
enc->output_plane.qBuffer timestamp(100004)
encoder_capture_plane_dq_callback Buffer timestamp(0)
enc->output_plane.qBuffer timestamp(100005)
enc->output_plane.qBuffer timestamp(100006)
encoder_capture_plane_dq_callback Buffer timestamp(1)
enc->output_plane.qBuffer timestamp(100007)
encoder_capture_plane_dq_callback Buffer timestamp(1)
enc->output_plane.qBuffer timestamp(100008)
enc->output_plane.qBuffer timestamp(100009)
encoder_capture_plane_dq_callback Buffer timestamp(1)
enc->output_plane.qBuffer timestamp(100010)
encoder_capture_plane_dq_callback Buffer timestamp(2)
enc->output_plane.qBuffer timestamp(100011)
enc->output_plane.qBuffer timestamp(100012)
encoder_capture_plane_dq_callback Buffer timestamp(2)
enc->output_plane.qBuffer timestamp(100013)
encoder_capture_plane_dq_callback Buffer timestamp(2)
enc->output_plane.qBuffer timestamp(100014)
encoder_capture_plane_dq_callback Buffer timestamp(3)
enc->output_plane.qBuffer timestamp(100015)
encoder_capture_plane_dq_callback Buffer timestamp(3)
enc->output_plane.qBuffer timestamp(100016)
encoder_capture_plane_dq_callback Buffer timestamp(3)
enc->output_plane.qBuffer timestamp(100017)
encoder_capture_plane_dq_callback Buffer timestamp(4)
enc->output_plane.qBuffer timestamp(100018)
encoder_capture_plane_dq_callback Buffer timestamp(4)
enc->output_plane.qBuffer timestamp(100019)
encoder_capture_plane_dq_callback Buffer timestamp(4)
enc->output_plane.qBuffer timestamp(100020)
encoder_capture_plane_dq_callback Buffer timestamp(5)
enc->output_plane.qBuffer timestamp(100021)
encoder_capture_plane_dq_callback Buffer timestamp(5)
enc->output_plane.qBuffer timestamp(100022)
encoder_capture_plane_dq_callback Buffer timestamp(5)
enc->output_plane.qBuffer timestamp(100023)

Is there any advice?
Thanks!

Hi zcs,
It is confirmed to be an issue. We will check and fix in coming release.

Could you use v4l2_buf.index for input-output buffer mapping?

Thank you DaneLLL,

I am looking forward this release.

If have B frame, how to get the pts and dts by v4l2_buf.index?

Hi zcs,
By combining tv_sec and tv_usec

@@ -77,7 +77,7 @@ encoder_capture_plane_dq_callback(struct v4l2_buffer *v4l2_buf, NvBuffer * buffe
         abort(ctx);
         return false;
     }
-
+printf("[%d]dqB timestamp [%lds%ldus] \n", v4l2_buf->index, v4l2_buf->timestamp.tv_sec, v4l2_buf->timestamp.tv_usec);
     // GOT EOS from encoder. Stop dqthread.
     if (buffer->planes[0].bytesused == 0)
     {

You can get the order, although the unit is incorrect. For 30fps, please set ‘-fps 300 1’ as a quick workaround.

Hi DaneLLL,

I have tested this issue on L4T R28.1, but it is still not resolved.
When can this issue be fixed?

Confirm the issue is in r28.1. The workaround ‘-fps 300 1’ is still required.

We are tracking the issue but for now there is no schedule yet.

I use the encoder in variational frame rate, so the ‘-fps 300 1’ is unconformable.

Hi zcs,
I think ‘-fps 10 1’ can be good for the case. It makes v4l2_buf->timestamp.tv_sec act like ‘index’ and you can do timestamp mapping.

I am aligned with you that it is uncomfortable, but what I can do is to give a solution on each release. If you are in contact with region salesperson of NVIDIA, please try to go through business process to get this issue prioritized.

Hi DaneLLL,

Thank you for your reply.

I will try your advice.

On r28.1, please apply attached package.
r28_1_encode_timestamp.zip (62.5 KB)

Hi DaneLLL,

Thank you for your relay, I will try this update.

Hi DaneLLL,

It works fine. Thank you very much!

Hi DaneLLL,

In the L4T R28.2 (Multimeida API R28.2),
is this issue fixed?

Yes, please give it a try.

Hi DaneLLL,

Thanks for your quick response.

With only the NvVideoEncoder, does it need ?

v4l2_buf.flags |= V4L2_BUF_FLAG_TIMESTAMP_COPY;

For NvVideoConverter and NvVideoDecoder, it is not required.

*Reference:
https://linuxtv.org/downloads/v4l-dvb-apis/uapi/v4l/buffer.html#buffer-flags
V4L2_BUF_FLAG_TIMESTAMP_COPY 0x00004000
The CAPTURE buffer timestamp has been taken from the corresponding OUTPUT buffer.
This flag applies only to mem2mem devices.

Hi,
You can use either ‘-fps’ to let encoder assign timestamps, or ‘–copy-timestamp’ and set timrstamps by yourself.

Hi DaneLLL,

I appreciate your support.

When viewed at a glance, I cannot comprehend your comment “-fps or --copy-timestamp”.
What you mean is for sample code of MMAPI !

I’ve understood that there is a big difference between samples of R28.1 and R28.2.

set timrstamps by yourself.

Of cource, I have set v4l2_buf.timestamp field, and saw illegal values at the capture plane.

Best Regards,

Hi mynaemi,
Onr28.2, we add the option ‘–copy-timestamp’ in 01_video_encode. Please run the command and check the sample code to understand how it works.

./video_encode test.yuv 1280 720 H264 test.264 --copy-timestamp 14

Sorry,

But, can you patch libtegrav4l2.so for L2T 24.2.1 on TX1 … we can’t update that with J120 Auvidea carryboard … thx

Comment #4 is good for r24.2.1. Please try.
https://devtalk.nvidia.com/default/topic/1002773/jetson-tx2/about-the-timestamp-of-video-encoder/post/5124401/#5124401

[UPDATE]06/25/2018 This is wrong. Is is an issue on r24.2.1