[MMAPI R28.1/R28.1] To reduce DPB delay of NvVideoDecoder

My H.264 parameters for live streaming:

  • Profile: V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE
  • Level: V4L2_MPEG_VIDEO_H264_LEVEL_5_1
  • IDR Interval: 30 (IDRx1, Px29)
  • Max Resolution: 3840x2160
  • Frame Rate: 30 fps

Now I’m debugging:

  • Sometimes a green picture is inserted
  • Sometimes a disorder pictures around it

Refering 00_video_decode sample souce for NvVideoDecoder / NvVideoConverter / EglRenderer / dqEvnet


When not using disableDPB(),
decoded pictures delays 6 - 10 frame time in the DPB (Decoded Picture Buffer: H.264 HRD),
and there are huge jitter after evry IDR decoding.

Because, when a NAL unit of IDR is queued to output plane,
all the decoded pictures in the DPB appear at capture plane with dqCallBack at once,
and the IDR and following several Ps are stored in the DPB.
The next capture_plane.dqBuffer() will be after 200-500msec.

  • This is the operation of standards-compliant.

Number of reference picture is only 1 for our streaming system.
But I cannot see any method to reduce number of stored pictures in the MMAPI.


Using disableDPB(),
the delay from output_plane.qBuff() to capture_plane.dqBuff() becomes very small, 2 - 7 msec.

  • At 4K IDR, it is about 20msec.
    When the capture_plane.dqBuff() is called, is it finished to decode a picture ?

I don’t use B-picture, can I use disableDPB() ?

Best Regards,

Hi mynaemi,
Please share a patch on 01_video_encode and steps so that we can reproduce your issue.

Do yo also hit the issue on r28.2?

Hi DaneLLL,

This topic is about delay of NvVideoDecoder.
I cannot understand the disableDPB() API enough.
I want to reduce the delay of H.264 decoding without any corrupted picture.

From NvVideoDecoder to NvVideoConverter images go via DMABUF,
from NvVideoConverter to EglRenderer images go via fd.
I’m looking for the cause / issue / bug that sometimes corrupted image appears in my code.


Is your comment “share a patch” about my another topic ?

[MMAPI R28.2] deinitPlane() of NvVideoEncoder
https://devtalk.nvidia.com/default/topic/1031348/jetson-tx2/-mmapi-r28-2-deinitplane-of-nvvideoencoder/

It is an issue of R28.2 (not R28.1) that my application for R28.1 could not run on R28.2.
I want to stop and re-configure NvVideoEncoder without delete the object (NvVideoEncoder),
but could not stop by sending EOS to NvVideoEncoder, and it did not return from output_plane.deinitPlane().

  • The cause is that it does not return from output_plane.dqBuffer() by EOS (bytesused=0).

Best Regards,

Hi mynaemi,
We would like to check the issues you are facing:

  • Sometimes a green picture is inserted
  • Sometimes a disorder pictures around it

Are you able to reproduce the issues by applying a patch to 01_video_encode on r28.2(or r28.1)? If yes, please share the patch and steps so that we can check further.

Hi DaneLLL,

Thanks for your support.
But I don’t have enough time to try your suggestion now.


I think that it is not due to encoding, networking nor NAL re-assembling.
If any NAL unit is corrupted, it will collapse bigger than these phenomenon.

  • Sometimes a green picture is inserted
    ----> without any block noise nor any mosquito noise
  • Sometimes a disorder pictures around it
    ----> to display previous one image back

I think that it is before encoding or after decoding.

It is begining, I suspect that the decode delay from output_plane to capture_plane is too small.
I don’t understand why the delay (2-10msec) is smaller than the frame interval (33msec).
About MPEG/HAVC/HEVC, the decoder refers previous images to decode an image.
My application use baseline profile, then it needs one image previous.
When capture_plane.dqBuffer() is called before output_plane.qBuffer() of the next NAL unit,
what does the decoder refer to decode the next NAL unit ?
If the decoder copy a reference image to an internal buffer, I can imagine the phenomenon.

There is another view point, I suspect that renderer timing may have something wrong.
The renderer has one image buffer, wait for frame interval and swap buffers.

NvEglRenderer.cpp

int
NvEglRenderer::render(int fd)
{
    this->render_fd = fd;
    pthread_mutex_lock(&render_lock);
    pthread_cond_broadcast(&render_cond);
    COMP_DEBUG_MSG("Rendering fd=" << fd);
    pthread_cond_wait(&render_cond, &render_lock);
    pthread_mutex_unlock(&render_lock);
    return 0;
}
int
NvEglRenderer::renderInternal()
{
    hEglImage = NvEGLImageFromFd(egl_display, render_fd);
    egl_sync = eglCreateSyncKHR(egl_display, EGL_SYNC_FENCE_KHR, NULL);
    if (last_render_time.tv_sec != 0)
    {
        pthread_mutex_lock(&render_lock);
        last_render_time.tv_sec += render_time_sec;
        last_render_time.tv_nsec += render_time_nsec;
        last_render_time.tv_sec += last_render_time.tv_nsec / 1000000000UL;
        last_render_time.tv_nsec %= 1000000000UL;
        pthread_cond_timedwait(&render_cond, &render_lock, &last_render_time);
        pthread_mutex_unlock(&render_lock);
    }
    eglSwapBuffers(egl_display, egl_surface);
    NvDestroyEGLImage(egl_display, hEglImage);
}

I’m going to try to change this timing by fps setting.

Best Regards,

Hi DaneLLL,

I’m facing these issues with R28.2:
(1) Green picture is inserted (It seems that 1 green picture for 1 GOP)
(2) Disorder picture is seen (back to the previous one)
(3) The upper and lower halves are different picture

I have modified the renderer timing, but the issues did not change.

Next, I tried to change:

  • DPB (Decoded Picture Buffer) management function: Enable / Disable
  • Number of buffers: NvVideoDecoder output plane
  • Number of buffers: NvVideoConverter output / capture plane
  • number of capture plane buffer of decoder is set to the return value of getMinimumCapturePlaneBuffers()

[DPB management: Enable]

  • 10/18/ 4/ 4: Green, Disorder, Upper/Lower
  • 10/29/10/ 4: Green (2min. After starting to decode)
  • 10/18/10/10: (Only Jitter)
  • 10/18/10/10: Green, Disorder, Upper/Lower
  • 10/18/18/10: (Only Jitter)
  • 10/18/18/10: Green, Disorder, Upper/Lower (2min. After starting to decode)
  • 10/18/10/18: Green, Disorder, Upper/Lower
  • 10/18/18/18: Green, Disorder, Upper/Lower (2min. After starting to decode)
  • 16/18/10/18: Green, Disorder, Upper/Lower (2min. After starting to decode)
  • 16/18/16/18: Green, Disorder, Upper/Lower (2min. After starting to decode)
  • 16/18/20/22: Green, Disorder, Upper/Lower
  • 10/18/18/22: Green, Disorder, Upper/Lower
  • 10/24/18/22: Green
  • 10/24/24/26: Green
  • 16/24/24/26: Green
  • Num of buffers: Output Dec/Capture Dec/Output Conv/Capture Conv

[DPB management: Disable] — call disableDPB()

  • 4/18/ 4/ 4: Green
  • 4/18/10/18: Green (2min. After starting to decode)
  • 6/13/ 6/ 6: Green (2min. After starting to decode)
  • 6/13/ 8/10: (none)
  • 6/13/10/12: Green (2min. After starting to decode)
  • 6/13/16/18: (none)
  • 6/13/24/26: (none)
  • 10/18/ 4/ 4: (none)
  • 10/18/10/18: (none)
  • 10/18/10/18: (none)
  • 10/18/10/18: Green (2min. After starting to decode)
  • 10/18/18/18: Green (2min. After starting to decode)
  • 16/18/10/18: (none)
  • 16/18/10/18: (none)
  • 16/18/16/18: (none)
  • Num of buffers: Output Dec/Capture Dec/Output Conv/Capture Conv

4 buffers of output plane of NvVideoDecoder is scarce.
Because NvVideoEncoder sometimes generate 2 set of SPS/PPS, then 5 NAL units will be queued to output plane of NvVideoDecoder in the same time.
(5 NAL units: SPS/PPS/SPS/PPS/IDR) – This is a isssue, I think, but not heavy.

When DPB management is enabled, it is scarce that the number of output plane buffer of NvVideoConverter is less than capture plane of NvVideoDecoder.
Because NvVideoDecoder saves up 6-16 pictures by the DPB management function and releases them at once at SPS/PPS/IDR receiving (queued to output plane).

Observing qBuffer() and dqBuffer() of output plane, when it might be missing, dequeue callback is called immediately after queueing by qBuffer().
I suspect that the management of output plane queue of NvVideoDecoder / NvVideoConverter is something wrong (e.g. overwrite, etc.)

Our system is:
Ethernet → Socket → RTP/NAL re-assemble → NvVideoDecoder → NvVideoConverter → EglRenderer → HDMI Display

Log when DPB management is enabled (excerpt)

1523962066.602568: [DEC]   qbuf output  plane[00] :1523962066.602382 Delay[   0]
1523962066.603057: [DEC]->[CNV] dqbuf / qbuf [11] :1523962066.080982 Delay[ 523]
1523962066.604708: [CNV] Render picture      [10] :1523962066.080982 Delay[ 524] Jitter[ 500]
1523962066.635867: [DEC]   qbuf output  plane[01] :1523962066.635682 Delay[   0]
1523962066.636316: [DEC]->[CNV] dqbuf / qbuf [12] :1523962066.111282 Delay[ 525]
1523962066.638229: [CNV] Skip picture        [11] :1523962066.111282 Delay[ 527] Jitter[ 500]
1523962066.669228: [DEC]   qbuf output  plane[02] :1523962066.668982 Delay[   1]
1523962066.669784: [DEC]->[CNV] dqbuf / qbuf [13] :1523962066.141682 Delay[ 528]
1523962066.671646: [CNV] Render picture      [12] :1523962066.141682 Delay[ 530] Jitter[ 500]
1523962066.702563: [DEC]   qbuf output  plane[03] :1523962066.702382 Delay[   0]
1523962066.702991: [DEC]->[CNV] dqbuf / qbuf [14] :1523962066.174982 Delay[ 528]
1523962066.704840: [CNV] Skip picture        [13] :1523962066.174982 Delay[ 530] Jitter[ 500]
1523962066.735804: [DEC]   qbuf output  plane[04] :1523962066.735682 Delay[   0] --> SPS
1523962066.735877: [DEC]   qbuf output  plane[05] :1523962066.735682 Delay[   0] --> PPS
1523962066.736030: [DEC]   qbuf output  plane[06] :1523962066.735682 Delay[   1] --> IDR
1523962066.736923: [DEC]->[CNV] dqbuf / qbuf [15] :1523962066.208282 Delay[ 528]
1523962066.736966: [DEC]->[CNV] dqbuf / qbuf [16] :1523962066.238582 Delay[ 498]
1523962066.737001: [DEC]->[CNV] dqbuf / qbuf [17] :1523962066.268982 Delay[ 469]
1523962066.737035: [DEC]->[CNV] dqbuf / qbuf [18] :1523962066.302382 Delay[ 435]
1523962066.737069: [DEC]->[CNV] dqbuf / qbuf [19] :1523962066.335682 Delay[ 402]
1523962066.737108: [DEC]->[CNV] dqbuf / qbuf [20] :1523962066.368982 Delay[ 369]
1523962066.737142: [DEC]->[CNV] dqbuf / qbuf [21] :1523962066.402282 Delay[ 335]
1523962066.737176: [DEC]->[CNV] dqbuf / qbuf [22] :1523962066.435682 Delay[ 302]
1523962066.737210: [DEC]->[CNV] dqbuf / qbuf [23] :1523962066.468982 Delay[ 269]
1523962066.737244: [DEC]->[CNV] dqbuf / qbuf [01] :1523962066.502382 Delay[ 235]
1523962066.737277: [DEC]->[CNV] dqbuf / qbuf [02] :1523962066.535682 Delay[ 202]
1523962066.737311: [DEC]->[CNV] dqbuf / qbuf [03] :1523962066.568982 Delay[ 169]
1523962066.737349: [DEC]->[CNV] dqbuf / qbuf [04] :1523962066.602382 Delay[ 135]
1523962066.737383: [DEC]->[CNV] dqbuf / qbuf [05] :1523962066.635682 Delay[ 102]
1523962066.737417: [DEC]->[CNV] dqbuf / qbuf [06] :1523962066.668982 Delay[  69]
1523962066.737452: [DEC]->[CNV] dqbuf / qbuf [07] :1523962066.702382 Delay[  35]
1523962066.739645: [CNV] Render picture      [14] :1523962066.208282 Delay[ 531] Jitter[ 500]
1523962066.769152: [DEC]   qbuf output  plane[07] :1523962066.768982 Delay[   1]
1523962066.770140: [CNV] Skip picture        [15] :1523962066.238582 Delay[ 532] Jitter[ 500]
1523962066.770168: [CNV] Render picture      [16] :1523962066.268982 Delay[ 502] Jitter[ 500]
1523962066.801549: [CNV] Render picture      [17] :1523962066.302382 Delay[ 499] Jitter[ 500]
1523962066.802635: [DEC]   qbuf output  plane[08] :1523962066.802382 Delay[   0]
1523962066.835096: [CNV] Render picture      [18] :1523962066.335682 Delay[ 500] Jitter[ 500]
1523962066.835942: [DEC]   qbuf output  plane[09] :1523962066.835682 Delay[   0]
1523962066.868243: [CNV] Render picture      [19] :1523962066.368982 Delay[ 500] Jitter[ 500]
1523962066.869409: [DEC]   qbuf output  plane[10] :1523962066.868982 Delay[   1]
1523962066.901734: [CNV] Render picture      [20] :1523962066.402282 Delay[ 499] Jitter[ 500]
1523962066.902714: [DEC]   qbuf output  plane[11] :1523962066.902382 Delay[   0]
1523962066.934918: [CNV] Render picture      [21] :1523962066.435682 Delay[ 499] Jitter[ 500]
1523962066.935913: [DEC]   qbuf output  plane[12] :1523962066.935682 Delay[   0]
1523962066.968434: [CNV] Render picture      [22] :1523962066.468982 Delay[ 500] Jitter[ 500]
1523962066.969234: [DEC]   qbuf output  plane[13] :1523962066.968982 Delay[   1]
1523962066.999983: [DEC]   qbuf output  plane[14] :1523962066.999382 Delay[   0]
1523962067.001621: [CNV] Render picture      [23] :1523962066.502382 Delay[ 499] Jitter[ 500]
1523962067.032884: [DEC]   qbuf output  plane[15] :1523962067.032682 Delay[   0]
1523962067.035252: [CNV] Render picture      [24] :1523962066.535682 Delay[ 500] Jitter[ 500]
1523962067.063595: [DEC]   qbuf output  plane[00] :1523962067.063082 Delay[   0]
1523962067.068220: [CNV] Render picture      [25] :1523962066.568982 Delay[ 500] Jitter[ 500]
1523962067.096593: [DEC]   qbuf output  plane[01] :1523962067.096382 Delay[   0]
1523962067.101834: [CNV] Render picture      [00] :1523962066.602382 Delay[ 499] Jitter[ 500]
1523962067.129879: [DEC]   qbuf output  plane[02] :1523962067.129682 Delay[   0]
1523962067.135037: [CNV] Render picture      [01] :1523962066.635682 Delay[ 500] Jitter[ 500]
1523962067.160713: [DEC]   qbuf output  plane[03] :1523962067.160082 Delay[   0]
1523962067.168503: [CNV] Render picture      [02] :1523962066.668982 Delay[ 500] Jitter[ 500]
1523962067.193575: [DEC]   qbuf output  plane[04] :1523962067.193382 Delay[   0]
1523962067.201619: [CNV] Render picture      [03] :1523962066.702382 Delay[ 499] Jitter[ 500]
1523962067.224322: [DEC]   qbuf output  plane[05] :1523962067.223682 Delay[   1]
1523962067.255064: [DEC]   qbuf output  plane[06] :1523962067.253982 Delay[   2]
1523962067.255676: [DEC]->[CNV] dqbuf / qbuf [08] :1523962066.735682 Delay[ 520]
1523962067.257033: [CNV] Skip picture        [04] :1523962066.735682 Delay[ 522] Jitter[ 500]
1523962067.287663: [DEC]   qbuf output  plane[07] :1523962067.287382 Delay[   0]
1523962067.288409: [DEC]->[CNV] dqbuf / qbuf [09] :1523962066.768982 Delay[ 520]
1523962067.289627: [CNV] Render picture      [05] :1523962066.768982 Delay[ 521] Jitter[ 500]
1523962067.321002: [DEC]   qbuf output  plane[08] :1523962067.320682 Delay[   1]
1523962067.321937: [DEC]->[CNV] dqbuf / qbuf [10] :1523962066.802382 Delay[ 519]
1523962067.323285: [CNV] Skip picture        [06] :1523962066.802382 Delay[ 521] Jitter[ 500]
1523962067.351856: [DEC]   qbuf output  plane[09] :1523962067.351082 Delay[   0]
1523962067.353041: [DEC]->[CNV] dqbuf / qbuf [11] :1523962066.835682 Delay[ 518]
1523962067.354226: [CNV] Render picture      [07] :1523962066.835682 Delay[ 519] Jitter[ 500]
1523962067.384673: [DEC]   qbuf output  plane[10] :1523962067.384382 Delay[   0]
1523962067.385518: [DEC]->[CNV] dqbuf / qbuf [12] :1523962066.868982 Delay[ 517]
1523962067.386874: [CNV] Skip picture        [08] :1523962066.868982 Delay[ 518] Jitter[ 500]
1523962067.417953: [DEC]   qbuf output  plane[11] :1523962067.417682 Delay[   0]
1523962067.418743: [DEC]->[CNV] dqbuf / qbuf [13] :1523962066.902382 Delay[ 516]
1523962067.419917: [CNV] Render picture      [09] :1523962066.902382 Delay[ 517] Jitter[ 500]
1523962067.448668: [DEC]   qbuf output  plane[12] :1523962067.448082 Delay[   0]
1523962067.449376: [DEC]->[CNV] dqbuf / qbuf [14] :1523962066.935682 Delay[ 514]
1523962067.450718: [CNV] Skip picture        [10] :1523962066.935682 Delay[ 515] Jitter[ 500]
1523962067.481586: [DEC]   qbuf output  plane[13] :1523962067.481382 Delay[   0]
1523962067.482120: [DEC]->[CNV] dqbuf / qbuf [15] :1523962066.968982 Delay[ 514]
1523962067.483376: [CNV] Render picture      [11] :1523962066.968982 Delay[ 515] Jitter[ 500]
1523962067.512306: [DEC]   qbuf output  plane[14] :1523962067.511782 Delay[   1]
1523962067.512773: [DEC]->[CNV] dqbuf / qbuf [16] :1523962066.999382 Delay[ 513]
1523962067.514095: [CNV] Skip picture        [12] :1523962066.999382 Delay[ 515] Jitter[ 500]
1523962067.545246: [DEC]   qbuf output  plane[15] :1523962067.545082 Delay[   0]
1523962067.545642: [DEC]->[CNV] dqbuf / qbuf [17] :1523962067.032682 Delay[ 513]
1523962067.546824: [CNV] Render picture      [13] :1523962067.032682 Delay[ 514] Jitter[ 500]
1523962067.578544: [DEC]   qbuf output  plane[00] :1523962067.578382 Delay[   0]
1523962067.578945: [DEC]->[CNV] dqbuf / qbuf [18] :1523962067.063082 Delay[ 515]
1523962067.580280: [CNV] Skip picture        [14] :1523962067.063082 Delay[ 517] Jitter[ 500]
1523962067.611835: [DEC]   qbuf output  plane[01] :1523962067.611682 Delay[   0]
1523962067.612204: [DEC]->[CNV] dqbuf / qbuf [19] :1523962067.096382 Delay[ 516]
1523962067.613387: [CNV] Render picture      [15] :1523962067.096382 Delay[ 517] Jitter[ 500]
1523962067.642533: [DEC]   qbuf output  plane[02] :1523962067.642082 Delay[   0]
1523962067.642902: [DEC]->[CNV] dqbuf / qbuf [20] :1523962067.129682 Delay[ 513]
1523962067.644237: [CNV] Skip picture        [16] :1523962067.129682 Delay[ 515] Jitter[ 500]
1523962067.675595: [DEC]   qbuf output  plane[03] :1523962067.675382 Delay[   0]
1523962067.675992: [DEC]->[CNV] dqbuf / qbuf [21] :1523962067.160082 Delay[ 515]
1523962067.677285: [CNV] Render picture      [17] :1523962067.160082 Delay[ 517] Jitter[ 500]
1523962067.708823: [DEC]   qbuf output  plane[04] :1523962067.708682 Delay[   0] ---> SPS
1523962067.708907: [DEC]   qbuf output  plane[05] :1523962067.708682 Delay[   0] ---> PPS
1523962067.709051: [DEC]   qbuf output  plane[06] :1523962067.708682 Delay[   1] ---> IDR
1523962067.709913: [DEC]->[CNV] dqbuf / qbuf [22] :1523962067.193382 Delay[ 516]
1523962067.709959: [DEC]->[CNV] dqbuf / qbuf [23] :1523962067.223682 Delay[ 486]
1523962067.709994: [DEC]->[CNV] dqbuf / qbuf [01] :1523962067.253982 Delay[ 456]
1523962067.710028: [DEC]->[CNV] dqbuf / qbuf [02] :1523962067.287382 Delay[ 423]
1523962067.710061: [DEC]->[CNV] dqbuf / qbuf [03] :1523962067.320682 Delay[ 390]
1523962067.710095: [DEC]->[CNV] dqbuf / qbuf [04] :1523962067.351082 Delay[ 359]
1523962067.710129: [DEC]->[CNV] dqbuf / qbuf [05] :1523962067.384382 Delay[ 326]
1523962067.710169: [DEC]->[CNV] dqbuf / qbuf [06] :1523962067.417682 Delay[ 293]
1523962067.710202: [DEC]->[CNV] dqbuf / qbuf [07] :1523962067.448082 Delay[ 262]
1523962067.710236: [DEC]->[CNV] dqbuf / qbuf [08] :1523962067.481382 Delay[ 229]
1523962067.710270: [DEC]->[CNV] dqbuf / qbuf [09] :1523962067.511782 Delay[ 199]
1523962067.710304: [DEC]->[CNV] dqbuf / qbuf [10] :1523962067.545082 Delay[ 165]
1523962067.710337: [DEC]->[CNV] dqbuf / qbuf [11] :1523962067.578382 Delay[ 132]
1523962067.710371: [DEC]->[CNV] dqbuf / qbuf [12] :1523962067.611682 Delay[  99]
1523962067.710408: [DEC]->[CNV] dqbuf / qbuf [13] :1523962067.642082 Delay[  68]
1523962067.710442: [DEC]->[CNV] dqbuf / qbuf [14] :1523962067.675382 Delay[  35]
1523962067.712066: [CNV] Skip picture        [18] :1523962067.193382 Delay[ 519] Jitter[ 500]
1523962067.712984: [CNV] Render picture      [19] :1523962067.223682 Delay[ 489] Jitter[ 500]
1523962067.741662: [CNV] Render picture      [20] :1523962067.253982 Delay[ 488] Jitter[ 497]
1523962067.742247: [DEC]   qbuf output  plane[07] :1523962067.742082 Delay[   0]
1523962067.768049: [CNV] Render picture      [21] :1523962067.287382 Delay[ 481] Jitter[ 494]
1523962067.775569: [DEC]   qbuf output  plane[08] :1523962067.775382 Delay[   0]
1523962067.801760: [CNV] Render picture      [22] :1523962067.320682 Delay[ 481] Jitter[ 490]
1523962067.809004: [DEC]   qbuf output  plane[09] :1523962067.808782 Delay[   1]
1523962067.834899: [CNV] Render picture      [23] :1523962067.351082 Delay[ 483] Jitter[ 487]
1523962067.842334: [DEC]   qbuf output  plane[10] :1523962067.842082 Delay[   0]
1523962067.868427: [CNV] Render picture      [24] :1523962067.384382 Delay[ 484] Jitter[ 486]
1523962067.875557: [DEC]   qbuf output  plane[11] :1523962067.875382 Delay[   0]
1523962067.901564: [CNV] Render picture      [25] :1523962067.417682 Delay[ 484] Jitter[ 486]
1523962067.908990: [DEC]   qbuf output  plane[12] :1523962067.908782 Delay[   0]
1523962067.935094: [CNV] Skip picture        [00] :1523962067.448082 Delay[ 487] Jitter[ 486]
1523962067.935120: [CNV] Render picture      [01] :1523962067.481382 Delay[ 454] Jitter[ 487]
1523962067.942279: [DEC]   qbuf output  plane[13] :1523962067.942082 Delay[   0]
1523962067.968246: [CNV] Render picture      [02] :1523962067.511782 Delay[ 457] Jitter[ 476]
1523962067.975665: [DEC]   qbuf output  plane[14] :1523962067.975482 Delay[   0]
1523962068.001718: [CNV] Render picture      [03] :1523962067.545082 Delay[ 456] Jitter[ 470]
1523962068.008859: [DEC]   qbuf output  plane[15] :1523962068.008682 Delay[   0]
1523962068.034885: [CNV] Render picture      [04] :1523962067.578382 Delay[ 456] Jitter[ 466]
1523962068.042249: [DEC]   qbuf output  plane[00] :1523962068.042082 Delay[   0]
1523962068.068458: [CNV] Render picture      [05] :1523962067.611682 Delay[ 457] Jitter[ 463]
1523962068.075560: [DEC]   qbuf output  plane[01] :1523962068.075382 Delay[   0]
1523962068.101597: [CNV] Render picture      [06] :1523962067.642082 Delay[ 459] Jitter[ 461]
1523962068.108944: [DEC]   qbuf output  plane[02] :1523962068.108782 Delay[   0]
1523962068.135184: [CNV] Render picture      [07] :1523962067.675382 Delay[ 460] Jitter[ 461]
1523962068.142267: [DEC]   qbuf output  plane[03] :1523962068.142082 Delay[   0]
1523962068.175575: [DEC]   qbuf output  plane[04] :1523962068.175382 Delay[   0]
1523962068.208983: [DEC]   qbuf output  plane[05] :1523962068.208782 Delay[   0]
1523962068.242281: [DEC]   qbuf output  plane[06] :1523962068.242082 Delay[   0]
1523962068.242719: [DEC]->[CNV] dqbuf / qbuf [15] :1523962067.708682 Delay[ 534]
1523962068.244635: [CNV] Render picture      [08] :1523962067.708682 Delay[ 536] Jitter[ 461]
1523962068.275763: [DEC]   qbuf output  plane[07] :1523962068.275482 Delay[   0]
1523962068.276455: [DEC]->[CNV] dqbuf / qbuf [16] :1523962067.742082 Delay[ 534]
1523962068.277828: [CNV] Skip picture        [09] :1523962067.742082 Delay[ 535] Jitter[ 494]
1523962068.309054: [DEC]   qbuf output  plane[08] :1523962068.308782 Delay[   1]
1523962068.309814: [DEC]->[CNV] dqbuf / qbuf [17] :1523962067.775382 Delay[ 534]
1523962068.311069: [CNV] Render picture      [10] :1523962067.775382 Delay[ 536] Jitter[ 500]
1523962068.342230: [DEC]   qbuf output  plane[09] :1523962068.342082 Delay[   0]
1523962068.342594: [DEC]->[CNV] dqbuf / qbuf [18] :1523962067.808782 Delay[ 534]
1523962068.343945: [CNV] Skip picture        [11] :1523962067.808782 Delay[ 535] Jitter[ 500]
1523962068.375662: [DEC]   qbuf output  plane[10] :1523962068.375482 Delay[   0]
1523962068.376121: [DEC]->[CNV] dqbuf / qbuf [19] :1523962067.842082 Delay[ 534]
1523962068.377362: [CNV] Render picture      [12] :1523962067.842082 Delay[ 535] Jitter[ 500]
1523962068.408979: [DEC]   qbuf output  plane[11] :1523962068.408782 Delay[   0]
1523962068.409474: [DEC]->[CNV] dqbuf / qbuf [20] :1523962067.875382 Delay[ 534]
1523962068.410818: [CNV] Skip picture        [13] :1523962067.875382 Delay[ 535] Jitter[ 500]
1523962068.442246: [DEC]   qbuf output  plane[12] :1523962068.442082 Delay[   0]
1523962068.442656: [DEC]->[CNV] dqbuf / qbuf [21] :1523962067.908782 Delay[ 534]
1523962068.443833: [CNV] Render picture      [14] :1523962067.908782 Delay[ 535] Jitter[ 500]
1523962068.475548: [DEC]   qbuf output  plane[13] :1523962068.475382 Delay[   0]
1523962068.475914: [DEC]->[CNV] dqbuf / qbuf [22] :1523962067.942082 Delay[ 533]
1523962068.477251: [CNV] Skip picture        [15] :1523962067.942082 Delay[ 535] Jitter[ 500]
1523962068.508938: [DEC]   qbuf output  plane[14] :1523962068.508782 Delay[   0]
1523962068.509313: [DEC]->[CNV] dqbuf / qbuf [23] :1523962067.975482 Delay[ 534]
1523962068.510495: [CNV] Render picture      [16] :1523962067.975482 Delay[ 535] Jitter[ 500]
1523962068.542235: [DEC]   qbuf output  plane[15] :1523962068.542082 Delay[   0]
1523962068.542597: [DEC]->[CNV] dqbuf / qbuf [01] :1523962068.008682 Delay[ 534]
1523962068.543960: [CNV] Skip picture        [17] :1523962068.008682 Delay[ 535] Jitter[ 500]
1523962068.575634: [DEC]   qbuf output  plane[00] :1523962068.575482 Delay[   0]
1523962068.575982: [DEC]->[CNV] dqbuf / qbuf [02] :1523962068.042082 Delay[ 533]
1523962068.577169: [CNV] Render picture      [18] :1523962068.042082 Delay[ 535] Jitter[ 500]
1523962068.608933: [DEC]   qbuf output  plane[01] :1523962068.608782 Delay[   0]
1523962068.609289: [DEC]->[CNV] dqbuf / qbuf [03] :1523962068.075382 Delay[ 534]
1523962068.610620: [CNV] Skip picture        [19] :1523962068.075382 Delay[ 535] Jitter[ 500]
1523962068.642233: [DEC]   qbuf output  plane[02] :1523962068.642082 Delay[   0]
1523962068.642594: [DEC]->[CNV] dqbuf / qbuf [04] :1523962068.108782 Delay[ 534]
1523962068.643883: [CNV] Render picture      [20] :1523962068.108782 Delay[ 535] Jitter[ 500]
1523962068.675650: [DEC]   qbuf output  plane[03] :1523962068.675482 Delay[   0]
1523962068.676050: [DEC]->[CNV] dqbuf / qbuf [05] :1523962068.142082 Delay[ 534]
1523962068.677393: [CNV] Skip picture        [21] :1523962068.142082 Delay[ 535] Jitter[ 500]
1523962068.708904: [DEC]   qbuf output  plane[04] :1523962068.708782 Delay[   0] --> SPS
1523962068.708967: [DEC]   qbuf output  plane[05] :1523962068.708782 Delay[   0] --> PPS
1523962068.709116: [DEC]   qbuf output  plane[06] :1523962068.708782 Delay[   1] --> IDR
1523962068.709976: [DEC]->[CNV] dqbuf / qbuf [06] :1523962068.175382 Delay[ 534]
1523962068.710018: [DEC]->[CNV] dqbuf / qbuf [07] :1523962068.208782 Delay[ 502]
1523962068.710057: [DEC]->[CNV] dqbuf / qbuf [08] :1523962068.242082 Delay[ 468]
1523962068.710091: [DEC]->[CNV] dqbuf / qbuf [09] :1523962068.275482 Delay[ 435]
1523962068.710125: [DEC]->[CNV] dqbuf / qbuf [10] :1523962068.308782 Delay[ 402]
1523962068.710159: [DEC]->[CNV] dqbuf / qbuf [11] :1523962068.342082 Delay[ 368]
1523962068.710193: [DEC]->[CNV] dqbuf / qbuf [12] :1523962068.375482 Delay[ 335]
1523962068.710227: [DEC]->[CNV] dqbuf / qbuf [13] :1523962068.408782 Delay[ 302]
1523962068.710261: [DEC]->[CNV] dqbuf / qbuf [14] :1523962068.442082 Delay[ 268]
1523962068.710299: [DEC]->[CNV] dqbuf / qbuf [15] :1523962068.475382 Delay[ 235]
1523962068.710333: [DEC]->[CNV] dqbuf / qbuf [16] :1523962068.508782 Delay[ 202]
1523962068.710367: [DEC]->[CNV] dqbuf / qbuf [17] :1523962068.542082 Delay[ 168]
1523962068.710401: [DEC]->[CNV] dqbuf / qbuf [18] :1523962068.575482 Delay[ 135]
1523962068.710434: [DEC]->[CNV] dqbuf / qbuf [19] :1523962068.608782 Delay[ 102]
1523962068.710468: [DEC]->[CNV] dqbuf / qbuf [20] :1523962068.642082 Delay[  68]
1523962068.710502: [DEC]->[CNV] dqbuf / qbuf [21] :1523962068.675482 Delay[  35]
1523962068.712007: [CNV] Render picture      [22] :1523962068.175382 Delay[ 536] Jitter[ 500]
1523962068.738328: [CNV] Skip picture        [23] :1523962068.208782 Delay[ 530] Jitter[ 500]
1523962068.738358: [CNV] Render picture      [24] :1523962068.242082 Delay[ 496] Jitter[ 500]
1523962068.742355: [DEC]   qbuf output  plane[07] :1523962068.742182 Delay[   0]
1523962068.768353: [CNV] Render picture      [25] :1523962068.275482 Delay[ 493] Jitter[ 499]
1523962068.775725: [DEC]   qbuf output  plane[08] :1523962068.775482 Delay[   0]
1523962068.801520: [CNV] Render picture      [00] :1523962068.308782 Delay[ 493] Jitter[ 497]
  • Logging time: DEC/CNV, qBuffer/dqBuffer: Time Stamp (Planned to display), delay, threshold to skip “Jitter”

Log when DPB management is disabled (excerpt)

1523964955.310345: [DEC]   qbuf output  plane[01] :1523964955.310162 Delay[   0]
1523964955.312368: [DEC]->[CNV] dqbuf / qbuf [11] :1523964955.310162 Delay[   2]
1523964955.316051: [CNV] Render picture      [03] :1523964955.276962 Delay[  40] Jitter[  66]
1523964955.343733: [DEC]   qbuf output  plane[02] :1523964955.343562 Delay[   0]
1523964955.345707: [DEC]->[CNV] dqbuf / qbuf [12] :1523964955.343562 Delay[   2]
1523964955.349133: [CNV] Render picture      [04] :1523964955.310162 Delay[  39] Jitter[  66]
1523964955.377012: [DEC]   qbuf output  plane[03] :1523964955.376862 Delay[   1] --> SPS
1523964955.377081: [DEC]   qbuf output  plane[04] :1523964955.376862 Delay[   1] --> PPS
1523964955.377211: [DEC]   qbuf output  plane[05] :1523964955.376862 Delay[   1] --> IDR
1523964955.379566: [DEC]->[CNV] dqbuf / qbuf [01] :1523964955.376862 Delay[   3]
1523964955.382690: [CNV] Render picture      [05] :1523964955.343562 Delay[  39] Jitter[  66]
1523964955.410439: [DEC]   qbuf output  plane[00] :1523964955.410262 Delay[   0]
1523964955.412416: [DEC]->[CNV] dqbuf / qbuf [02] :1523964955.410262 Delay[   2]
1523964955.415840: [CNV] Render picture      [00] :1523964955.376862 Delay[  39] Jitter[  66]
1523964955.443745: [DEC]   qbuf output  plane[01] :1523964955.443562 Delay[   0]
1523964955.445771: [DEC]->[CNV] dqbuf / qbuf [03] :1523964955.443562 Delay[   2]
1523964955.449379: [CNV] Render picture      [01] :1523964955.410262 Delay[  39] Jitter[  66]
1523964955.477059: [DEC]   qbuf output  plane[02] :1523964955.476862 Delay[   1]
1523964955.479135: [DEC]->[CNV] dqbuf / qbuf [04] :1523964955.476862 Delay[   3]
1523964955.482463: [CNV] Render picture      [02] :1523964955.443562 Delay[  39] Jitter[  66]
1523964955.510496: [DEC]   qbuf output  plane[03] :1523964955.510262 Delay[   0]
1523964955.512673: [DEC]->[CNV] dqbuf / qbuf [05] :1523964955.510262 Delay[   2]
1523964955.516044: [CNV] Render picture      [03] :1523964955.476862 Delay[  40] Jitter[  66]
1523964955.543813: [DEC]   qbuf output  plane[04] :1523964955.543562 Delay[   0]
1523964955.546055: [DEC]->[CNV] dqbuf / qbuf [06] :1523964955.543562 Delay[   3]
1523964955.549180: [CNV] Render picture      [04] :1523964955.510262 Delay[  39] Jitter[  66]
1523964955.577261: [DEC]   qbuf output  plane[05] :1523964955.576962 Delay[   1]
1523964955.579699: [DEC]->[CNV] dqbuf / qbuf [07] :1523964955.576962 Delay[   3]
1523964955.582743: [CNV] Render picture      [05] :1523964955.543562 Delay[  39] Jitter[  66]
1523964955.610563: [DEC]   qbuf output  plane[00] :1523964955.610262 Delay[   0]
1523964955.613001: [DEC]->[CNV] dqbuf / qbuf [08] :1523964955.610262 Delay[   3]
1523964955.615847: [CNV] Render picture      [00] :1523964955.576962 Delay[  39] Jitter[  66]
1523964955.643755: [DEC]   qbuf output  plane[01] :1523964955.643562 Delay[   0]
1523964955.645777: [DEC]->[CNV] dqbuf / qbuf [09] :1523964955.643562 Delay[   2]
1523964955.649378: [CNV] Render picture      [01] :1523964955.610262 Delay[  39] Jitter[  66]
1523964955.677242: [DEC]   qbuf output  plane[02] :1523964955.676962 Delay[   1]
1523964955.679691: [DEC]->[CNV] dqbuf / qbuf [10] :1523964955.676962 Delay[   3]
1523964955.682557: [CNV] Render picture      [02] :1523964955.643562 Delay[  39] Jitter[  66]
1523964955.710477: [DEC]   qbuf output  plane[03] :1523964955.710262 Delay[   0]
1523964955.712614: [DEC]->[CNV] dqbuf / qbuf [11] :1523964955.710262 Delay[   2]
1523964955.716031: [CNV] Render picture      [03] :1523964955.676962 Delay[  40] Jitter[  66]
1523964955.743872: [DEC]   qbuf output  plane[04] :1523964955.743662 Delay[   0]
1523964955.745981: [DEC]->[CNV] dqbuf / qbuf [12] :1523964955.743662 Delay[   2]
1523964955.749172: [CNV] Render picture      [04] :1523964955.710262 Delay[  39] Jitter[  66]
1523964955.777174: [DEC]   qbuf output  plane[05] :1523964955.776962 Delay[   1]
1523964955.779227: [DEC]->[CNV] dqbuf / qbuf [01] :1523964955.776962 Delay[   3]
1523964955.782673: [CNV] Render picture      [05] :1523964955.743662 Delay[  39] Jitter[  66]
1523964955.810458: [DEC]   qbuf output  plane[00] :1523964955.810262 Delay[   0]
1523964955.812526: [DEC]->[CNV] dqbuf / qbuf [02] :1523964955.810262 Delay[   2]
1523964955.815884: [CNV] Render picture      [00] :1523964955.776962 Delay[  39] Jitter[  66]
1523964955.843841: [DEC]   qbuf output  plane[01] :1523964955.843662 Delay[   0]
1523964955.845848: [DEC]->[CNV] dqbuf / qbuf [03] :1523964955.843662 Delay[   2]
1523964955.849407: [CNV] Render picture      [01] :1523964955.810262 Delay[  39] Jitter[  66]
1523964955.877151: [DEC]   qbuf output  plane[02] :1523964955.876962 Delay[   1]
1523964955.879167: [DEC]->[CNV] dqbuf / qbuf [04] :1523964955.876962 Delay[   3]
1523964955.882492: [CNV] Render picture      [02] :1523964955.843662 Delay[  39] Jitter[  66]
1523964955.910539: [DEC]   qbuf output  plane[03] :1523964955.910362 Delay[   0]
1523964955.912551: [DEC]->[CNV] dqbuf / qbuf [05] :1523964955.910362 Delay[   2]
1523964955.916061: [CNV] Render picture      [03] :1523964955.876962 Delay[  40] Jitter[  66]
1523964955.943852: [DEC]   qbuf output  plane[04] :1523964955.943662 Delay[   0]
1523964955.945885: [DEC]->[CNV] dqbuf / qbuf [06] :1523964955.943662 Delay[   2]
1523964955.949198: [CNV] Render picture      [04] :1523964955.910362 Delay[  39] Jitter[  66]
1523964955.977188: [DEC]   qbuf output  plane[05] :1523964955.976962 Delay[   1]
1523964955.979353: [DEC]->[CNV] dqbuf / qbuf [07] :1523964955.976962 Delay[   3]
1523964955.982727: [CNV] Render picture      [05] :1523964955.943662 Delay[  39] Jitter[  66]
1523964956.010605: [DEC]   qbuf output  plane[00] :1523964956.010362 Delay[   0]
1523964956.012836: [DEC]->[CNV] dqbuf / qbuf [08] :1523964956.010362 Delay[   2]
1523964956.015851: [CNV] Render picture      [00] :1523964955.976962 Delay[  39] Jitter[  66]
1523964956.043976: [DEC]   qbuf output  plane[01] :1523964956.043662 Delay[   0]
1523964956.046494: [DEC]->[CNV] dqbuf / qbuf [09] :1523964956.043662 Delay[   3]
1523964956.049563: [CNV] Render picture      [01] :1523964956.010362 Delay[  39] Jitter[  66]
1523964956.077314: [DEC]   qbuf output  plane[02] :1523964956.076962 Delay[   1]
1523964956.079941: [DEC]->[CNV] dqbuf / qbuf [10] :1523964956.076962 Delay[   3]
1523964956.082502: [CNV] Render picture      [02] :1523964956.043662 Delay[  39] Jitter[  66]
1523964956.110511: [DEC]   qbuf output  plane[03] :1523964956.110262 Delay[   0]
1523964956.112730: [DEC]->[CNV] dqbuf / qbuf [11] :1523964956.110262 Delay[   2]
1523964956.116081: [CNV] Render picture      [03] :1523964956.076962 Delay[  40] Jitter[  66]
1523964956.143988: [DEC]   qbuf output  plane[04] :1523964956.143662 Delay[   0]
1523964956.146515: [DEC]->[CNV] dqbuf / qbuf [12] :1523964956.143662 Delay[   3]
1523964956.149186: [CNV] Render picture      [04] :1523964956.110262 Delay[  39] Jitter[  66]
1523964956.177245: [DEC]   qbuf output  plane[05] :1523964956.176962 Delay[   1]
1523964956.179623: [DEC]->[CNV] dqbuf / qbuf [01] :1523964956.176962 Delay[   3]
1523964956.182741: [CNV] Render picture      [05] :1523964956.143662 Delay[  39] Jitter[  66]
1523964956.210602: [DEC]   qbuf output  plane[00] :1523964956.210362 Delay[   0]
1523964956.212816: [DEC]->[CNV] dqbuf / qbuf [02] :1523964956.210362 Delay[   2]
1523964956.215895: [CNV] Render picture      [00] :1523964956.176962 Delay[  39] Jitter[  66]
1523964956.243860: [DEC]   qbuf output  plane[01] :1523964956.243662 Delay[   0]
1523964956.245940: [DEC]->[CNV] dqbuf / qbuf [03] :1523964956.243662 Delay[   2]
1523964956.249395: [CNV] Render picture      [01] :1523964956.210362 Delay[  39] Jitter[  66]
1523964956.277150: [DEC]   qbuf output  plane[02] :1523964956.276962 Delay[   1]
1523964956.279208: [DEC]->[CNV] dqbuf / qbuf [04] :1523964956.276962 Delay[   3]
1523964956.282525: [CNV] Render picture      [02] :1523964956.243662 Delay[  39] Jitter[  66]
1523964956.310549: [DEC]   qbuf output  plane[03] :1523964956.310362 Delay[   0]
1523964956.312575: [DEC]->[CNV] dqbuf / qbuf [05] :1523964956.310362 Delay[   2]
1523964956.316056: [CNV] Render picture      [03] :1523964956.276962 Delay[  40] Jitter[  66]
1523964956.343833: [DEC]   qbuf output  plane[04] :1523964956.343662 Delay[   0]
1523964956.345820: [DEC]->[CNV] dqbuf / qbuf [06] :1523964956.343662 Delay[   2]
1523964956.349181: [CNV] Render picture      [04] :1523964956.310362 Delay[  39] Jitter[  66]
1523964956.377128: [DEC]   qbuf output  plane[05] :1523964956.376962 Delay[   1] --> SPS
1523964956.377199: [DEC]   qbuf output  plane[00] :1523964956.376962 Delay[   1] --> PPS
1523964956.377362: [DEC]   qbuf output  plane[01] :1523964956.376962 Delay[   1] --> IDR
1523964956.380035: [DEC]->[CNV] dqbuf / qbuf [07] :1523964956.376962 Delay[   4]
1523964956.382716: [CNV] Render picture      [05] :1523964956.343662 Delay[  39] Jitter[  66]
1523964956.410556: [DEC]   qbuf output  plane[02] :1523964956.410362 Delay[   0]
1523964956.412615: [DEC]->[CNV] dqbuf / qbuf [08] :1523964956.410362 Delay[   2]
1523964956.415880: [CNV] Render picture      [00] :1523964956.376962 Delay[  39] Jitter[  66]
1523964956.443866: [DEC]   qbuf output  plane[03] :1523964956.443662 Delay[   0]
1523964956.445964: [DEC]->[CNV] dqbuf / qbuf [09] :1523964956.443662 Delay[   2]
1523964956.449372: [CNV] Render picture      [01] :1523964956.410362 Delay[  39] Jitter[  66]
1523964956.477302: [DEC]   qbuf output  plane[04] :1523964956.477062 Delay[   0]
1523964956.479531: [DEC]->[CNV] dqbuf / qbuf [10] :1523964956.477062 Delay[   2]
1523964956.482535: [CNV] Render picture      [02] :1523964956.443662 Delay[  39] Jitter[  66]
1523964956.510661: [DEC]   qbuf output  plane[05] :1523964956.510362 Delay[   0]
1523964956.513092: [DEC]->[CNV] dqbuf / qbuf [11] :1523964956.510362 Delay[   3]
1523964956.516137: [CNV] Render picture      [03] :1523964956.477062 Delay[  39] Jitter[  66]
1523964956.544011: [DEC]   qbuf output  plane[00] :1523964956.543662 Delay[   1]
1523964956.546724: [DEC]->[CNV] dqbuf / qbuf [12] :1523964956.543662 Delay[   3]
1523964956.549150: [CNV] Render picture      [04] :1523964956.510362 Delay[  39] Jitter[  66]
1523964956.577380: [DEC]   qbuf output  plane[01] :1523964956.577062 Delay[   0]
1523964956.579953: [DEC]->[CNV] dqbuf / qbuf [01] :1523964956.577062 Delay[   2]
1523964956.582744: [CNV] Render picture      [05] :1523964956.543662 Delay[  39] Jitter[  66]
1523964956.610650: [DEC]   qbuf output  plane[02] :1523964956.610362 Delay[   0]
1523964956.613061: [DEC]->[CNV] dqbuf / qbuf [02] :1523964956.610362 Delay[   3]
1523964956.615915: [CNV] Render picture      [00] :1523964956.577062 Delay[  38] Jitter[  66]
1523964956.644060: [DEC]   qbuf output  plane[03] :1523964956.643762 Delay[   1]
1523964956.646578: [DEC]->[CNV] dqbuf / qbuf [03] :1523964956.643762 Delay[   3]
1523964956.649403: [CNV] Render picture      [01] :1523964956.610362 Delay[  39] Jitter[  66]
1523964956.677286: [DEC]   qbuf output  plane[04] :1523964956.677062 Delay[   0]
1523964956.679481: [DEC]->[CNV] dqbuf / qbuf [04] :1523964956.677062 Delay[   2]
1523964956.682542: [CNV] Render picture      [02] :1523964956.643762 Delay[  39] Jitter[  66]
1523964956.710589: [DEC]   qbuf output  plane[05] :1523964956.710362 Delay[   0]
1523964956.712745: [DEC]->[CNV] dqbuf / qbuf [05] :1523964956.710362 Delay[   2]
1523964956.716068: [CNV] Render picture      [03] :1523964956.677062 Delay[  39] Jitter[  66]
1523964956.743960: [DEC]   qbuf output  plane[00] :1523964956.743762 Delay[   0]
1523964956.746040: [DEC]->[CNV] dqbuf / qbuf [06] :1523964956.743762 Delay[   3]
1523964956.749217: [CNV] Render picture      [04] :1523964956.710362 Delay[  39] Jitter[  66]
1523964956.777255: [DEC]   qbuf output  plane[01] :1523964956.777062 Delay[   0]
1523964956.779293: [DEC]->[CNV] dqbuf / qbuf [07] :1523964956.777062 Delay[   2]
1523964956.782742: [CNV] Render picture      [05] :1523964956.743762 Delay[  39] Jitter[  66]
1523964956.810539: [DEC]   qbuf output  plane[02] :1523964956.810362 Delay[   0]
1523964956.812555: [DEC]->[CNV] dqbuf / qbuf [08] :1523964956.810362 Delay[   2]
1523964956.815847: [CNV] Render picture      [00] :1523964956.777062 Delay[  38] Jitter[  66]
1523964956.843833: [DEC]   qbuf output  plane[03] :1523964956.843662 Delay[   0]
1523964956.845817: [DEC]->[CNV] dqbuf / qbuf [09] :1523964956.843662 Delay[   2]
1523964956.849416: [CNV] Render picture      [01] :1523964956.810362 Delay[  39] Jitter[  66]
1523964956.877241: [DEC]   qbuf output  plane[04] :1523964956.877062 Delay[   0]
1523964956.879249: [DEC]->[CNV] dqbuf / qbuf [10] :1523964956.877062 Delay[   2]
1523964956.882518: [CNV] Render picture      [02] :1523964956.843662 Delay[  39] Jitter[  66]
1523964956.910566: [DEC]   qbuf output  plane[05] :1523964956.910362 Delay[   0]
1523964956.912664: [DEC]->[CNV] dqbuf / qbuf [11] :1523964956.910362 Delay[   2]
1523964956.916077: [CNV] Render picture      [03] :1523964956.877062 Delay[  39] Jitter[  66]
1523964956.943983: [DEC]   qbuf output  plane[00] :1523964956.943762 Delay[   0]
1523964956.946144: [DEC]->[CNV] dqbuf / qbuf [12] :1523964956.943762 Delay[   3]
1523964956.949202: [CNV] Render picture      [04] :1523964956.910362 Delay[  39] Jitter[  66]
1523964956.977345: [DEC]   qbuf output  plane[01] :1523964956.977062 Delay[   0]
1523964956.979629: [DEC]->[CNV] dqbuf / qbuf [01] :1523964956.977062 Delay[   2]
1523964956.982717: [CNV] Render picture      [05] :1523964956.943762 Delay[  39] Jitter[  66]
1523964957.010682: [DEC]   qbuf output  plane[02] :1523964957.010362 Delay[   0]
1523964957.013164: [DEC]->[CNV] dqbuf / qbuf [02] :1523964957.010362 Delay[   3]
1523964957.016064: [CNV] Render picture      [00] :1523964956.977062 Delay[  39] Jitter[  66]
1523964957.044071: [DEC]   qbuf output  plane[03] :1523964957.043762 Delay[   1]
1523964957.046561: [DEC]->[CNV] dqbuf / qbuf [03] :1523964957.043762 Delay[   3]
1523964957.049388: [CNV] Render picture      [01] :1523964957.010362 Delay[  39] Jitter[  66]
1523964957.077274: [DEC]   qbuf output  plane[04] :1523964957.077062 Delay[   0]
1523964957.079328: [DEC]->[CNV] dqbuf / qbuf [04] :1523964957.077062 Delay[   2]
1523964957.082548: [CNV] Render picture      [02] :1523964957.043762 Delay[  39] Jitter[  66]
1523964957.110759: [DEC]   qbuf output  plane[05] :1523964957.110462 Delay[   0]
1523964957.113297: [DEC]->[CNV] dqbuf / qbuf [05] :1523964957.110462 Delay[   3]
1523964957.116104: [CNV] Render picture      [03] :1523964957.077062 Delay[  39] Jitter[  66]
1523964957.144014: [DEC]   qbuf output  plane[00] :1523964957.143762 Delay[   1]
1523964957.146190: [DEC]->[CNV] dqbuf / qbuf [06] :1523964957.143762 Delay[   3]
1523964957.149257: [CNV] Render picture      [04] :1523964957.110462 Delay[  39] Jitter[  66]
1523964957.177279: [DEC]   qbuf output  plane[01] :1523964957.177062 Delay[   0]
1523964957.179383: [DEC]->[CNV] dqbuf / qbuf [07] :1523964957.177062 Delay[   2]
1523964957.182711: [CNV] Render picture      [05] :1523964957.143762 Delay[  39] Jitter[  66]
1523964957.210667: [DEC]   qbuf output  plane[02] :1523964957.210462 Delay[   0]
1523964957.212764: [DEC]->[CNV] dqbuf / qbuf [08] :1523964957.210462 Delay[   2]
1523964957.215894: [CNV] Render picture      [00] :1523964957.177062 Delay[  38] Jitter[  66]
1523964957.243955: [DEC]   qbuf output  plane[03] :1523964957.243762 Delay[   0]
1523964957.245989: [DEC]->[CNV] dqbuf / qbuf [09] :1523964957.243762 Delay[   2]
1523964957.249392: [CNV] Render picture      [01] :1523964957.210462 Delay[  39] Jitter[  66]
1523964957.277236: [DEC]   qbuf output  plane[04] :1523964957.277062 Delay[   0]
1523964957.279235: [DEC]->[CNV] dqbuf / qbuf [10] :1523964957.277062 Delay[   2]
1523964957.282536: [CNV] Render picture      [02] :1523964957.243762 Delay[  39] Jitter[  66]
1523964957.310627: [DEC]   qbuf output  plane[05] :1523964957.310462 Delay[   0]
1523964957.312612: [DEC]->[CNV] dqbuf / qbuf [11] :1523964957.310462 Delay[   2]
1523964957.316109: [CNV] Render picture      [03] :1523964957.277062 Delay[  39] Jitter[  66]
1523964957.343937: [DEC]   qbuf output  plane[00] :1523964957.343762 Delay[   0]
1523964957.345928: [DEC]->[CNV] dqbuf / qbuf [12] :1523964957.343762 Delay[   2]
1523964957.349209: [CNV] Render picture      [04] :1523964957.310462 Delay[  39] Jitter[  66]
1523964957.377303: [DEC]   qbuf output  plane[01] :1523964957.377162 Delay[   0] --> SPS
1523964957.377374: [DEC]   qbuf output  plane[02] :1523964957.377162 Delay[   0] --> PPS
1523964957.377539: [DEC]   qbuf output  plane[03] :1523964957.377162 Delay[   0] --> IDR
1523964957.380312: [DEC]->[CNV] dqbuf / qbuf [01] :1523964957.377162 Delay[   3]
1523964957.382757: [CNV] Render picture      [05] :1523964957.343762 Delay[  39] Jitter[  66]
1523964957.410634: [DEC]   qbuf output  plane[04] :1523964957.410462 Delay[   0]
1523964957.412635: [DEC]->[CNV] dqbuf / qbuf [02] :1523964957.410462 Delay[   2]
1523964957.415855: [CNV] Render picture      [00] :1523964957.377162 Delay[  38] Jitter[  66]
1523964957.443955: [DEC]   qbuf output  plane[05] :1523964957.443762 Delay[   0]
1523964957.446016: [DEC]->[CNV] dqbuf / qbuf [03] :1523964957.443762 Delay[   3]
1523964957.449420: [CNV] Render picture      [01] :1523964957.410462 Delay[  39] Jitter[  66]
1523964957.477287: [DEC]   qbuf output  plane[00] :1523964957.477062 Delay[   0]
1523964957.479447: [DEC]->[CNV] dqbuf / qbuf [04] :1523964957.477062 Delay[   2]
1523964957.482531: [CNV] Render picture      [02] :1523964957.443762 Delay[  39] Jitter[  66]
  • Logging time: DEC/CNV, qBuffer/dqBuffer: Time Stamp (Planned to display), delay, threshold to skip “Jitter”

I’m planning to test MMAPI R28.1 and R27.1 with same code.

Best Regards,

Hi DaneLLL,

Last year, you informed us of the range of number of buffers about NvVideoEncoder.

in multimedia api. enc->output_plane.setupPlane request ten bufs。but return is 6 bufs.
[url]https://devtalk.nvidia.com/default/topic/1027941/[/url]

I tried to get large number (16-24) of buffers with setupPlane(),
then it was allocated (confirmed by getNumBuffer() ),
and it seems that there is no limitation (except Memory implementation amount).

Are there any specification about NvVideoDecoder and NvVideoConverter.

Best Regards,

Hi mynaemi,
Please realize we cannot help further if we cannot reproduce the issue you are facing. Need you to help us reproduce the issue with 01_video_encode.

Hi DaneLLL,

Sorry, I did not think your reproduction environment.

I’ve modified the sample codes:

  • 10_camera_recording
  • 00_video_decode

And made a shell script for test data (h.264 multi-resolution stream).

Summary:
(1) Recoding

  • H.264 Profile: HIGH → BASELINE
  • H.264 Level: 5.0 → 5.1
  • IDR interval: 30
  • Insert SPS/PPS: Enable

(2) Decoding

  • NvBuf Transform API: Not Use
  • Display Window Size: Fix (Resizing by NvVideoConverter)
  • Copy Timestamp: Enable (Decoder → Converter)
  • Add logging

Before timestamp modification, decoded video was normally displayed.
But after copying timestamp from decoder to converter, 3 issues were reproduced.
(Please wait for 20sec-2min from starting a stream)

Best Regards,

20180418_DecConvIssue.zip (471 KB)

Hi mynaemi,
We don’t support dynamic resolution change. You need to re-initialize decoder once the resolution is changed. Is it reproduced in fix resolution?

Hi DaneLLL,

Thanks for your reply.

When the resolution of images is changed,
do I have to delete the object of NvVideoDecoder ?

If it is right, I cannot re-initialize the decoder,
because I can recognize resolution-change from dqEvent of the decoder.
When a dqEvent is informed to user application,
H.264 SPS/PPS/IDR NAL units have already been queued into output plane buffers.
When the decoder is re-initialized, the SPS/PPS/IDR NAL units will be lost.

What do I do, when the resolution is changed ?

Best Regards

Hi mynaemi,
We don’t verify the case that not to delete NvVideoDeocder when resolution is changed. The functionality is not guaranteed.

Hi DaneLLL,

Thanks for your reply.

I have 2 question about resolution-change.

(1) Sample code
I cannot understand why the function query_and_set_capture(ctx) is called twice in the sample code: 00_video_decode.

static void *
dec_capture_loop_fcn(void *arg)
{
    do
    {
        ret = dec->dqEvent(ev, 50000);
                (Abbreviated)
    }
    while ((ev.type != V4L2_EVENT_RESOLUTION_CHANGE) && !ctx->got_error);

    // query_and_set_capture acts on the resolution change event
    if (!ctx->got_error)
        <b>query_and_set_capture(ctx)</b>;

    // Exit on error or EOS which is signalled in main()
    while (!(ctx->got_error || dec->isInError() || ctx->got_eos))
    {
            (Abbreviated)
        // Check for Resolution change again
        ret = dec->dqEvent(ev, false);
        if (ret == 0)
        {
            switch (ev.type)
            {
                case V4L2_EVENT_RESOLUTION_CHANGE:
                    <b>query_and_set_capture(ctx)</b>;
                    continue;
            }
        }

        while (1)
        {
                (Abbreviated)
            // Dequeue a filled buffer
            if (dec->capture_plane.dqBuffer(v4l2_buf, &dec_buffer, NULL, 0))
            {
                (Abbreviated)
            }
        }
    }
        (Abbreviated)
    return NULL;
}

(2) In your comment “re-initialize”
Before “re-initialize”, I think that the decoder has to be stopped and cleared,
then it will be re-configured with new parameters.
NvVideoDecoder engine does not have any STOP function (in MMAPI).
NvVideoEncoder and NvVideoConverter can be stopped by STREAMOFF and deinit the both planes.

I guess that stopping NvVideDecoder needs one of these operations:

  • STREAMOFF and deinit the both plane
  • close device ----> execute it in the destuctor of NvV4l2Element
  • delete NvVideoDecoder object —> call the destructor and close device
  • reboot Linux for Tegra
  • cold start TegraTX2 SoC (with hardware reset operation)
  • Power down and up JetsonTX2

Are there any other method ?

If STREAMOFF and deinit are not sufficient for “re-initialize”,
it needs delete object, reboot Linux or power down JetsonTX2, I think.

Of cource, reboot and power down are senseless specification.

Then I asked you:
[i]>> When the resolution of images is changed,

do I have to delete the object of NvVideoDecoder ?[/i]

In the sample 00_video_decode, deinit function of capture plane of the decoder is called.
If initializing decoder is only once, there is no necessity for deinit,
because it is before setupPlane of capture plane.

Why is there the function call capture_plane.deinit() ?

static void
query_and_set_capture(context_t * ctx)
{
    ret = dec->capture_plane.getFormat(format);
    ret = dec->capture_plane.getCrop(crop);
    if (ctx->conv)
    {
        ret = <b>sendEOStoConverter</b>(ctx);
        ctx->conv->capture_plane.waitForDQThread(2000);
        ctx->conv-><b>output_plane.deinitPlane()</b>;
        ctx->conv-><b>capture_plane.deinitPlane()</b>;
        while(!ctx->conv_output_plane_buf_queue->empty())
        {
            ctx->conv_output_plane_buf_queue->pop();
        }
    }
    if (!ctx->disable_rendering)
    {
        <b>delete ctx->renderer</b>;
        ctx->renderer = NvEglRenderer::createEglRenderer("renderer0", window_width,
                                           window_height, ctx->window_x,
                                           ctx->window_y);
        ctx->renderer->setFPS(ctx->fps);
    }

    // deinitPlane unmaps the buffers and calls REQBUFS with count 0
    <b><u>dec->capture_plane.deinitPlane();</u></b>

    ret = dec->setCapturePlaneFormat(format.fmt.pix_mp.pixelformat,
                                     format.fmt.pix_mp.width,
                                     format.fmt.pix_mp.height);
    ctx->video_height = format.fmt.pix_mp.height;
    ctx->video_width = format.fmt.pix_mp.width;
    // Get the minimum buffers which have to be requested on the capture plane
    ret = dec->getMinimumCapturePlaneBuffers(min_dec_capture_buffers);
    ret = dec->capture_plane.setupPlane(V4L2_MEMORY_MMAP,
                                       min_dec_capture_buffers + 5, false, false);
    if (ctx->conv)
    {
        ret = ctx->conv->setOutputPlaneFormat(format.fmt.pix_mp.pixelformat,
                                              format.fmt.pix_mp.width,
                                              format.fmt.pix_mp.height,
                                              V4L2_NV_BUFFER_LAYOUT_BLOCKLINEAR);
        ret = ctx->conv->setCapturePlaneFormat((ctx->out_pixfmt == 1 ?
                                                    V4L2_PIX_FMT_NV12M :
                                                    V4L2_PIX_FMT_YUV420M),
                                                crop.c.width,
                                                crop.c.height,
                                                V4L2_NV_BUFFER_LAYOUT_PITCH);
        ret = ctx->conv->setCropRect(0, 0, crop.c.width, crop.c.height);
        ret = ctx->conv->output_plane.setupPlane(V4L2_MEMORY_DMABUF,
                                                dec->capture_plane.
                                                getNumBuffers(), false, false);
        ret = ctx->conv->capture_plane.setupPlane(V4L2_MEMORY_MMAP,
                                                 dec->capture_plane.
                                                 getNumBuffers(), true, false);
        ret = ctx->conv->output_plane.setStreamStatus(true);
        ret = ctx->conv->capture_plane.setStreamStatus(true);
        for (uint32_t i = 0; i < ctx->conv->output_plane.getNumBuffers(); i++)
        {
            ctx->conv_output_plane_buf_queue->push(ctx->conv->output_plane. getNthBuffer(i));
        }
        for (uint32_t i = 0; i < ctx->conv->capture_plane.getNumBuffers(); i++)
        {
            ret = ctx->conv->capture_plane.qBuffer(v4l2_buf, NULL);
        }
        ctx->conv->output_plane.startDQThread(ctx);
        ctx->conv->capture_plane.startDQThread(ctx);
    }
    ret = dec->capture_plane.setStreamStatus(true);
    for (uint32_t i = 0; i < dec->capture_plane.getNumBuffers(); i++)
    {
        ret = dec->capture_plane.qBuffer(v4l2_buf, NULL);
    }
    return;
}

Best Regards,

Hi mynaemi,
We don’t test dynamic-resolution change with this sample. After checking your code, you set copy_timestamp=true. Does it work if copy_timestamp=false?

Hi DaneLLL,

Thanks for your reply.

I understood that you had not yet tested the decoder for dyanamic resolution change environment.
But MMAPI R28.1 00_video_decode runs normally with the same multi-resolution test data (made by that script).


The point of this thread is a work around of large decode jitter.
Whenis given the DPB management function is enabled,
the decoder has large (6-10 frames) delay
and immediately after SPS/PPS/IDR the images (pictures) in the DPB buffer are dequeued at once.
Then the delay become the same jitter.

For this reason, I want to use timestamp with each decoded picture.
(1) measure delay and jitter, (2) work around with tiemstamp

In the R28.2 sample code, the condition of timestamp handling is copy_timestamp flag.

if( ctx.copy_timestamp )

And in the R28.2, the v4l2 buffer flag has to be set to true for queueing H.264 NAL unit with timestamp.

v4l2_buf.flags |= V4L2_BUF_FLAG_TIMESTAMP_COPY;

(In the R28.1, when this flag is not set any value, timestamp is handled normally.)


You said:
> Does it work if copy_timestamp=false?

Only for testing, I can set it false.
Also you can, I think, because I already my testing environment at #9 comment.

To modify #if 0 in a funtion set_defaults(), make it, and execute it.

% ./00_video_decode/video_decode H264 --input-nalu --copy-timestamp 12345 30 ./testStream.h264
% ./00_video_decode/video_decode H264 --input-nalu 12345 30 ./testStream.h264

All the timestamps in the log become 0000000000.000000.
Unfortunately, there are still corrupted pictures.


If close v4l2 devide and re-open v4l2 device are needed for changing resolution,
what limitation are there ?
(e.g. - wait for 1sec)

Best Regards,

Hi mynaemi,

from your past comments it seems you are using a baseline profile stream of single reference frame only.
you can call disable dpb for such streams.

Are you facing issues like segmentation fault for decoding DRC (Dynamic Resolution Change) streams ?

Is your primary concern about switching time from one resolution to other resolution ?

There is no need to close and reopen v4l2 device for DRC stream decode. Decoder de-allocate and re-allocate all the buffers on capture plane at the time of resolution change and it will take time to allocate new buffers depending upon resolution of the next stream.

Currently we don’t have the support to use same capture buffers if the resolution and number of reference frames required, of the next stream remains same. Considering that re-allocation will take some time for the new resolution.

Hi Viranjan,

I appreciate your comment, and I’m sorry to too late my reply.

I’m facing these issues:

(1) Encoder: memory leak with dynamic resolution change
(2) Decoder: corrupted images with dynamic resolution change

Your comment was very useful.

There is no need to close and reopen v4l2 device for DRC stream decode. Decoder de-allocate and re-allocate all the buffers on capture plane at the time of resolution change and it will take time to allocate new buffers depending upon resolution of the next stream.

My code about decoding refers sample/00_video_decode, and it executes de-allocateing and re-allocating exept ouput plane of the decoder.

Best Regards

Hi mynaemi,
Can we reproduce corrupted images with default 00_video_decode? How’s the repro rate? Happens in every resolution change?

Hi DaneLLL,

Thanks for your effort.

I have already posted “How to reproduce” at #9 comment by an attached zip file.
(1) Make a multi-resolution stream data by sample/10_camera_recording
(2) Reproduce this issue with sample/00_video_decode
** You can separate (1) and (2), please see shell script in the zip file.

When an event of the resolution change occors, SPS/PPS/IDR are already in the NvVideoDecoder.
Then I cannot reset and re-initialize the NvVideoDecoder.
(Also I cannot de-allocate and re-allocate the output plane of the NvVideoDecoder.)

After several times of resolution-change events, corrupted images appear with every resolution.

This issue is only about R28.2/TX2 (not about R28.1/TX2).
** I have not yet tested R28.2/TX1 and T28.1/TX1.

Best Regards

Hi mynaemi,
Only the first frame is corrupted after the resolution change?

We have tried to reproduce the issue but so far have not seen it. What’s your reproduce rate?