Multimedia API encoder framerate & VUI

Hi,

I have two problems with the H264/H265 encoders in the Multimedia API. I am on L4T 28.1. Due to product release date it is not feasible to update version right now.

1. Framerate setting
   The setFrameRate function has a numerator and denumerator parameter. If I set them something/1 (e.g. 60/1), the calculated bitrate is correct. However, if I set it to 60000/1000, it seems that the denumerator is not taken into account, and the resulting bitrate is very low. For non-integer framerates (e.g. 59.94 → 60000/1001) this is required.

2. VUI insertion with H264
   When I enable SPS/PPS and VUI insertion, the output elementary stream contains VUI timing information. Unfortunately no matter what framerate I set, I always get the same values:
   num_units_in_tick : 1000
   time_scale : 60000

3. VUI insertion with H265
   SPS/PPS insertion also works with H265, but I have problems with VUI insertion.
   The vui_parameters_present_flag is set to 1, but all VUI parameter flags (e.g. vui_timing_info_present_flag) are set to 0.

Any comment/help is appreciated.

Regards,

Peter

For frame rate issue on r28.1, please try attachment in below link:
[url]https://devtalk.nvidia.com/default/topic/1002773/jetson-tx2/about-the-timestamp-of-video-encoder/post/5208637/#5208637[/url]

We have it fixed on r28.2.

Hi DaneLLL,

I copied libtegrav4l2.so from the attachment in the other thread to /usr/lib/aarch64-linux-gnu/tegra.

When I encode 2000 frames into H264 using

ctx.enc1->setFrameRate(60, 1);
ctx.enc1->setBitrate(4000000);

the resulting file size is 16,340,549 bytes, which is correct.

However, if I encode using

ctx.enc1->setFrameRate(60000, 1000);
ctx.enc1->setBitrate(4000000);

the file size is 2,544,079 bytes.
So the framerate-bitrate problem is not solved.

About VUI.
If I use

ctx.enc1->setFrameRate(40, 1);
ctx.enc1->setBitrate(4000000);

The file size changes correctly to 24,800,684 bytes, but the encoded elementary stream contains

timing_info_present_flag : 1 
num_units_in_tick : 1000 
time_scale : 60000

This is not 40 FPS.

Regards,

Peter

Hi Tessier,
We have confirmed the issues are also present on r28.2.1. Need to do further check.

Hi DaneLLL,

Could you please let me know in this thread when there is new information about this issue?

Regards,

Peter

Hi DaneLLL,

I have the same problem than Tessier.

Is there a workaround for this problem? or a patch?

Thanks,
Regards

Hi DaneLLL,

Like Tessier and AurelienV i have the same issue.

Do you have any update on this problem?

Regards.

Hi,
We are working on it. Please contact your NVIDIA region salesperson to prioritize this issue.

[Out of context]

Please share your location. Are you in Europe or North America?

[Out of context]

Hi AurelienV,

Could you go to NVIDIA.com and then click ‘Contact Us’ in the footer. This will ensure that you get the local version of the link.

Thanks

[Out of context]

Hi,

I checked L4T 32.1 on a TX2. There is definitely an improvement, but it seems to me that that VUI timing parameters are not absolutely correct, the resolution seems to be 0.25 frame/sec:

• FPS setting is 29000/1000 → num_units_in_tick : 1, time_scale : 58 (29 FPS)
• FPS setting is 29100/1000 → num_units_in_tick : 4, time_scale : 232 (29 FPS)
• FPS setting is 29200/1000 → num_units_in_tick : 4, time_scale : 232 (29 FPS)
• FPS setting is 29300/1000 → num_units_in_tick : 4, time_scale : 234 (29.25 FPS)
• FPS setting is 29400/1000 → num_units_in_tick : 4, time_scale : 234 (29.25 FPS)
• FPS setting is 29500/1000 → num_units_in_tick : 4, time_scale : 236 (29.5 FPS)
• FPS setting is 29600/1000 → num_units_in_tick : 4, time_scale : 236 (29.5 FPS)

Also, the encoded file size (thus bitrate) only changes with the above resolution, e.g. the encoded file size is the same for 29000/1000 and 29100/1000 setting, so it seems that the effect is also present is the encoder’s bitrate calculation.

Frame rates below 1 (e.g. 500/1000) are handled incorrectly, the result is 30 FPS - both in VUI parameters and in file size.

Is this a known issue?

Regards

Forgot to mention: the above is valid for H.264.
For H.265 I did not check VUI, but the file size issue is present, so I estimate that bitrate calculation is the same for both encoders.

Hi,
For r32.1, you may check Jetson AGX Xavier/r32.1 patch - eLinux.org

Hi DaneLLL,

I replaced libnvtvmr.so, does not seem to solve the issue.
Please try to run 01_video_encode:
video_encode bunny_i420.bin 1920 1080 H264 bunny.h264 --insert-aud --insert-spspps-idr --insert-vui -fps 29970 1000

In the resulting H.264 file I have:
num_units_in_tick : 4
time_scale : 238

This is definitely not 29.97 fps.

Thanks.

Hi,
Please also share frame rate in ffprobe. The command is at
[url]https://devtalk.nvidia.com/default/topic/1050004/jetson-tx2/encode-h265-vps-does-not-enable-vps_timing_info_present_flag-vui-does-not-enable-timing_info_prese-/post/5329524/#5329524[/url]

Hi,

Stream #0:0: Video: h264 (Constrained Baseline), yuv420p(tv, progressive), 1920x1080, 29.75 fps, 29.75 tbr, 1200k tbn, 59.50 tbc

Hi,
The result looks fine. Should be rounding to 29.75 instead of 29.97. Please try ‘-fps 30000 1001’ and check if it is 29.97.