NvEnc With FFMPEG for transcoding : Closed Caption Text display order issue while using "-bf 2"

I am facing issue while transcoding of ts file which has H264 encoded data with Closed Captions (A53, CEA 608/708) data.

I am having “GeForce GTX 1070”

I am using FFMPEG with “-codec:v h264_nvenc” for transcoding this file, when I don’t have bframes flag. it seems coping Captions properly from source file to destination file but when I use “-bf 2” in encoding params which tells encoder to generate Bframe I P B B P B B etc, it make order of captions text wrong…

Like “Test One” will be displayed like “Tets eOn” or something.

If I use software encoder “-codec:v libx264” it works perfect with “-bf 2” as well but it is NOT working with Nvidia Encoder “h264_nvenc”.

Can you please check and let me know solution for this?

It seems “h264_nvenc” encoder do not handle side data (SEI) provided in input frame while calling “nvEncEncodePicture” function.

Can someone help as I am stuck with this.

Can you please share your input clip along with the exact command line you are using to encode? You can share via Google drive (preferably).

Hello @Abhijit,

First of all, Thank you very much for your reply.

I have incorporate following 2 patches from GIT to copy/pass a53 caption from CUVID decoder to NVidia encoder in ffmpeg.

Decoder Patch:
https://github.com/tea/FFmpeg/commit/ba3df26047ca2e2c2a79c4cab436ace032c73d57

Encoder Patch:
https://github.com/tea/FFmpeg/commit/5112cc90fbffddddbd785564fbc03e1790b23983

Command which I am using with “-bf 2”:

./ffmpeg -analyzeduration 5M -probesize 5M -loglevel 40 -c:v h264_cuvid -i ‘/home/test_files/cuc_ieschool-withscc.ts’
-map 0:v -strict experimental -codec:v h264_nvenc -a53cc 1 -gpu 3 -bf 2 -preset slow -g 15 -r 60000/1001 -threads 0 -profile:v high -level 4.2 -ac 2 -b:v 5800k -bufsize 2200k -minrate 5800k -maxrate 5800k
-c:a copy -flush_packets 0 -metadata service_provider=MyCompany -metadata service_name=TestChannel
-max_muxing_queue_size 400
-f mpegts -mpegts_flags +resend_headers+pat_pmt_at_frames -muxrate 7250000 -pcr_period 20
-y ‘/home/test_files/output_bf2_nvidia.ts’

Output output_bf2_nvidia.ts TS file which has problem in caption order.

Refer below URL to download this output file:
https://drive.google.com/open?id=0B4p5828r07iSbl9wMjM4ZWp5bWM

Refer below URL to download original input file:
https://drive.google.com/open?id=0B4p5828r07iSc1lyR2hObE5Da00

For your info:

This is working perfect if I do not use B frames in output encoded NVidia frames. Like if I remove “-bf 2” from above command it shows proper captions.

Also as mentioned above, If I use software encoder “-codec:v libx264” it works perfect with “-bf 2”.

It seems “-codec:v h264_nvenc” encoder reorder frames as per need but do not handle (reorder) captions (sei data) provided as input with encode function.

Please guide me so that I can move forward using NVidia encoder as I am stuck due to this issue.

Hope to get your reply soon.

Regards,
Hardik

I have one more request. We have fixed a related bug recently. Which driver were you using when you reported the issue?

Can you try with the latest NVIDIA driver?

Hello Abhijit,

Thanks for your reply.

cat /proc/driver/nvidia/version

===========
NVRM version: NVIDIA UNIX x86_64 Kernel Module 381.22 Thu May 4 00:55:03 PDT 2017
GCC version: gcc version 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC)

In which driver you think this issue is fixed??

Can you please share me link to that driver changes.txt?

Hi all

I’m developing a video application under the following environments:

Local PC

  • Ubuntu Desktop 16.04 LTS
  • NVIDIA driver 384.90 (Quadro P2000)

Amazon Cloud EC2 instance

  • Ubuntu Server 16.04 LTS
  • NVIDIA driver 367.57 (GRID K520)

The application re-encodes a RTMP stream using FFmpeg 3.3.4. On both cases the AC53 Closed Captions information is omitted by NVENC because it works with x264 encoder (software encoding).

I also tried @hardik.patel1786 's path on the encoder but is does not work (maybe a compilation flag or encoder param?).

FFmpeg 3.4 does not seems to have this issue fixed on NVENC.

Any ideas?

EDIT: I compile FFmpeg using NVENC’s SDK v5.0.1.

A similar issue was fixed in 387 driver. Can you try it? Linux x64 (AMD64/EM64T) Display Driver | 387.22 | Linux 64-bit | NVIDIA

Sure… Let me try and get back to you with testing results.

Compiling with FFmpeg 3.4 does work either.

Even though, patch from @hardik.patel1786 make the CC button to get enabled but no text appears.

I opened a ticket on FFmpeg to solve this issue here [url]http://trac.ffmpeg.org/ticket/6802[/url]. The issue contains the patch needed to fix the closed caption not appearing issue on FFmpeg 3.3.4 and 3.4 based on @hardik.patel1786 patch for the NVENC encoder.

In my case the closed captions appear as expected on both environments (Local and Amazon EC2)