Nvidia Video Codec SDK 8.0 HEVC HDR SEI Message Issues

Hello,

I try to insert HDR SEI messages both for HLG and PQ, and I have some issues detailed below:

HLG SEI Message (alternative_transfer_characteristics = 147)
I can insert this sei message without out problem, if I insert it for every frame. However, If I try insert it, for example every 50th frame, it is not added to the output stream.

PQ SEI Messages (mastering_display_colour_volume = 137, content_light_level_info = 144)
I couldn’t insert either of them. I tried to insert together and separately but, I could not manage to insert them to the output stream, as if they are not supported. Is there limitation for sei message payload type?

Thank you advance for your kind concern.
Salih

Hello Salih,

Can you please specify, how do you add above messages into SEI & check for it’s presence or absence?

I can insert CC SEI messages without a problem to H264 and I try to insert HDR SEI messages for HEVC as detailed below.

For example, PQ SEI messages,

mastering_display display_info;
... (initilaze code)

light_level levels;
... (initialize code)

NV_ENC_SEI_PAYLOAD *sei = new NV_ENC_SEI_PAYLOAD[2];
sei[0].payloadSize	= sizeof(display_info);
sei[0].payloadType	= 137;
sei[0].payload		= new uint8_t;
memcpy(sei[0].payload,&display_info,sizeof(display_info));

sei[1].payloadSize	= sizeof(light_level);
sei[1].payloadType	= 144;
sei[1].payload		= new uint8_t;
memcpy(sei[1].payload,&levels,sizeof(levels));

pic_params.codecPicParams.hevcPicParams.seiPayloadArrayCnt	= 2;
pic_params.codecPicParams.hevcPicParams.seiPayloadArray		= sei;

For a single content_light_level_info SEI message,

light_level levels;
... (initialize code)

NV_ENC_SEI_PAYLOAD *sei = new NV_ENC_SEI_PAYLOAD;

sei->payloadSize	= sizeof(light_level);
sei->payloadType	= 144;
sei->payload		= new uint8_t;
memcpy(sei->payload,&levels,sizeof(levels));

pic_params.codecPicParams.hevcPicParams.seiPayloadArrayCnt	= 1;
pic_params.codecPicParams.hevcPicParams.seiPayloadArray		= sei;

I check them by parsing encoded bitstream.

Now, I manually insert them into to the received encoded bitstream, therefore I have to parse encoded bitstream for insertion point and make extra copy operations. It would be good, if I receive them in the encoded bitstream from encoder.

Salih,
Thanks for the quick reply. Can you also specify versions of SDK & driver, and what OS are you using.

Hi,

Thank you for your kind concern.

SDK Ver: 8.0.14
Driver: 388.13
OS: Windows 8.1 Pro

Salih,
One more question - what GPU are you using? And please update to the latest driver to check if this issue reproduces on most recent driver release.

Hi,

I tested on GeForce GTX 1060 (Pascal). I will test it with 391.01 soon.

Hello,

I tested with driver ver. 391.01 and the problems still remain as I described in my first post.

Hello Salih,

You should be able to pass any SEI message explicitly using NV_ENC_PIC_PARAMS_HEVC structure defined at nvEncodeAPI.h header which is part of Video Codec SDK samples:

uint32_t seiPayloadArrayCnt;                         /**< [in]: Specifies the number of elements allocated in  seiPayloadArray array. */
uint32_t reserved;                                   /**< [in]: Reserved and must be set to 0. */
NV_ENC_SEI_PAYLOAD* seiPayloadArray;                 /**< [in]: Array of SEI payloads which will be inserted for this frame. */

Hello,

I tried to pass them as code blocks above. However for HDR10 SEI messages, the output bitsream does not have any; for HLG sei messages, the output bitstream has them, if I add it for every frame.

Hello Salih,

SEI messages 137 & 144 have CVS & CLVS persistence scope, they can be inserted only to IDR frames. For non-IDR they are filtered out.

Hi Salih,

The persistence scope for payload type 147 is also the entire sequence, so it should be inserted into the bitstream only at IDR frames. The driver had a bug, due to which you were able to insert messages of that payload type for every frame. The fix will be available in future driver versions.

You said that you were not able to see the SEI messages being inserted, if specified at certain intervals. Can you provide a standalone application which demonstrates that?

Hi Salih,
Have you insert the 137 type sei messade into the bitstream? The PQ sei message data pasred from the bitstream is wrong when I use nvenc encoder.

Hello All,

Sorry for my late reply.

I manually insert them into to the received encoded bitstream, therefore I could not make any further tests but I will make more tests as soon as possible.

Picture type decision is made by encoder (enablePTD == 1) threfore, should I use “NV_ENC_PIC_FLAG_FORCEIDR” flag when I want to insert 137, 144 and 147 sei messages?

Yes, forcing the frame to be IDR is the best option, unless these messages are already available at the start of your stream.

Actually, “They should be written for every b_keyframe non matter if it is IDR or not. Otherwise if you encode open-gop or PIR stream and then cut few GOPs from start than stream will start without this metadata.”

And yeah, x264 now supports HDR10 meta!