Nvidia Video Codec SDK 8.0 HEVC add custom SEI with payloadType=5

Dear NVIDIA experts,

I’m trying to add sei message for HEVC with payloadType=5 (according to user_data_unregistered in spec) but failed.
My step is:

  1. create a NV_ENC_SEI_PAYLOAD structure and allocate memory;
    NV_ENC_SEI_PAYLOAD *psei_payload;
    psei_payload = (NV_ENC_SEI_PAYLOAD *)malloc (sizeof (NV_ENC_SEI_PAYLOAD));
  2. allocate sei data buffer:
    #define USER_DATA_SIZE 100
    uint8_t *psei_data;
    psei_data = (uint8_t *)malloc (USER_DATA_SIZE);
  3. fill in payload and give it to pic_params.codecPicParams.h264PicParams.seiPayloadArray:
    psei_payload->payloadSize = USER_DATA_SIZE;
    psei_payload->payloadType = 5;
    psei_payload->payload = psei_data;
    pic_params.codecPicParams.hevcPicParams.seiPayloadArray = psei_payload;
  4. set seiPayloadArrayCnt to 1:
    pic_params.codecPicParams.hevcPicParams.seiPayloadArrayCnt = 1;
  5. finally start encoding
    nv_status = p_nvenc->nvEncEncodePicture(ctx->nvencoder, &pic_params);

The return status is NV_ENC_SUCCESS, but no sei message in the stream. The test is based on ffmpeg, I’ve modify the code libavcodec/nvenc.c.

Did I miss some settings? Is there any example for adding sei message? I didn’t find it in SDK samples.

Thanks and Best Regards,

Duplicate of https://devtalk.nvidia.com/default/topic/976304/video-technologies/nvidia-video-codec-sdk-7-0-hevc-custom-sei-support-problem/

…codecPicParams.hevcPicParams.seiPayloadArray != …codecPicParams.h264PicParams.seiPayloadArray ?

Thanks for your help. It is my mistake when post the topic, and this mistask is not in the code. I’ve print out the content of “pic_params.codecPicParams.hevcPicParams.seiPayloadArray” before nvEncEncodePicture and it looks alright.
The duplicated issue looks the same with mine, it’s a pity that the owner didn’t give an answer.

My previous test is based on the code in ffmpeg, I’m going to have a try on sample encoder. Below is the part of my test code in libavcode/nvenc.c:

for (int m=0;m<USER_DATA_SIZE;m++)
        {
            ctx->psei_data[m] = (m+100) & 0xff;
        }

        ctx->psei_payload->payloadSize = USER_DATA_SIZE;
        ctx->psei_payload->payloadType = 5;
        ctx->psei_payload->payload = ctx->psei_data;

        switch (avctx->codec->id) {
        case AV_CODEC_ID_H264:
          pic_params.codecPicParams.h264PicParams.seiPayloadArray = ctx->psei_payload;
          pic_params.codecPicParams.h264PicParams.seiPayloadArrayCnt = 1;
          break;
        case AV_CODEC_ID_H265:
          pic_params.codecPicParams.hevcPicParams.seiPayloadArray = ctx->psei_payload;
          pic_params.codecPicParams.hevcPicParams.seiPayloadArrayCnt = 1;
          printf("insert hevc sei, %d, %p : %d, %d, %p\n", pic_params.codecPicParams.hevcPicParams.seiPayloadArrayCnt, pic_params.codecPicParams.hevcPicParams.seiPayloadArray, pic_params.codecPicParams.hevcPicParams.seiPayloadArray->payloadSize, pic_params.codecPicParams.hevcPicParams.seiPayloadArray->payloadType, pic_params.codecPicParams.hevcPicParams.seiPayloadArray->payload);
          break;
        default:
          av_log(avctx, AV_LOG_ERROR, "Unknown codec name\n");
          return AVERROR(EINVAL);
        }
        nv_status = p_nvenc->nvEncEncodePicture(ctx->nvencoder, &pic_params);

The debug print is:
insert hevc sei, 1, 0x2ab3a20 : 100, 5, 0x2aeb0a0

I’ve reproduce this issue with Video_Codec_SDK_7.1.9, in sample code NvEncoder. The code works for h264 encode, but nothing changed in hevc. Here is more info:

  1. Driver version
    Driver Version: 375.39

  2. SDK version
    Video_Codec_SDK_7.1.9

  3. Application you are using
    Almost the same with Video_Codec_SDK_7.1.9 sample code. The only changed file is: Video_Codec_SDK_7.1.9/Samples/common/src/NvHWEncoder.cpp
    Please download here, or just copy the code below and put it in the sample code common/src/NvHWEncoder.cpp.

    All sample code: Samples_Video_Codec_SDK_7.1.9_forHEVC_SEI.zip_免费高速下载|百度网盘-分享无限制
    Modified file ‘common/src/NvHWEncoder.cpp’ NvHWEncoder.cpp_免费高速下载|百度网盘-分享无限制

  4. Exact sequence of steps (with source code) in the application
    The test is based on sample code NvEncoder. I’ve add the code below in NvHWEncoder.cpp, in function NvEncEncodeFrame() before nvEncEncodePicture() is called.

#if 1 //add sei
    printf("try to add sei with payloadType 5\n");
#define USER_DATA_SIZE 100

    NV_ENC_SEI_PAYLOAD *psei_payload = (NV_ENC_SEI_PAYLOAD *)malloc (sizeof (NV_ENC_SEI_PAYLOAD));
    uint8_t *psei_data = (uint8_t *)malloc (USER_DATA_SIZE);
    for (int i=0;i<USER_DATA_SIZE;i++)
    {
        psei_data[i] = (i+100) & 0xff;
    }

    psei_payload->payloadSize = USER_DATA_SIZE;
    psei_payload->payloadType = 5;
    psei_payload->payload = psei_data;

    if (codecGUID != NV_ENC_CODEC_HEVC_GUID)
    {
        encPicParams.codecPicParams.h264PicParams.seiPayloadArray = psei_payload;
        encPicParams.codecPicParams.h264PicParams.seiPayloadArrayCnt = 1;
        printf("set for h264\n");
    }
    else
    {
        encPicParams.codecPicParams.hevcPicParams.seiPayloadArray = psei_payload;
        encPicParams.codecPicParams.hevcPicParams.seiPayloadArrayCnt = 1;
        printf("set for hevc\n");
    }
#endif

Just hard code to test the issue, undefine the ‘#if 1’ for none SEI encoding and compare the output streams. Looks like it works only for h264, not for hevc. The encoded hevc stream has nothing changed.

My test command is:
./NvEncoder -i in_3840x2160.yuv -size 3840 2160 -inputFormat 0 -codec 1 -o tmp1.hevc

Hello,

I am facing similar issue, I am passing captions while transcoding H264.

Captions works perfect without “-bf 2” flag in ffmpeg command. I am using nvenc in encoding.
But when I use “-bf 2” which enables encoder to generate bframes, captions order became wrong.

Like “Hardik Patel” display like “Harikd Paelt”.

Can someone please help, how to solve this?