NVENC problem with video height not multiple of 4

Accelerated Computing GPU-Accelerated Libraries
1

NVENC has a bug encoding H264 video whose video height is not a multiple of 4. The encoded video has a green line at the bottom of approx 4-6 pixels.

This happens with NV12 input of heights that are 4X + 2.

For example, 300x268 video is encoded properly (no green line), while 300x266 is not (has a green line).

The only workaround that I have found is to truncate the video height to a multiple of 4. The old encoding engine (NVCUVENC) didn’t have this problem.

2

This is caused by not padding the image width to 32 when creating your input buffer and then later creating your chrominance offset into the input buffer. E.g.

#define PAD(v, p) ((v+(p)-1)&(~((p)-1)))
#define PADSIZE 32 // NVENC wants 32 byte alignment

    NV_ENC_CREATE_INPUT_BUFFER nvCreateInput = {
        .version = NV_ENC_CREATE_INPUT_BUFFER_VER,
        .width =  PAD(imagewidth, 32),
        .height = PAD(imageheight, 32),
        .memoryHeap = NV_ENC_MEMORY_HEAP_SYSMEM_CACHED,
        .bufferFmt = videoFormat,
    };

...

    NV_ENC_LOCK_INPUT_BUFFER lock = {
        .version = NV_ENC_LOCK_INPUT_BUFFER_VER,
        .inputBuffer = inputBuffer,
    };
    nvStatus = pEncodeAPI->nvEncLockInputBuffer(hEncoder, &lock);
    if (nvStatus != NV_ENC_SUCCESS) {
        return false;
    }
    
    uint8_t *lumbuf = lock.bufferDataPtr;
    uint8_t *chrbuf = lumbuf + (PAD(imageheight, PADSIZE) * lock.pitch);

Update: Note this padding is conflating pixels and bytes; I can’t find any documentation about which is correct, and don’t have time to test. Caveat emptor.