NvBufferTransform not preserving alpha channel on ARGB -> ARGB

Hi,

I have an imaging pipeline that has an input GPU buffer (block linear, ARGB) being converted into an output pitch linear buffer (also ARGB) using NvBufferTransform so that this is done in the hardware. The input buffer has an alpha channel that has information in it that needs to be preserved.

Unexpectedly, when the conversion is done, the input color data is scaled by its alpha channel (as in, blended with black), and the alpha channel is lost, so that it is set always to 0xFF (fully opaque) in the output. NvBufferTransform is not a composition operation, and not a conversion to XRGB, so it doesn’t make sense that alpha is getting lost.

Input buffer:

NvBufferCreateParams input_params = {
        .width = (int32_t)width,
        .height = (int32_t)height,
        .payloadType = NvBufferPayload_SurfArray,
        .memsize = 0,
        .layout = NvBufferLayout_BlockLinear,
        .colorFormat = NvBufferColorFormat_ARGB32,
        .nvbuf_tag = NvBufferTag_VIDEO_CONVERT,
    };

    ret = NvBufferCreateEx (&nvBufBlock, &input_params);
    if (ret)
        qFatal("Unable to create block linear buffer.\n");

Output buffer:

NvBufferCreateParams input_params = {
        .width = (int32_t)width,
        .height = (int32_t)height,
        .payloadType = NvBufferPayload_SurfArray,
        .memsize = 0,
        .layout = NvBufferLayout_Pitch,
        .colorFormat = NvBufferColorFormat_ARGB32,
        .nvbuf_tag = NvBufferTag_VIDEO_CONVERT,
    };

    ret = NvBufferCreateEx (&nvBufPitch, &input_params);
    if (ret)
        qFatal("Unable to create pitch buffer.\n");

Conversion:

const NvBufferRect src_rect = {
        .top = 0,
        .left = 0,
        .width = width,
        .height = height,
    };

    const NvBufferRect dest_rect = src_rect;

    NvBufferTransformParams transform_params = {
        .transform_flag = NVBUFFER_TRANSFORM_FILTER | NVBUFFER_TRANSFORM_FLIP,
        .transform_flip = NvBufferTransform_FlipY,
        .transform_filter = NvBufferTransform_Filter_Smart,
        .src_rect = src_rect,
        .dst_rect = dest_rect,
        .session = NULL,
    };

    // Convert Blocklinear to PitchLinear
    int ret = NvBufferTransform(nvBufBlock, nvBufPitch, &transform_params);
    if (ret == -1)
        qFatal("Transform failed");

If I map both the buffers and memcpy, I can see that the alpha channel is there and correct, but obviously the surface format is wrong. And this is slow, with memcpy being done on the CPU.

Is there a different way to do this conversion correctly that still performs well, so that I can convert between surface formats with the same ARGB format, and preserve the alpha channel?

Kurt

Hi,
This is expected behavior in current implementation. It is a bit strange that input GPU buffer (block linear, ARGB) is with non-0xFF value in alpha channel. Please share a sample that we can reproduce the issue. It is possible the usecase is not tested. We would like to reproduce it and do further check.

Hi Dane,

I have put together an example, and I found that even a pitch → pitch ARGB conversion is failing to preserve the alpha channel. So I’ve updated the title, and made the example work this way since it’s an even more basic case.

The example can be found here:
https://github.com/kekiefer/nvbuffertransform-test

As for the use-case of blocklinear with ARGB, I am using EGL to render to an EGLImage with an alpha channel. I need to download this buffer to the CPU in real time, and using NvBufferTransform handles the conversion on the VIC, which has little CPU overhead and is much faster than glReadPixels.

Thanks,

Kurt

Hi,
Thanks for sharing the test sample. We will check and update.

Hi,
The sample code does not look like a real usecase. It is actually not required to do RGBA(pitch)->RGBA(pitch) conversion. Could you share the sample that sets alpha channel to RGBA(block linear) buffers and then does RGBA(block linear)->RGBA(pitch) conversion?

We think it is expected alpha channel is applied in RGBA(pitch)->RGBA(pitch) conversion.

I have updated the example.

Hi,
With the updated example, it is still not clear why it it required to keep the alpha value in conversion. The example does

sets alpha channel to RGBA(block linear) -> RGBA(block linear) to RGBA(pitch) conversion

However, we actually can do

RGBA(block linear) to RGBA(pitch) conversion -> sets alpha channel to RGBA(pitch)

Could you try to set alpha channel after converting to pitch-linear buffers? Doesn’t seem to have a strong reason to touch alpha channel of block-linear buffers.

Hi Dane,

Only for the sake of the example fill_input_buffer() is being done on the CPU – this buffer would normally be filled by the GPU (renders to EGLImage/nvbuf backed FBO), and in that step the GPU is generating the alpha.

Kurt

Hi,

Please share which APIs are called for this operation. Or you use CUDA code to fill in alpha value of block-linear buffers?

Hi Dane,

I’m using EGL to create an NvBuffer-backed framebuffer, then rendering with GLES. So it goes something like this:

NvBufferCreateEx → NvEGLImageFromFd → glEGLImageTargetRenderbufferStorageOES

Then I use the normal ES drawing commands to fill the buffer.

Kurt

Hi,
We are checking it. Will update once there is progress.

Hi,
For information, please share which release version you are using. The nvbuf_utils.h in the example is not r32.3.1.

I am using 32.3.1

Hi,
Please use attached lib and try again. Before the test, you need to update nvbuf_utils.h to r32.3.1 and modify

#define INPUT_BUFFER_LAYOUT (NvBufferLayout_Pitch)

check_alpha_valid() is only eligible for pitch to pitch conversion.
r32_3_1_TEST_libnvbuf_utils.zip (15 KB)

Dane,

What you have sent does work for pitch->pitch. However, pitch->pitch format conversion isn’t relevant to this problem.

To reiterate, the need is taking a GPU-rendered buffer with alpha (as by EGL/GLES) and converting it into an output pitch linear buffer using NvBufferTransform so that this is done in the hardware. EGL/GLES can’t render to a pitch buffer directly, but if it could that would also solve the problem.

Kurt

Hi,
So your usecase still does not work with attached lib? We would need a sample to run real usecase as requested in #5

I’m afraid there must be a misunderstanding.

To be clear, the update I made to the example in response to #5, converting blocklinear to pitch as the example is right now, is what I need to do.

Hi,
So your usescase is

NvBufferCreate(BLockLinear) -> set alpha channel -> NvBufferTransform(BlockLinear -> Pitch)

It’s confusing to us that BlockLinear is private and we wonder how you set alpha channel to the buffer. Thelogic is your sample does not look like a real usecase.

Like I said, I’m rendering to the buffer with EGL. I just can’t justify putting more time into modifying the example, because I fail to see how the buffer gets filled makes a difference. In the end, a blocklinear buffer gets filled, and it’s the buffer transform from blocklinear to pitch that’s not working as expected.

If the expected behavior is that the alpha channel always gets blended during an nvbuffer transform as if it were a composition operation, then I’ll accept that and move on. I’ll try to use cuda to convert the buffers instead, it looks like this shouldn’t try to do anything like this.

Hi,
We have proposed a solution for r32.3.1. If it doesn’t fit your usecase, please refer to proposal in #7

RGBA(block linear) to RGBA(pitch) conversion -> sets alpha channel to RGBA(pitch)

Probably there is certain issue in our software stacks but it is not easy to investigate further without a sample fitting the real usecase. Once you are able to modify the example, please kindly share it then.