Encoding YUV420 received from CSI

I plan to implement an FPGA based MIPI CSI-2 video source connected to the Tegra TX2 to encode the video into H264/H265. As the encoder uses 420 input format and the FPGA transmit data rate is limited (so I have to use the least bandwidth intensive format possible) I plan to transmit frames in YUV420 format over MIPI (either legacy or CSPS).
According to the TX1 SoM datasheet (AFAIK TX2 is not available yet) VI can output YUV420 to memory in the exact same format as it was received on MIPI, e.g. U_Y_Y_U_Y_Y… V_Y_Y_V_Y_Y… for the legacy YUV420 MIPI mode. So to be able to use it as an encoder input, it should be converted to I420 or NV12. To me, it seems that VIC may be capable of doing this, but the nvvidconv gstreamer plugin only supports I420, UYVY, and NV12 input formats.
Is there a way to receive YUV420 video on CSI and convert it to planar format for the encoder using HW accelerated conversion?

Hi Tessier
Below gstreamer pipeline already verify with YUV sensor. I think there’s no problem for your case. You can reference to below topic for the YUV sensor relative.

gst-launch-1.0 -v v4l2src device="/dev/video0" ! "video/x-raw,width=1920,height=1080,framerate=60/1, format=string)I420" ! nvvidconv ! "video/x-raw(memory:NVMM)" ! nvoverlaysink

https://devtalk.nvidia.com/default/topic/972192

https://devtalk.nvidia.com/default/topic/976709

https://devtalk.nvidia.com/default/topic/977871

https://devtalk.nvidia.com/default/topic/981601

Hi Shane,

Thank you for the quick response.
Do I understand it correctly, that in your pipeline your v4l2 device outputs I420 when YUV420 is received over MIPI?

What is not clear to me that the Tegra datasheet states that when YUV420 format is received it is stored in memory in the exact byte order it was received over MIPI:

There are two YUV420 versions in MIPI:

So the memory layout written by the Video Input unit (VI3) is not planar and I do not see how to convert it to I420 or NV12 - if I am not mistaken your v4l2 device in the above example pipeline already presents planar I420 frames at the output, and how to do this is not clear to me.

Thanks.

Hi Tessier
Most current use case are sensor output YUV422 and VI output to YUYV, YVYU … then VIC conver as YUV42o or NV12

Hi Shane,

And is it possible to have YUV420 non-planar sensor output converted to planar YUV420 or NV12?

Thanks.

Hi Tessier
Please check below link. YUV420 will drop the UV only keep Y.

https://devtalk.nvidia.com/default/topic/1004084/jetson-tx1/tx1-camera-driver-format-issue/post/5131620/#5131620

Hi Shane,
Thank you, then I will have to use YUV422.

Hi, What format Jetson TX2 encoder support? only YUV420 ? YUV422 / YUV444 can support?

Hi
You can download the document for the detail information.

External Media

Hi ShaneCCC,

Where can I find that document? Do you have a link?

Have a check this
https://docs.nvidia.com/jetson/l4t/index.html