nvidia mipi YUV setting

obenezra · September 11, 2019, 7:37pm

hi,
we design FPGA with RGB888 and we success do get video over mipi.

now we are trying to get YUV422 format video.

i need some help with format i should enable in camera_common.c
and what i should change in the DTSI file.

thank you
omer.

JerryChang · September 12, 2019, 2:18am

hello obenezra,

may I know which YUV422 formats is your FPGA device outputting?
you may also refer to VI4 drivers as below, there’re several supported YUV formats by default.
for example,

<i>$l4t-r32.2/public_sources/kernel/nvidia/drivers/media/platform/tegra/camera/vi/sensor_common.c</i>
 
static int extract_pixel_format(const char *pixel_t, u32 *format)
{
 ...
	else if (strncmp(pixel_t, "yuv_yuyv16", size) == 0)
		*format = V4L2_PIX_FMT_YUYV;
	else if (strncmp(pixel_t, "yuv_yvyu16", size) == 0)
		*format = V4L2_PIX_FMT_YVYU;
	else if (strncmp(pixel_t, "yuv_uyvy16", size) == 0)
		*format = V4L2_PIX_FMT_UYVY;
	else if (strncmp(pixel_t, "yuv_vyuy16", size) == 0)
		*format = V4L2_PIX_FMT_VYUY;

you might also refer to pixel_phase property in Sensor Software Driver Programming Guide,
please configure the correct settings in your sensor device tree property.

BTW,
you’ll need to bypass ISP to access YUV sources.
here are some examples to access your camera streaming with v4l2 standard controls and v4l2src.

# v4l2 standard controls
$ v4l2-ctl -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat=YUYV --set-ctrl bypass_mode=0 --stream-mmap --stream-count=100
 
# v4l2src
$ gst-launch-1.0 v4l2src num-buffers=1 ! "video/x-raw, format=(string)UYVY, width=(int)1920, height=(int)1080, framerate=(fraction)30/1" ! nvvidconv ! nvjpegenc ! filesink location=cam0.jpg

obenezra · September 12, 2019, 3:17am

We are using yuv 2x8.

What about dtsi.

Pixel_t is on rgb888 should change it?

Thank you

obenezra · September 12, 2019, 3:19am

Where Can i see Pixel_t formats supportted?

obenezra · September 12, 2019, 4:48am

nvidia@nvidia-desktop:~$ gst-launch-1.0 v4l2src num-buffers=1 ! “video/x-raw, format=(string)UYVY, width=(int)1280, height=(int)720, framerate=(fraction)30/1” ! nvvidconv ! nvjpegenc ! filesink location=cam0.jpg
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
ERROR: pipeline doesn’t want to preroll.
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …
nvidia@nvidia-desktop:~$ v4l2-ctl -d /dev/video0 --set-fmt-video=width=1280,height=720,pixelformat=YUYV --set-ctrl bypass_mode=0 --stream-mmap --stream-count=100
VIDIOC_S_FMT: failed: Invalid argument
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.04 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.00 fps
<<<<<<<<<
nvidia@nvidia-desktop:~$

what is this error VIDIOC_S_FMT: failed: Invalid argument?

JerryChang · September 12, 2019, 6:19am

hello obenezra,

please refer to several comments as below, please review your sensor driver and also sensor device tree. thanks

VI driver parse the device tree property and convert it as V4L2 pixel format types,
you may also check below kernel sources about reading device tree property.

<i>$l4t-r32.2/public_sources/kernel/nvidia/drivers/media/platform/tegra/camera/sensor_common.c</i>

static int sensor_common_parse_image_props()
{
	err = of_property_read_string(node, "pixel_t", &temp_str);

2)according to Sensor Software Driver Programming Guide, pixel_t property is deprecated after release l4t-r32.x
hence, please working with following properties instead: mode_type, csi_pixel_bit_depth, and pixel_phase.

due to different pipelines, it’s expected that v4l2src and v4l2 standard control have different results sometimes,
you might also refer to Camera Architecture Stack for details.
your v4l2 standard controls report frame-rate and seems also perform frame capture correctly.
suggest you should check below function to check which i/o controls caused the failures.

<i>$l4t-r32.2/public_sources/kernel/nvidia/drivers/media/platform/tegra/camera/vi/channel.c</i>

static const struct v4l2_ioctl_ops tegra_channel_ioctl_ops = {...}

please update your device tree property and check if v4l2src could also access camera streaming correctly.

obenezra · September 12, 2019, 8:09am

hi,

v4l2src access camera correctly.

i;m still getting the VIDIOC_S_FMT: failed: Invalid argument error

when i’m doing v4l2-compliance it freeze look below

nvidia@nvidia-desktop:~$ v4l2-compliance
v4l2-compliance SHA : not available

Driver Info:
Driver name : tegra-video
Card type : vi-output, crosslink 2-0037
Bus info : platform:15700000.vi:2
Driver version: 4.9.140
Capabilities : 0x84200001
Video Capture
Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04200001
Video Capture
Streaming
Extended Pix Format

Compliance test for device /dev/video0 (not using libv4l2):

Required ioctls:
test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
test second video open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OK

Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
test VIDIOC_LOG_STATUS: OK

Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK
test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 1 Audio Inputs: 0 Tuners: 0

Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported)
test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
test VIDIOC_ENUMAUDOUT: OK (Not Supported)
test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
test VIDIOC_G/S_EDID: OK (Not Supported)

Test input 0:

Control ioctls:
	test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
	test VIDIOC_QUERYCTRL: OK

any suggest?

obenezra · September 12, 2019, 5:49pm

anyone help why the v4l2-compliance freeze when its get to the line

test VIDIOC_QUERYCTRL: OK

?

JerryChang · September 17, 2019, 2:49am

hello obenezra,

shown workable v4l2-compliance results with reference ov5693 camera sensor as below for your reference.

$ v4l2-compliance -d /dev/video0 

v4l2-compliance SHA   : not available

Driver Info:
	Driver name   : tegra-video
	Card type     : vi-output, ov5693 2-0036
	Bus info      : platform:15700000.vi:2
	Driver version: 4.9.140
	Capabilities  : 0x84200001
		Video Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps   : 0x04200001
		Video Capture
		Streaming
		Extended Pix Format

Compliance test for device /dev/video0 (not using libv4l2):

Required ioctls:
	test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
	test second video open: OK
	test VIDIOC_QUERYCAP: OK
	test VIDIOC_G/S_PRIORITY: OK
	test for unlimited opens: OK

Debug ioctls:
	test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported)
	test VIDIOC_LOG_STATUS: OK

Input ioctls:
	test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported)
	test VIDIOC_ENUMAUDIO: OK (Not Supported)
	test VIDIOC_G/S/ENUMINPUT: OK
	test VIDIOC_G/S_AUDIO: OK (Not Supported)
	Inputs: 1 Audio Inputs: 0 Tuners: 0

Output ioctls:
	test VIDIOC_G/S_MODULATOR: OK (Not Supported)
	test VIDIOC_G/S_FREQUENCY: OK (Not Supported)
	test VIDIOC_ENUMAUDOUT: OK (Not Supported)
	test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported)
	test VIDIOC_G/S_AUDOUT: OK (Not Supported)
	Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
	test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported)
	test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported)
	test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported)
	test VIDIOC_G/S_EDID: OK (Not Supported)

Test input 0:

	Control ioctls:
		test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK
		test VIDIOC_QUERYCTRL: OK
		test VIDIOC_G/S_CTRL: OK
		test VIDIOC_G/S/TRY_EXT_CTRLS: OK
		test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK
		test VIDIOC_G/S_JPEGCOMP: OK (Not Supported)
		Standard Controls: 1 Private Controls: 20

	Format ioctls:
		test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK
		fail: v4l2-test-formats.cpp(1184): ret && node->has_frmintervals
		test VIDIOC_G/S_PARM: FAIL
		test VIDIOC_G_FBUF: OK (Not Supported)
		test VIDIOC_G_FMT: OK
		test VIDIOC_TRY_FMT: OK
		test VIDIOC_S_FMT: OK
		test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported)
		test Cropping: OK (Not Supported)
		test Composing: OK (Not Supported)
		test Scaling: OK (Not Supported)

	Codec ioctls:
		test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported)
		test VIDIOC_G_ENC_INDEX: OK (Not Supported)
		test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

	Buffer ioctls:
		test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK
		test VIDIOC_EXPBUF: OK

Test input 0:

Total: 43, Succeeded: 42, Failed: 1, Warnings: 0

following the suggestion in comment#6, please check below function to check which i/o controls caused the freeze.
thanks

$l4t-r32.2/public_sources/kernel/nvidia/drivers/media/platform/tegra/camera/vi/channel.c

static const struct v4l2_ioctl_ops tegra_channel_ioctl_ops = {...}