TX2 Dev Kit, V4L2 and OV5693 camera

XavierR · January 23, 2018, 4:34pm

Hello,

We are encountering a lot of issues while using the Jetson TX2 Dev Kit embedded camera (ov5693) with Video4Linux (V4L2).

First of all, here is a summary of devices detection by V4L and the supported formats for /dev/video0

nvidia@tegra-ubuntu:~$ v4l2-ctl --list-devices
VIDIOC_QUERYCAP: failed: Inappropriate ioctl for device
VIDIOC_QUERYCAP: failed: Inappropriate ioctl for device
vi-output, ov5693 2-0036 (platform:15700000.vi:2):
	/dev/video0
	/dev/v4l-subdev1
	/dev/v4l-subdev0

nvidia@tegra-ubuntu:~$ v4l2-ctl -d /dev/video0 --list-formats
ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'RGGB'
	Name        : 8-bit Bayer RGRG/GBGB

	Index       : 1
	Type        : Video Capture
	Pixel Format: 'RG10'
	Name        : 10-bit Bayer RGRG/GBGB

	Index       : 2
	Type        : Video Capture
	Pixel Format: 'BG10'
	Name        : 10-bit Bayer BGBG/GRGR

	Index       : 3
	Type        : Video Capture
	Pixel Format: 'RG12'
	Name        : 12-bit Bayer RGRG/GBGB

We tried camera acquisition with v4l2-ctl utility for each supported format.
Here are the corresponding test results:

nvidia@tegra-ubuntu:~$ v4l2-ctl --set-fmt-video=width=2592,height=1944,pixelformat=RGGB --stream-mmap --stream-count=5 -d /dev/video0 --stream-to=ov5693.raw
<< 1.00 fps
< 0.99 fps
< 0.99 fps
< 0.99 fps

nvidia@tegra-ubuntu:~$ v4l2-ctl --set-fmt-video=width=2592,height=1944,pixelformat=RG10 --stream-mmap --stream-count=5 -d /dev/video0 --stream-to=ov5693.raw
<<<<<

nvidia@tegra-ubuntu:~$ v4l2-ctl --set-fmt-video=width=2592,height=1944,pixelformat=BG10 --stream-mmap --stream-count=5 -d /dev/video0 --stream-to=ov5693.raw
<<<<<

nvidia@tegra-ubuntu:~$ v4l2-ctl --set-fmt-video=width=2592,height=1944,pixelformat=RG12 --stream-mmap --stream-count=5 -d /dev/video0 --stream-to=ov5693.raw
<< 1.00 fps
< 1.00 fps
< 0.99 fps
< 0.99 fps

It appears that acquisition seems to work only for 10bits Bayer formats. With RGGB and RG12, the camera delivers less than 1FPS.

Last and not the least, the image size and bytes per line values provided by v4l2-ctl seems quite strange.
(RGGB is supposed to be 8bits Bayer but the bytes per line is more than a 16bits image):

nvidia@tegra-ubuntu:~$ v4l2-ctl --all
Driver Info (not using libv4l2):
	Driver name   : tegra-video
	Card type     : vi-output, ov5693 2-0036
	Bus info      : platform:15700000.vi:2
	Driver version: 4.4.38
	Capabilities  : 0x84200001
		Video Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps   : 0x04200001
		Video Capture
		Streaming
		Extended Pix Format
Priority: 2
Video input : 0 (Camera 2: no power)
Format Video Capture:
	Width/Height      : 2592/1944
	Pixel Format      : 'RGGB'
	Field             : None
	Bytes per Line    : 5376
	Size Image        : 10450944
	Colorspace        : sRGB
	Transfer Function : Default
	YCbCr Encoding    : Default
	Quantization      : Default
	Flags             : 

Camera Controls

                   frame_length (int)    : min=0 max=32767 step=1 default=1984 value=1984 flags=slider
                    coarse_time (int)    : min=2 max=32761 step=1 default=1978 value=1978 flags=slider
              coarse_time_short (int)    : min=2 max=32761 step=1 default=1978 value=1978 flags=slider
                     group_hold (intmenu): min=0 max=1 default=0 value=0
                     hdr_enable (intmenu): min=0 max=1 default=0 value=0
                       otp_data (str)    : min=0 max=1024 step=2 value='93b28b1382215ebc0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000570000000000000000001a0000000000' flags=read-only, has-payload
                        fuse_id (str)    : min=0 max=16 step=2 value='93b28b1382215ebc' flags=read-only, has-payload
                           gain (int)    : min=256 max=4096 step=1 default=256 value=256 flags=slider
                    bypass_mode (intmenu): min=0 max=1 default=0 value=1
                override_enable (intmenu): min=0 max=1 default=0 value=1
                   height_align (int)    : min=1 max=16 step=1 default=1 value=1
                     size_align (intmenu): min=0 max=2 default=0 value=0
               write_isp_format (int)    : min=1 max=1 step=1 default=1 value=1
       sensor_signal_properties (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
        sensor_image_properties (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
      sensor_control_properties (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
              sensor_dv_timings (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
                   sensor_modes (int)    : min=0 max=30 step=1 default=30 value=3 flags=read-only

Would you have any clues or V4L2 working code for the OV5693 camera provided with the dev kit?

Thanks and best regards,
Xavier

ShaneCCC · January 24, 2018, 3:43am

It’s know issue for report not support pixel format. it’s will fixed by next release.
This sensor should only have BG10 support.

XavierR · January 24, 2018, 4:13pm

Hi Shane,

Thanks for the information. Do you have an estimated release date for this fix?

Is there also an issue with the reported image size?
Actually, if we try to acquire BG10 images (10 bits per pixel stored on 16bits words), the v4l2-ctl utility reports:

nvidia@tegra-ubuntu:~$ v4l2-ctl -d /dev/video0 --all
...
Format Video Capture:
	Width/Height      : 2592/1944
	Pixel Format      : 'BG10'
	Field             : None
	Bytes per Line    : 5376
	Size Image        : 10450944
...

We were expecting a different “bytes per line” value 5184 bytes (=16bit*2592) rather than 5376 bytes.
Just to be sure, does it means that there are 192 unused bytes per line?

Thanks and best regards,
Xavier

ShaneCCC · January 25, 2018, 2:26am

The size problem is cause by the 256 alignment. You can modify the TEGRA_STRIDE_ALIGNMENT in …/vi/core.h to change it.

XavierR · February 1, 2018, 11:13am

Thanks.
We managed to retrieve the “width step” from the V4L API so we can now acquire images from the on-board CSI camera.

Unfortunately the acquisition seems to work on L4T28.2 but not on L4T28.1 (it seems that V4L support was introduced with L4T28.1 according to release notes).

Also, considering the provided image resolution and depth CPU processing is really sub-optimal even for simple display.

To optimize the acquisition pipeline, we had a look to the Multimedia API documentation.
Unfortunately, we did not find any useful information regarding CSI camera capture/display/processing.
Most of the time the documentation refers to YUV / USB cameras and we did not find a working sample for the on-board CSI camera.
Sample “09_camera_jpeg_capture” just work few seconds before crashing with the following output:

nvidia@tegra-ubuntu:~/tegra_multimedia_api/samples/09_camera_jpeg_capture$ ./camera_jpeg_capture 
[INFO] (NvEglRenderer.cpp:109) <renderer0> Setting Screen width 640 height 480
OFParserGetVirtualDevice: virtual device driver node not found in proc device-tree
OFParserGetVirtualDevice: virtual device driver node not found in proc device-tree
LoadOverridesFile: looking for override file [/Calib/camera_override.isp] 1/16LoadOverridesFile: looking for override file [/data/nvcam/settings/camera_overrides.isp] 2/16LoadOverridesFile: looking for override file [/opt/nvidia/nvcam/settings/camera_overrides.isp] 3/16LoadOverridesFile: looking for override file [/var/nvidia/nvcam/settings/camera_overrides.isp] 4/16LoadOverridesFile: looking for override file [/data/nvcam/camera_overrides.isp] 5/16LoadOverridesFile: looking for override file [/data/nvcam/settings/e3326_front_P5V27C.isp] 6/16LoadOverridesFile: looking for override file [/opt/nvidia/nvcam/settings/e3326_front_P5V27C.isp] 7/16LoadOverridesFile: looking for override file [/var/nvidia/nvcam/settings/e3326_front_P5V27C.isp] 8/16---- imager: No override file found. ----
PRODUCER: Creating output stream
PRODUCER: Launching consumer thread
CONSUMER: Waiting until producer is connected...
CONSUMER: Waiting until producer is connected...
PRODUCER: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
CONSUMER: Producer has connected; continuing.
SCF: Error InvalidState:  NonFatal ISO BW requested not set. Requested = 2147483647 Set = 4687500 (in src/services/power/PowerServiceCore.cpp, function setCameraBw(), line 653)
CONSUMER: Done.
CONSUMER: Done.
PRODUCER: Done -- exiting.
nvbuf_utils: dmabuf_fd 1828717873 mapped entry NOT found
nvbuf_utils: Can not get HW buffer from FD... Exiting...

Is there any working sample to acquire CSI camera and process data with OpenCV, CUDA (or even just display the video)?
What are the recommended API/approach? Is there a way to use ISP or CUDA or … to do debayering and depth conversion?

Thanks and best regards,
Xavier

Honey_Patouceul · February 1, 2018, 12:24pm

You may check [url]https://devtalk.nvidia.com/default/topic/1028611/jetson-tx2/is-there-a-parameter-for-enable-disable-the-tx2-on-board-camera-/post/5232247/#5232247[/url].

With gstreamer, you can use opencv as appsink if your opencv version has gstreamer support. Here is an example in C++: [url]https://devtalk.nvidia.com/default/topic/1001696/jetson-tx1/failed-to-open-tx1-on-board-camera/post/5117370/#5117370[/url].

If you want to use opencv GPU functions or CUDA, you may use gstreamer plugin nvivafilter providing your custom lib. Here is an example: [url]gstreamer NVMM <-> opencv gpuMat - Jetson TX2 - NVIDIA Developer Forums.