[Jeston TX2][JetPack-L4T-3.2.1] /dev/video0 can't grow up normally.

Hi Nvidia:
We are using Jeston TX2 to integrate AR0521 CMOS sensor.

Our /dev/video0 device can’t grow up normally.
The fail log is below.

[    5.336873] creating links for entity 150c0000.nvcsi-0
[    5.336897] creating links for entity ar0521 2-0036
[    5.336912] tegra-vi4 15700000.vi: invalid port number 0 on /i2c@3180000/ar0521_a@36
[    5.336913] tegra-vi4 15700000.vi: notifier registration failed

I think this is device tree issue.
But we can’t find any problem for our device tree setting.

Can you help to answer why tegra video driver said the port number is invalid?
Thanks

DarkHou
tx2_fail.txt (57 KB)

AR0521 device tree.rar (3.67 KB)

hello dark.hou,

there’s port binding failure in your sensor device tree,
suggest you refer to Sensor Driver Programming Guide,
and please also check the Tutorials page for the [V4L2 Sensor Driver Development Tutorial] training video.
thanks

Hi Jerry:
Thank you for your help.
We solved /dev/video0 issue by Sensor Driver porting guide.

We can get frame from CMOS sensor now.
But the frame content is wrong. (All gray color. No image)

The error log is below.
We think Jeston TX2 can’t recognize AR0521 mipi signal.
Do you have any idea for it?

yavta -n1 -c1 -fSGRBG10 -Fframecapture.raw10 -s2592x1944
Device /dev/video0 opened: vi-output, ar0521 2-0036 (platform:15700000.vi:0).
Video format set: width: 2592 height: 1944 buffer size: 10450944
Video format: BA10 (30314142) 2592x1944
3 buffers requested.
length: 10450944 offset: 0
Buffer 0 mapped at address 0x7f8cd4b000.
length: 10450944 offset: 10452992
Buffer 1 mapped at address 0x7f8c353000.
length: 10450944 offset: 20905984
Buffer 2 mapped at address 0x7f8b95b000.
[   25.045068] start streaming
[   25.160340] tegra-vi4 15700000.vi: Status:  4 channel:00 frame:00010 (0) [E] 0 10450944 bytes 0.000000 1558613235.587186
[   25.166738] tegra-vi4 15700000.vi:          timestamp sof 35882091168 eof 35882097408 data 0x00000100
[   25.179003] tegra-vi4 15700000.vi:          capture_id 1 stream  0 vchan  0
Unable to queue buffer (5).
Unable to requeue buffer (5).
Unable to release buffers: 16.
[   26.156510] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[   27.160550] tegra-vi4 15700000.vi: ATOMP_FE syncpt timeout!
[   27.166603] stop streaming

DarkHou

hello dark.hou,

according to the kernel message it shows the sensor driver is recognized.
we should debug further to understand why there are syncpt timeout failures.
could you please enable the VI tracing from debugfs to gather more details,
thanks

echo 1 > /sys/kernel/debug/tracing/tracing_on
echo 30720 > /sys/kernel/debug/tracing/buffer_size_kb
echo 1 > /sys/kernel/debug/tracing/events/tegra_rtcpu/enable
echo 1 > /sys/kernel/debug/tracing/events/freertos/enable
echo 2 > /sys/kernel/debug/camrtc/log-level
echo > /sys/kernel/debug/tracing/trace
cat /sys/kernel/debug/tracing/trace

The log is attached.

# entries-in-buffer/entries-written: 78/78 #P:4 # # _-----=> irqs-off # / _----=> need-resched # | / _---=> hardirq/softirq # || / _--=> preempt-depth # ||| / delay # TASK-PID CPU# |||| TIMESTAMP FUNCTION # | | | |||| | | kworker/0:1-51 [000] ...1 192.248440: rtos_queue_peek_from_isr_failed: tstamp:6341171318 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 192.248455: rtcpu_start: tstamp:6341172408 kworker/0:1-51 [000] ...1 192.300492: rtcpu_vinotify_handle_msg: tstamp:6343822707 tag:CHANSEL_PXL_SOF channel:0x00 frame:1 vi_tstamp:2048854636 data:0x00000001 kworker/0:1-51 [000] ...1 192.300498: rtcpu_vinotify_handle_msg: tstamp:6343822932 tag:ATOMP_FS channel:0x00 frame:1 vi_tstamp:2048854641 data:0x00000000 kworker/0:1-51 [000] ...1 192.300500: rtcpu_vinotify_handle_msg: tstamp:6343823068 tag:CHANSEL_FAULT channel:0x00 frame:1 vi_tstamp:2048854780 data:0x00000100 kworker/0:1-51 [000] ...1 192.300502: rtcpu_vinotify_handle_msg: tstamp:6343823889 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:1 vi_tstamp:2048855917 data:0x08000000 kworker/0:1-51 [000] ...1 192.300504: rtcpu_vinotify_handle_msg: tstamp:6343824022 tag:CHANSEL_FAULT_FE channel:0x01 frame:1 vi_tstamp:2048855917 data:0x00000001 kworker/0:1-51 [000] ...1 192.300507: rtcpu_vinotify_handle_msg: tstamp:6343824186 tag:ATOMP_FE channel:0x00 frame:1 vi_tstamp:2048855921 data:0x00000000 kworker/0:1-51 [000] ...1 192.404487: rtos_queue_peek_from_isr_failed: tstamp:6346172204 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 192.560491: rtos_queue_peek_from_isr_failed: tstamp:6351172711 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 192.716542: rtos_queue_peek_from_isr_failed: tstamp:6356173215 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 192.872537: rtos_queue_peek_from_isr_failed: tstamp:6361173721 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 193.028498: rtos_queue_peek_from_isr_failed: tstamp:6366174210 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 193.184568: rtos_queue_peek_from_isr_failed: tstamp:6371174737 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 193.340492: rtos_queue_peek_from_isr_failed: tstamp:6376175242 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 193.496494: rtos_queue_peek_from_isr_failed: tstamp:6381175748 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 193.704515: rtos_queue_peek_from_isr_failed: tstamp:6386176255 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 193.860484: rtos_queue_peek_from_isr_failed: tstamp:6391176763 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 194.016485: rtos_queue_peek_from_isr_failed: tstamp:6396177271 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 194.172529: rtos_queue_peek_from_isr_failed: tstamp:6401177776 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 194.328485: rtos_queue_peek_from_isr_failed: tstamp:6406178284 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 194.484485: rtos_queue_peek_from_isr_failed: tstamp:6411178790 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 194.640486: rtos_queue_peek_from_isr_failed: tstamp:6416179297 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 194.796487: rtos_queue_peek_from_isr_failed: tstamp:6421179806 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 194.952488: rtos_queue_peek_from_isr_failed: tstamp:6426180311 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 195.108484: rtos_queue_peek_from_isr_failed: tstamp:6431180819 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 195.264492: rtos_queue_peek_from_isr_failed: tstamp:6436181324 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 195.420484: rtos_queue_peek_from_isr_failed: tstamp:6441181831 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 195.576491: rtos_queue_peek_from_isr_failed: tstamp:6446182339 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 195.784497: rtos_queue_peek_from_isr_failed: tstamp:6451182843 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 195.940506: rtos_queue_peek_from_isr_failed: tstamp:6456183352 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 196.096522: rtos_queue_peek_from_isr_failed: tstamp:6461183858 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 196.252498: rtos_queue_peek_from_isr_failed: tstamp:6466184365 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 196.408492: rtos_queue_peek_from_isr_failed: tstamp:6471184872 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 196.564492: rtos_queue_peek_from_isr_failed: tstamp:6476185380 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 196.720497: rtos_queue_peek_from_isr_failed: tstamp:6481185889 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 196.876495: rtos_queue_peek_from_isr_failed: tstamp:6486186391 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 197.032493: rtos_queue_peek_from_isr_failed: tstamp:6491186897 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 197.188576: rtos_queue_peek_from_isr_failed: tstamp:6496187406 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 197.344491: rtos_queue_peek_from_isr_failed: tstamp:6501187913 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 197.500491: rtos_queue_peek_from_isr_failed: tstamp:6506188417 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 197.656553: rtos_queue_peek_from_isr_failed: tstamp:6511188927 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 197.864540: rtos_queue_peek_from_isr_failed: tstamp:6516189430 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 198.020494: rtos_queue_peek_from_isr_failed: tstamp:6521189941 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 198.176515: rtos_queue_peek_from_isr_failed: tstamp:6526190447 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 198.336471: rtos_queue_peek_from_isr_failed: tstamp:6531190951 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 198.488474: rtos_queue_peek_from_isr_failed: tstamp:6536191460 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 198.644471: rtos_queue_peek_from_isr_failed: tstamp:6541191965 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 198.800501: rtos_queue_peek_from_isr_failed: tstamp:6546192475 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 198.956497: rtos_queue_peek_from_isr_failed: tstamp:6551192980 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 199.112492: rtos_queue_peek_from_isr_failed: tstamp:6556193488 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 199.268510: rtos_queue_peek_from_isr_failed: tstamp:6561193991 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 199.424484: rtos_queue_peek_from_isr_failed: tstamp:6566194502 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 199.580484: rtos_queue_peek_from_isr_failed: tstamp:6571195008 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 199.736491: rtos_queue_peek_from_isr_failed: tstamp:6576195515 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 199.944484: rtos_queue_peek_from_isr_failed: tstamp:6581196021 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 200.100462: rtos_queue_peek_from_isr_failed: tstamp:6586196529 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 200.256495: rtos_queue_peek_from_isr_failed: tstamp:6591197034 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 200.412483: rtos_queue_peek_from_isr_failed: tstamp:6596197543 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 200.568483: rtos_queue_peek_from_isr_failed: tstamp:6601198047 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 200.724489: rtos_queue_peek_from_isr_failed: tstamp:6606198556 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 200.880486: rtos_queue_peek_from_isr_failed: tstamp:6611199060 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 201.036503: rtos_queue_peek_from_isr_failed: tstamp:6616199569 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 201.192508: rtos_queue_peek_from_isr_failed: tstamp:6621200078 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 201.348493: rtos_queue_peek_from_isr_failed: tstamp:6626200584 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 201.504491: rtos_queue_peek_from_isr_failed: tstamp:6631201091 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 201.660502: rtos_queue_peek_from_isr_failed: tstamp:6636201577 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 201.816490: rtos_queue_peek_from_isr_failed: tstamp:6641202103 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 202.024485: rtos_queue_peek_from_isr_failed: tstamp:6646202610 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 202.180515: rtos_queue_peek_from_isr_failed: tstamp:6651203117 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 202.336483: rtos_queue_peek_from_isr_failed: tstamp:6656203623 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 202.492491: rtos_queue_peek_from_isr_failed: tstamp:6661204129 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 202.648485: rtos_queue_peek_from_isr_failed: tstamp:6666204634 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 202.804493: rtos_queue_peek_from_isr_failed: tstamp:6671205145 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 202.960485: rtos_queue_peek_from_isr_failed: tstamp:6676205653 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 203.116484: rtos_queue_peek_from_isr_failed: tstamp:6681206158 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 203.272493: rtos_queue_peek_from_isr_failed: tstamp:6686206666 queue:0x0b4a3c58 kworker/0:1-51 [000] ...1 203.376582: rtos_queue_peek_from_isr_failed: tstamp:6689839819 queue:0x0b4a3c58 [act_fail2.txt|attachment](upload://r2ERmUPmQKUIA8x0Kl1bM2IdJMv.txt) (13.7 KB)

hello dark.hou,

below failures show the PIXEL_LONG_LINE error.

kworker/0:1-51    [000] ...1   192.248455: rtcpu_start: tstamp:6341172408
     kworker/0:1-51    [000] ...1   192.300492: rtcpu_vinotify_handle_msg: tstamp:6343822707 tag:CHANSEL_PXL_SOF channel:0x00 frame:1 vi_tstamp:2048854636 data:0x00000001
     kworker/0:1-51    [000] ...1   192.300498: rtcpu_vinotify_handle_msg: tstamp:6343822932 tag:ATOMP_FS channel:0x00 frame:1 vi_tstamp:2048854641 data:0x00000000
     kworker/0:1-51    [000] ...1   192.300500: rtcpu_vinotify_handle_msg: tstamp:6343823068 tag:CHANSEL_FAULT channel:0x00 frame:1 vi_tstamp:2048854780 data:0x00000100
     kworker/0:1-51    [000] ...1   192.300502: rtcpu_vinotify_handle_msg: tstamp:6343823889 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:1 vi_tstamp:2048855917 data:0x08000000
     kworker/0:1-51    [000] ...1   192.300504: rtcpu_vinotify_handle_msg: tstamp:6343824022 tag:CHANSEL_FAULT_FE channel:0x01 frame:1 vi_tstamp:2048855917 data:0x00000001
     kworker/0:1-51    [000] ...1   192.300507: rtcpu_vinotify_handle_msg: tstamp:6343824186 tag:ATOMP_FE channel:0x00 frame:1 vi_tstamp:2048855921 data:0x00000000

please check the sensor specification and check the active_w and line_length of your device tree settings.
you may also update dtb file and perform the partial update with below commands.

$ sudo ./flash.sh -r -k kernel-dtb jetson-tx2 mmcblk0p1

Hi Jerry:
We reviewed our device tree setting for AR0521.
The active_w is 1920.
The ling length is 1928.

But how should we set ling length for AR0521?
The TX2 camera porting guide just said the line length should be greater than active_w.

mode0 { // ar0521_MODE_1920X1080
				mclk_khz = "27000";
				num_lanes = "4";
				tegra_sinterface = "serial_a";
				discontinuous_clk = "yes";
				dpcm_enable = "false";
				cil_settletime = "0";

				active_w = "1920";
				active_h = "1080";
				pixel_t = "bayer_bggr";
				readout_orientation = "90";
				line_length = "1928";
				inherent_gain = "1";
				mclk_multiplier = "16";
				pix_clk_hz = "207000000";

				min_gain_val = "1.0";
				max_gain_val = "16.0";
				min_hdr_ratio = "1";
				max_hdr_ratio = "1";
				min_framerate = "1.5";
				max_framerate = "60";
				min_exp_time = "16.165";
				max_exp_time = "666637";
				embedded_metadata_height = "1";
			};

DarkHou
tegra186-camera-ar0521-a00.rar (2.71 KB)

hello dark.hou,

line length and frame length were related to sensor timing controls. we also take these settings to calculate the exposure time, frame rate, …etc.
you may also check the total horizontal timing size for the line length configure; to check total vertical timing size the for frame length configure.
these timing registers usually represent with two bytes, please also note the endianness (big-endian or little-endian) for the values.

BTW, here’s another formula for you to check the settings,
thanks

Sensor frame rate = Total sensor output pixel clock / (Line length * Frame length)

Hi Jerry:
We still can’t solve PXL_SOF syncpt timeout issue.

[ 26.156510] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[ 27.160550] tegra-vi4 15700000.vi: ATOMP_FE syncpt timeout!

Our sensor active_w and line_length setting is right already.

active_w = “1920”;
active_h = “1080”;
dynamic_pixel_bit_depth = “10”;
csi_pixel_bit_depth = “10”;
mode_type = “bayer”;
pixel_phase = “rggb”;
pixel_t = “bayer_rggb”;
readout_orientation = “0”;
line_length = “1984”;

Do you have any suggestion?
DarkHou

hello dark.hou,

please arrange oscilloscope to probe the MIPI signaling, also please check comment #4 to enable VI tracing logs for checking.
BTW, had you ran v4l2-compliance test to verify sensor driver yet.
thanks

Hi Jerry:
We can get total frame from AR0521 now.
The root cause is we set wrong active_w for AR0521 5M resolution.
The right active_w is 2600 not 2592.

After we solved PIXEL_LONG_LINE issue, we can get frame normally now.
But the color is wrong.

We use color bar mode to test it first.
AR0521 mode is set as color bar mode.

The black color should be (R:0,G:0,B:0)
But the black color become (R:168,G:168,B:168)

DarkHou
!(upload://rAFl1RZvqOkMRjKcgC9ttvgLdyg.png)

Hi Jerry:
Our AR0521 sensor is MIPI 10bit 4-lane.
Resolution is 2600 x 1952.

Pixel format is bayer RGB.
The pixel order is GRBG.

Our device tree setting is below.

dynamic_pixel_bit_depth = "10";
csi_pixel_bit_depth = "10";
mode_type = "bayer";
pixel_phase = "bggr";
pixel_t = "bayer_bggr";

But the color is wrong still.
DarkHou

hello dark.hou,

please update the device tree settings to report the correct pixel format.
GRBG pixel format should be supported according to the kernel sources as below.
thanks

<i>$l4t-r32.1/kernel_src/kernel/nvidia/drivers/media/platform/tegra/camera/sensor_common.c</i>

static int extract_pixel_format(
        const char *pixel_t, u32 *format)
{
...
        else if (strncmp(pixel_t, "bayer_grbg10", size) == 0)
                *format = V4L2_PIX_FMT_SGRBG10;

Hi Jerry:

dynamic_pixel_bit_depth = "10";
csi_pixel_bit_depth = "10";
mode_type = "bayer";
pixel_phase = "grbg";
pixel_t = "bayer_grbg";

We changed pixel format to grbg.
The result is still the same.

DarkHou
!(upload://rAFl1RZvqOkMRjKcgC9ttvgLdyg.png)

hello dark.hou,

please share the sensor formats capability,
for example,

$ v4l2-ctl -d /dev/video0 --list-formats-ext

Hi Jerry:
The result is below.

root@tegra-ubuntu:/mnt# v4l2-ctl -d /dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
        Type: Video Capture

        [0]: 'BA10' (10-bit Bayer GRGR/BGBG)
                Size: Discrete 2592x1944
                        Interval: Discrete 0.017s (60.000 fps)

DarkHou

Hi Jerry:
I don’t think the root cause is pixel format.
Because the black color is wrong too.

If bayer RGB order is wrong, the black color should be right still.

DarkHou

hello dark.hou,

had you also update the badge property in your sensor device tree?
for example,

modules {
                        module0 {
                                badge = "e3326_front_P5V27C";

please check you had define an unique name for the badge_info property,
you may check Sensor Driver Programming Guide and check the property description.
thanks

Hi Jerry:
We set unique id for badge filed already.
The color is wrong still.

modules {
			module0 {
				badge = "ar0521_rear_2C09SU";
				position = "rear";
				status = "okay";
				orientation = "1";
				drivernode0 {
					/* Declare PCL support driver (classically known as guid)  */
					pcl_id = "v4l2_sensor";
					devname = "ar0521 2-0036";
					status = "okay";
					/* Declare the device-tree hierarchy to driver instance */
					proc-device-tree = "/proc/device-tree/i2c@3180000/ar0521_a@36";
				};
			};
		};

DarkHou

hello dark.hou,

could you please share the raw files for us checking,
for example,

$ v4l2-ctl -d /dev/video0 --set-fmt-video=width=2592,height=1944,pixelformat=BA10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=5 --stream-to=test.raw