Camera outputs to 1 buffer only

Hi,

We are trying to get raw frames from a camera using v4l2 but it looks like it is only writing into 1 v4l2 buffer (buffer.index = 0). Other buffers have null/black images. There is also an image tear in the captured frames. Only the first frame is consistently complete. The others that are not null usually has the bottom part of the frame coming from a previous time stamp (buffer[0].frame[1] has lines that exactly the same as buffer[0].frame[0]).

argus_camera and nvgstcapture are displaying good video streams. However, we cannot use these because we need to process raw frames from the camera.

qv4l2 (v4l2 test bench GUI) seems to be showing the same problem because the output video stream has alternating black and good frames.

Output from image sensor is parallel then goes through serializer to CSI input of TX1. We are not quite sure whether it is an issue with serializer HW or the driver.

Can anyone suggest any other things to check or possible solutions to this issue?

Please try the attached patch.

0001-drivers-camera-fix-FE-syncpt-wait.patch.txt (3.02 KB)
0002-media-tegra-fix-missing-timestamp-for-vi4.patch.txt (3.52 KB)

Hi Shane,

Thanks for your prompt reply.

Are these patches compatible with R28.1? We haven’t updated to the latest JetPack yet.

Hi Shane,

I’ve already update to L4T R28.2 and applied the patch you provided. I also translated the driver for the camera that we are using (sensor not yet included in the drivers available in the release). I was able to compile the kernel and DTB successfully.

However, video0 is no longer detected. It was working in the R28.1 version.

  1. Can you help check what is the problem with the R28.2 version of our camera driver? I just translated the driver based on the Camera Sensor Drivers Porting Guide (28.1 to 28.2). However, I did not use the sensor_mode_properties, and just used the frame length registry address when setting the frame length, unlike the other camera drivers in the R28.2 release.

  2. Can you also provide the R28.1 version of the patch files that you uploaded here? I just want to make sure that this will indeed fix the 1 buffer problem that we are having.

Thanks.

28.1 should have the problem need those patch to fixed. Your problem may not the same.
Your problem may be the alignment problem. Does your resolution 256 alignment? If not try to modify the below to 1 to try.

./camera/vi/core.h:#define TEGRA_STRIDE_ALIGNMENT       256

Hi Shane,

It is currently set at 256 stride alignment. I’m compiling now with your suggested setting to TEGRA_STRIDE_ALIGNMENT 1 for R28.1.

By the way, I was able to fix the R28.2 and detect video0. However, the output frames are now all black.

How many frames did you dump all of them are black? Does the hexdump all the same value?

~50 frames but not all of them are black.

I might also have a problem with the power rail in the driver. The camera only successfully turns on if the deserializer board that we are using has a floating ground. But it looks like the buffers are being used now (not just the first one). I just need to figure out this new issue.

Hi Shane,

I reflashed the TX1 board and was now able to power on the camera with no errors.

It is also writing to more buffers now, instead of only writing to buffer.index = 0. However, it is still intermittent. Some buffers are still empty/black and almost always the first 4 frames that I dequeue is completely black.

The time between each dequeue also doesn’t make sense. It is faster than the expected frame rate.

I also still get the image tear where the bottom lines of the frame are from a previous timestamp or previous frame.

Is it because the buffers are dequeued even if they are not yet done (on the outgoing queue)?

Is it YUV sensor?

No, it is an RGB-IR sensor that is why we need to get the raw image from the sensor.

Hi Shane,

The driver might not be successfully writing to the memory buffer.
The saved frames are sometimes identical if they came from the same buffer index.
The buffer flags are also mostly errors (0x2045).
The kernel log also indicates errors in the vi and syncpoint time out:

May  4 06:54:35 tegra-ubuntu kernel: [  797.510958] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:35 tegra-ubuntu kernel: [  797.533660] vi 54080000.vi: tegra_channel_error_status:error 20022 frame 5
May  4 06:54:35 tegra-ubuntu kernel: [  797.540912] vi 54080000.vi: tegra_channel_error_status:error 20022 frame 6
May  4 06:54:35 tegra-ubuntu kernel: [  797.647168] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:35 tegra-ubuntu kernel: [  797.779628] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:35 tegra-ubuntu kernel: [  797.911282] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:35 tegra-ubuntu kernel: [  798.043267] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:35 tegra-ubuntu kernel: [  798.179801] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:35 tegra-ubuntu kernel: [  798.311191] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:36 tegra-ubuntu kernel: [  798.443383] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:36 tegra-ubuntu kernel: [  798.579441] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:36 tegra-ubuntu kernel: [  798.711398] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:36 tegra-ubuntu kernel: [  798.843513] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:36 tegra-ubuntu kernel: [  798.979597] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:36 tegra-ubuntu kernel: [  799.111556] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:36 tegra-ubuntu kernel: [  799.243996] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:36 tegra-ubuntu kernel: [  799.375920] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:37 tegra-ubuntu kernel: [  799.511650] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:37 tegra-ubuntu kernel: [  799.532287] vi 54080000.vi: tegra_channel_error_status:error 4000 frame 82
May  4 06:54:37 tegra-ubuntu kernel: [  799.539420] vi 54080000.vi: tegra_channel_error_status:error 24022 frame 83
May  4 06:54:37 tegra-ubuntu kernel: [  799.548188] vi 54080000.vi: tegra_channel_error_status:error 20022 frame 84
May  4 06:54:37 tegra-ubuntu kernel: [  799.643864] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:37 tegra-ubuntu kernel: [  799.643874] video4linux video0: frame start syncpt timeout!0
May  4 06:54:37 tegra-ubuntu kernel: [  799.665670] vi 54080000.vi: tegra_channel_error_status:error 20022 frame 90
May  4 06:54:37 tegra-ubuntu kernel: [  799.775730] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:37 tegra-ubuntu kernel: [  799.911816] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:37 tegra-ubuntu kernel: [  800.043797] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:37 tegra-ubuntu kernel: [  800.176227] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:37 tegra-ubuntu kernel: [  800.307984] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:38 tegra-ubuntu kernel: [  800.443861] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:38 tegra-ubuntu kernel: [  800.575928] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:38 tegra-ubuntu kernel: [  800.708438] video4linux video0: MW_ACK_DONE syncpoint time out!0
May  4 06:54:38 tegra-ubuntu kernel: [  800.844025] video4linux video0: MW_ACK_DONE syncpoint time out!0

Looks like Tegra doesn’t support RGB-IR sensors.

Hi Shane,

The application is only streaming the raw data without debayering. Shouldn’t Tegra be able to support this?

Does the data format and data size the same with any of the bayer format in the TX1 support list in the TRM?

Hi Shane,

Yes, 12-bit raw is supported in TX1.

OK, then you may need to probe the MIPI timing to make sure it match the MIPI spec.

Hi Shane,

We will also check in the deserializer spec if there are specific settings that we missed.

Just to clarify, if it is a MIPI timing or any hardware timing issue, wouldn’t it also show up as a problem in the argus_camera or nvgstcapture stream? When we use those two, we can see continuous streaming with no blank frames. We only see an issue in V4L2. Also, in V4L2, we do get a few good frames which means the data has been parsed correctly.

Does the 28.2 have the same symptom? argsu/gstreamer have no problem but v4l2 utility.

Hi Shane,

We only observed that in 28.1 where gstreamer and argus are both working properly and v4l2 has issues (only buffer.index = 0 is used).

In 28.2, gstreamer and argus are no longer working (no video stream) while v4l2 still has issues.

What do these observations mean?