MIPI CSI2 camera: Bayer format support?

The imx135 camera module can work on R21.3 for me, but the blue element of output more than R/G, it looks like color pattern problem. But I set Bayer RGGB10 format on TK1, the log tell me this format not found. Dose TK1 support RGGB10 format?

Hi yumeng,

You can refer to Tegra K1 Datasheet in this link:
https://developer.nvidia.com/hardware-design-and-development

Description of Video Input Interfaces section indicates that RGGB10 is not supported by MIPI CSI.

Thanks.

Hi Mouda

Thanks for your reply. The RGGB10 is kind of RAW10, it should be supported by MIPI CSI.
There is video output when BGGR8 and GBRG8, but BGGR10 and RGGB10 not. I’m not sure why the TK1 can’t find 10bits format.

Thanks.

Is the CSI interface working now? Drivers are available? Any links/references on this?

Thanks guys!

If the camera can work now, I guess the CSI interface is available.

This thread has more detail.
https://devtalk.nvidia.com/default/topic/801916/embedded-systems/tested-cameras-on-the-jetson/

Thanks.

Hi yumeng,

Sorry I have a mistake, RGGB10 should be supported on CSI MIPI.
Is your use case as the flow:
IMX135 camera → RGGB10 → CSI → YUV420 → Display.
If so, which part of this flow broken when you setup the color format?

By the way, did you mean your IMX135 camera is workable but incorrect color display at RGGB8 configuration
Could you post a snapshot of the display under this condition?

Thanks.

Hi Mouda

There are different color pattern output snapshot as flow:

  1. IMX135 camera → RGGB10 → CSI → YUV420 → Display

  2. IMX135 camera → RGGB10 → CSI → BGGR8 → Display

  3. IMX135 camera → RGGB10 → CSI → GBRG8 → Display

  4. IMX135 camera → RGGB10 → CSI → RGGB10 → Display

    dmesg log:
    [ 23.969830] vi vi.0: Format 32315559 not found
    [ 23.969964] vi vi.0: Format 32315659 not found
    [ 23.970053] vi vi.0: Format 33524742 not found
    [ 23.970188] vi vi.0: Format 33424752 not found

  5. IMX135 camera → RGGB10 → CSI → RGB24 → Display

  6. IMX135 camera → RGGB10 → CSI → RGB32 → Display

Before enable camera, first I check vi format which is use by yavta tool.
http://git.ideasonboard.org/yavta.git
example:

ubuntu@tegra-ubuntu:~/yavta$ ./yavta /dev/video0 -l
Device /dev/video0 opened.
Device `vi' on `' is a video capture (without mplanes) device.
No control found.
Video format: SRGGB10 (30314752) 1280x720 (stride 2560) field none buffer size 1843200

I think that driver color format need to match vi color format.
Thanks.

RGGB10bit is supported on JTK1 with IMX135, IMX135 should be able to work without any change on R21.3
Resolution: 4208 x 3120
NVRAW format: 10 Bit Linear
Bayer order: RGGB
Samples/Pixel: 1
Bits/Sample: 10

@edli1983, I re-download and re-flash R21.3 again, R21.3 default config without soc_camera_imx135, how did you insmod IMX135 camera?

I tested IMX135 camera by two ways.

  1. without any change, command as following
sudo modprobe -r nvhost-vi
sudo modprobe tegra_camera

Only have soc-camera-pdrv.1 with RGBA 8-8-8-8

  1. config soc_camera_imx135 as module
sudo modprobe -r nvhost-vi
sudo insmod imx135_v4l2.ko
sudo modprobe tegra_camera

enable stream with 4208 x 3120 and get log
vi vi.0: dma_alloc_coherent of size 13131776 failed

Did I lost something?
Thanks.

@Yumeng
After you flash the R21.3 release, Plug the HDMI display, run gst and you will see the camera preview.
nvgstcapture-1.0 --svs=“nvhdmioverlaysink” --camsrc=1

@edli1983
I got log as follow after I run gst.
nvgstcapture-1.0 --svs=“nvhdmioverlaysink” --camsrc=1

Runtime CSI Camera Commands:

  Help : 'h'
  Quit : 'q'
  Set Capture Mode:
      mo:<val>
          (1): image
          (2): video

...

Inside NvxLiteH264DecoderLowLatencyInitNvxLiteH264DecoderLowLatencyInit set DPB and MjstreamingNvCamHwHalDeviceRead: 18 - -1
NvCamHwHalDeviceRead: 19 - -1
NvCamHwHalDeviceRead: 20 - -1
NvCamHwHalDeviceRead: 21 - -1
NvCamHwHalDeviceRead: 22 - -1
NvCamHwHalDeviceRead: 23 - -1
IMX135 ****  Can not open camera device: No such file or directory
NvOdmImagerOpenExpanded 458: Sensor ERR
NvOdmImagerOpenExpanded FAILED!
camera_open failed

** (nvgstcapture-1.0:2540): ERROR **: <create_capture_pipeline:2752> can't set camera to playing

Trace/breakpoint trap
pcl-generic pcl-generic: layout empty.
pcl-generic pcl-generic: virtual_device_add
pcl-generic pcl-generic: camera_chip_add chip pcl_IMX135 already added.
tegra-i2c tegra12-i2c.2: no acknowledge from address 0x10
pcl-generic pcl-generic: virtual_device_add
pcl-generic pcl-generic: camera_chip_add chip pcl_DW9718 already added.
tegra-i2c tegra12-i2c.2: no acknowledge from address 0xc
pcl-generic pcl-generic: virtual_device_add
pcl-generic pcl-generic: camera_chip_add chip pcl_AS3648 already added.
tegra-i2c tegra12-i2c.2: no acknowledge from address 0x30

And here is my monitor resolution

1280x1024      60.0*+   75.0
1280x960       60.0
1280x720       60.0     50.0
1152x864       75.0
1024x768       75.1     70.1     60.0
832x624        75.0
800x600        75.0     72.2     60.3     56.3
720x480        59.9
720x400        70.0
640x480        75.0     72.8     66.8     60.0

I guess that is i2c problem, and following is measurable wave between

  1. LDO1(+1.8_RUN_CAM) and 2) RESET

Is it need to set extra config?

I can’t see your last comment. How did you make the IMX135 connect to JTK1 ?

I refer from http://elinux.org/File:J3A2_I2C-highlighted.png, and double check hardware connection,
IMX135 connect to JTK1 by a adapter board, the pin mapping and schematic diagram following

<b>JTK1                      IMX135 board</b>
LDO1(+1.8V_RUN_CAM)       -> Vif
LDO4(+2.8V_RUN_CAM)       -> Vana
LDO7(+1.05V_RUN_CAM_REAR) -> Vdig 
CAM_I2C_SCL/SDA           -> SCL/SDA
CAM_RST_L                 -> RESET
CAM1_MCLK                 -> MCLK
CSI_A_CLK_P/N             -> CSI_CLK_P/N
CSI_A_D0_P/N              -> DATA_LANE_0_P/N
CSI_A_D1_P/N              -> DATA_LANE_1_P/N
CSI_B_D0_P/N              -> DATA_LANE_2_P/N
CSI_B_D1_P/N              -> DATA_LANE_3_P/N

The output from V4L2 is raw format with CSI camera, you need use Gstreamer to process the CSI camera data. Could you help attach the log with “nvgstcapture-1.0 --svs=“nvhdmioverlaysink” --camsrc=1” ?

@edli1983
Thanks for your help. IMX135 can works on my JTK1. R21.3 default control PWDN pin, but I connect IMX135 RESET to JTK1 CAM_RST_L, so RESET always down.

On the other hand, my IMX135 focuser is pt5110, not dw9718, after I command nvgstcapture, it still occur

IMX135 ****  Can not open camera device: No such file or directory

At present I havn’t pt5110 driver, so I disable focuser and flash, then IMX135 can work, just haven’t foucuser and flash function.

But I have a little doubt, why can’t I use soc_camera to implement csi camera?
And if I want to do digital image processing from video capture source by opencv, opencv default capture video(0) from /dev/video0, but nvgstcapture command I can’t find the /dev/video0, how can I do digital image processing by nvgstcapture?

Thanks.

The soc_camera and /dev/video0 support YUV camera data, it doesn’t support raw camera data.
If you use IMX135 raw camera, Both won’t work.
The nvgstcapture is based on Gstreamer, you can use Gstreamer appsync to do digital image process.

The Gstreamer command including uri parameter, it can play media address and file, like

gst-launch-0.10 v4l2src device="/dev/video0" ! "video/x-raw-yuv, width=640, height=480, format=(fourcc)I420" ! xvimagesink

It will create a pipeline for /dev/video0, this command was tested by usb webcam.

I’m not sure how to create a pipeline for IMX135 by Gstreamer, …Could you give me some hint? Thanks…!

Two ways to get the preview on the screen

  1. nvgstcapture-1.0 --svs=“nvhdmioverlaysink” --camsrc=1
  2. gst-launch-1.0 nvcamerasrc ! nvtee ! nvhdmioverlaysink -v (For preview only).

For detail about nvcamerasrc plugin

gst-inspect-1.0 nvcamerasrc

I use gst-inspect-1.0 to check the nvcamerasrc supported features, there is I420 support, so I write a sample code for I420 output format, to capture IMX135 buffer base on gstreamer-1.0 and opencv, then display picture, here is the part of sample code,

//gstreamer command
gchar *descr = g_strdup(
    "nvcamerasrc ! video/x-raw\(memory:NVMM\), width=\(int\)640, height=\(int\)480, format=\(string\)I420, framerate=\(fraction\)0/1 ! "
    "nvtee ! video/x-raw\(memory:NVMM\), width=\(int\)640, height=\(int\)480, format=\(string\)I420, framerate=\(fraction\)0/1 ! "
    "appsink name=sink");
//Appsink callback function
GstMapInfo map;
gst_buffer_map (buffer, &map, GST_MAP_READ);
Mat frame(Size(width, height + height/2), CV_8UC1, (char*)map.data, Mat::AUTO_STEP);
frameQueue.push_back(frame);
//Show image
while(1) {
    if (frameQueue.size() > 0) {
        Mat frame = frameQueue.front();
        Mat bgrImg;
        cvtColor(frame, bgrImg, CV_YUV2BGR_I420);
        imshow("BGR Image", bgrImg);
        frameQueue.clear();
    }
}

But I got the result as follow

on other hand, I have tested usb webcam for v4l2src and videotestsrc, the source code like above,
gstreamer set I420 output and capture I420 buffer then convert to BGR, the result is fine. Did I lose something?

Hi, yumeng.

I asked for your help among several days. I’m really grateful for your help.

In fact, I can’t see the preview yet. So I have one more question about this posting.

We connected the board and sensor with your advice. (RESET pin of IMX135 need connect to POWERDOWN pin of TK1.)

And I think that “disable focuser and flash” means disable in menuconfig.
But still, we get the error “IMX135 **** Can not open camera device: No such file or directory.”

How can I disable the focuser and flash ?

Sorry for asking continuously about the detail.

I look forward to hearing from you.
Thank you.

Best regards.