Porting AR0135 + DS90UB964 driver to Jetson TX2 (Chansel Fault?)

Hello there,

I’ve just noticed that the forum software destroyed my post. I’ll edit this post and write another one as reply to myself…

Hello there,

for about a week I’m working on porting an Aptina AR0135 driver to the Jetson TX2.
We use a DS90UB964 deserializer as CSI source on 2 lanes on a custom base board.

As the OV5693 was already working on the dev kit I decided to alter only as much as needed to archive my goal.
I’ve used the original ov5693 driver as base for my own.
I’ve removed all references inside the device tree to the ov5693 and replaced them by my ar0135 ones.

Here is my hardware/nvidia/platform/t18x/common/kernel-dts/t18x-common-modules/tegra186-camera-e3326-a00.dtsi:

/*
 * Copyright (c) 2015-2016, NVIDIA CORPORATION.  All rights reserved.
 *
 * This program is free software; you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation; either version 2 of the License, or
 * (at your option) any later version.
 *
 * This program is distributed in the hope that it will be useful, but WITHOUT
 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
 * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
 * more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this program.  If not, see <http://www.gnu.org/licenses/>.
 */

/ {
        host1x {
                vi@15700000 {
                        num-channels = <1>;
                        ports {
                                #address-cells = <1>;
                                #size-cells = <0>;
                                port@0 {
                                        reg = <0>;
                                        e3326_vi_in0: endpoint {
                                                csi-port = <2>;
                                                bus-width = <2>;
                                                remote-endpoint = <&e3326_csi_out0>;
                                        };
                                };
                        };
                };

                nvcsi@150c0000 {
                        num-channels = <1>;
                        #address-cells = <1>;
                        #size-cells = <0>;
                        channel@0 {
                                reg = <0>;
                                ports {
                                        #address-cells = <1>;
                                        #size-cells = <0>;
                                        port@0 {
                                                reg = <0>;
                                                e3326_csi_in0: endpoint@0 {
                                                        csi-port = <2>;
                                                        bus-width = <2>;
                                                        remote-endpoint = <&e3326_ar0135_out0>;
                                                };
                                        };
                                        port@1 {
                                                reg = <1>;
                                                e3326_csi_out0: endpoint@1 {
                                                        remote-endpoint = <&e3326_vi_in0>;
                                                };
                                        };
                                };
                        };
                };
        };

        i2c@3180000 {
                ar0135_c@18 {
                        compatible = "nvidia,ar0135";
                        /* I2C device address */
                        reg = <0x18>;

                        /* V4L2 device node location */
                        devnode = "video0";

                        /* Physical dimensions of sensor */
                        physical_w = "3.674";
                        physical_h = "2.738";

                        /* Define any required hw resources needed by driver */
                        /* ie. clocks, io pins, power sources */
                        avdd-reg = "vana";
                        iovdd-reg = "vif";

                        /**
                        * A modeX node is required to support v4l2 driver
                        * implementation with NVIDIA camera software stack
                        *
                        * mclk_khz = "";
                        * Standard MIPI driving clock, typically 24MHz
                        *
                        * num_lanes = "";
                        * Number of lane channels sensor is programmed to output
                        *
                        * tegra_sinterface = "";
                        * The base tegra serial interface lanes are connected to
                        *
                        * discontinuous_clk = "";
                        * The sensor is programmed to use a discontinuous clock on MIPI lanes
                        *
                        * dpcm_enable = "true";
                        * The sensor is programmed to use a DPCM modes
                        *
                        * cil_settletime = "";
                        * MIPI lane settle time value.
                        * A "0" value attempts to autocalibrate based on mclk_multiplier
                        *
                        *
                        *
                        *
                        * active_w = "";
                        * Pixel active region width
                        *
                        * active_h = "";
                        * Pixel active region height
                        *
                        * pixel_t = "";
                        * The sensor readout pixel pattern
                        *
                        * readout_orientation = "0";
                        * Based on camera module orientation.
                        * Only change readout_orientation if you specifically
                        * Program a different readout order for this mode
                        *
                        * line_length = "";
                        * Pixel line length (width) for sensor mode.
                        * This is used to calibrate features in our camera stack.
                        *
                        * mclk_multiplier = "";
                        * Multiplier to MCLK to help time hardware capture sequence
                        * TODO: Assign to PLL_Multiplier as well until fixed in core
                        *
                        * pix_clk_hz = "";
                        * Sensor pixel clock used for calculations like exposure and framerate
                        *
                        *
                        *
                        *
                        * inherent_gain = "";
                        * Gain obtained inherently from mode (ie. pixel binning)
                        *
                        * min_gain_val = ""; (floor to 6 decimal places)
                        * max_gain_val = ""; (floor to 6 decimal places)
                        * Gain limits for mode
                        *
                        * min_exp_time = ""; (ceil to integer)
                        * max_exp_time = ""; (ceil to integer)
                        * Exposure Time limits for mode (us)
                        *
                        *
                        * min_hdr_ratio = "";
                        * max_hdr_ratio = "";
                        * HDR Ratio limits for mode
                        *
                        * min_framerate = "";
                        * max_framerate = "";
                        * Framerate limits for mode (fps)
                        */
                        mode0 { // AR0135_MODE_1280X960
                                mclk_khz = "24000";
                                num_lanes = "2";
                                tegra_sinterface = "serial_c";
                                discontinuous_clk = "no";
                                dpcm_enable = "false";
                                cil_settletime = "0";

                                active_w = "1280";
                                active_h = "960";
                                pixel_t = "bayer_bggr";
                                readout_orientation = "90";
                                line_length = "1388";
                                inherent_gain = "1";
                                mclk_multiplier = "6.67";
                                //pix_clk_hz = "74250000";
                                pix_clk_hz = "160000000";

                                min_gain_val = "1.0";
                                max_gain_val = "16";
                                min_hdr_ratio = "1";
                                max_hdr_ratio = "64";
                                min_framerate = "1.816577";
                                max_framerate = "54";
                                min_exp_time = "34";
                                max_exp_time = "550385";
                        };

                        ports {
                                #address-cells = <1>;
                                #size-cells = <0>;

                                port@0 {
                                        reg = <0>;
                                        e3326_ar0135_out0: endpoint {
                                                csi-port = <2>;
                                                bus-width = <2>;
                                                remote-endpoint = <&e3326_csi_in0>;
                                        };
                                };
                        };
                };
        };

        e3326_lens_ar0135@P5V27C {
                min_focus_distance = "0.0";
                hyper_focal = "0.0";
                focal_length = "2.67";
                f_number = "2.0";
                aperture = "2.0";
        };

        tegra-camera-platform {
                compatible = "nvidia, tegra-camera-platform";
                /**
                * Physical settings to calculate max ISO BW
                *
                * num_csi_lanes = <>;
                * Total number of CSI lanes when all cameras are active
                *
                * max_lane_speed = <>;
                * Max lane speed in Kbit/s
                *
                * min_bits_per_pixel = <>;
                * Min bits per pixel
                *
                * vi_peak_byte_per_pixel = <>;
                * Max byte per pixel for the VI ISO case
                *
                * vi_bw_margin_pct = <>;
                * Vi bandwidth margin in percentage
                *
                * max_pixel_rate = <>;
                * Max pixel rate in Kpixel/s for the ISP ISO case
                *
                * isp_peak_byte_per_pixel = <>;
                * Max byte per pixel for the ISP ISO case
                *
                * isp_bw_margin_pct = <>;
                * Isp bandwidth margin in percentage
                */
                num_csi_lanes = <4>;
                max_lane_speed = <1500000>;
                min_bits_per_pixel = <10>;
                vi_peak_byte_per_pixel = <2>;
                vi_bw_margin_pct = <25>;
                isp_peak_byte_per_pixel = <5>;
                isp_bw_margin_pct = <25>;

                /**
                * The general guideline for naming badge_info contains 3 parts, and is as follows,
                * The first part is the camera_board_id for the module; if the module is in a FFD
                * platform, then use the platform name for this part.
                * The second part contains the position of the module, ex. “rear” or “front”.
                * The third part contains the last 6 characters of a part number which is found
                * in the module's specsheet from the vender.
                */
                modules {
                        module0 {
                                badge = "e3326_front_P5V27C";
                                position = "rear";
                                orientation = "1";
                                drivernode0 {
                                        /* Declare PCL support driver (classically known as guid)  */
                                        pcl_id = "v4l2_sensor";
                                        /* Driver v4l2 device name */
                                        devname = "ar0135 2-0018";
                                        /* Declare the device-tree hierarchy to driver instance */
                                        proc-device-tree = "/proc/device-tree/i2c@3180000/ar0135_c@18";
                                };
                                drivernode1 {
                                        /* Declare PCL support driver (classically known as guid)  */
                                        pcl_id = "v4l2_lens";
                                        proc-device-tree = "/proc/device-tree/e3326_lens_ar0135@P5V27C/";
                                };
                        };
                };
        };
};

I assume that this configuration is used as the ov5693 was detected as ov5693_c when active.
The lane wasn’t changed. I’ve configured the DS90UB964 to output 800 MHz on 2 lanes.
According my measure using an oscilloscope actual CSI signals are present on both lanes and a continous clock is activated.

I’ve used two programs to actually test. First I’ve used samples using the argus library. The sample 10_camera_recording was working using the dev kit so I decided to use it as well:
This program is very explicit about the source files that might be interesting to read to get more informations about this problem. But libargus doesn’t seems to be open source.

sudo /home/nvidia/tegra_multimedia_api/samples/10_camera_recording/camera_recording

Set governor to performance before enabling profiler
LSC: LSC surface is not based on full res!
LSC: LSC surface is not based on full res!
PRODUCER: Creating output stream
PRODUCER: Launching consumer thread
Failed to query video capabilities: Inappropriate ioctl for device
NvMMLiteOpen : Block : BlockType = 4
===== MSENC =====
NvMMLiteBlockCreate : Block : BlockType = 4
875967048
842091865
create vidoe encoder return true
CONSUMER: Waiting until producer is connected...
PRODUCER: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
SCF: Error Timeout:  (propagating from src/services/autocontrol/NvCameraIspDriver.cpp, function programIsp(), line 1199)
SCF: Error Timeout:  (propagating from src/services/autocontrol/NvCameraIspDriver.cpp, function programIsp(), line 1241)
SCF: Error Timeout:  (propagating from src/services/capture/CaptureServiceCore.cpp, function programIsp(), line 321)
SCF: Error Timeout:  (propagating from src/services/capture/CaptureServiceCore.cpp, function doCSItoISPCapture(), line 446)
SCF: Error Timeout:  (propagating from src/services/capture/CaptureServiceCore.cpp, function issueCapture(), line 344)
SCF: Error Timeout:  (propagating from src/services/capture/CaptureServiceDevice.cpp, function issueCaptures(), line 1190)
SCF: Error Timeout:  (propagating from src/services/capture/CaptureServiceDevice.cpp, function issueCaptures(), line 1042)
SCF: Error Timeout:  (propagating from src/services/capture/CaptureServiceEvent.cpp, function wait(), line 59)
Error: Camera HwEvents wait, this may indicate a hardware timeout occured,abort current/incoming cc
launchCC abort cc 104
SCF: Error Timeout:  (propagating from src/api/Session.cpp, function capture(), line 750)
(Argus) Error Timeout: Failed to submit first capture request (propagating from src/api/CaptureSessionImpl.cpp, function submitCaptureRequests(), line 301)
(Argus) Error Timeout:  (propagating from src/api/CaptureSessionImpl.cpp, function threadFunction(), line 741)
SCF: Error Timeout: ISP port 0 timed out! (in src/services/capture/CaptureServiceDeviceIsp.cpp, function waitIspFrameEnd(), line 663)
SCF: Error Timeout:  (propagating from src/services/capture/CaptureServiceDeviceIsp.cpp, function waitIspFrameEnd(), line 704)
SCF: Error Timeout:  (propagating from src/common/Utils.cpp, function workerThread(), line 113)
SCF: Error Timeout: Worker thread IspHw frameComplete failed (in src/common/Utils.cpp, function workerThread(), line 130)
Error: waitCsiFrameStart timeout guid 0
VI Stream Id = 2 Virtual Channel = 0
************VI Debug Registers**********
VI_CSIMUX_STAT_FRAME_8   = 0x00010002
VI_CSIMUX_FRAME_STATUS_0         = 0x00000100
VI_CFG_INTERRUPT_STATUS_0        = 0x00000000
VI_ISPBUFA_ERROR_0       = 0x00000000
VI_FMLITE_ERROR_0        = 0x00000000
VI_NOTIFY_ERROR_0        = 0x00000000
*****************************************
CSI Stream Id = 2 Brick Id = 1
************CSI Debug Registers**********
NVCSI_PHY_0_CILA_INTR_STATUS_CILA_0      = 0x00000000
NVCSI_PHY_0_CILB_INTR_STATUS_CILB_0      = 0x00000000
NVCSI_STREAM_0_INTR_STATUS_0     = 0x00000000
NVCSI_STREAM_0_INTR_STATUS_0     = 0x00000000
NVCSI_STREAM_0_ERR_INTR_STATUS_0         = 0x00000000
NVCSI_STREAM_0_ERROR_STATUS2VI_VC0_0     = 0x9f80bf00
NVCSI_STREAM_0_ERROR_STATUS2VI_VC1_0     = 0x00000000
NVCSI_STREAM_0_ERROR_STATUS2VI_VC2_0     = 0x00000000
NVCSI_STREAM_0_ERROR_STATUS2VI_VC3_0     = 0x00000000
*****************************************
Error: waitCsiFrameStart Something went wrong with waiting on frame start
SCF: Error Timeout: ISP Stats timed out! (in src/services/capture/CaptureServiceDeviceIsp.cpp, function waitIspStatsFinished(), line 746)

After execution the kernel log gives me some output.
I apologize for the verbose output. I’ve instrumented the kernel to get more informations on written registers and such.
But as you can see the i2c driver itself is working as expected.
The first actual error occurs on line 70 where it says "fence timeout on ".
I’m also very much confused why not a lot CSI registers are written here.

[  251.885292] ar0135 2-0018: ar0135_power_on: power on
[  251.885300] ar0135 ar0135_power_on 314
[  251.888751] csi tegra_csi_enum_framesizes 205
[  251.888761] csi tegra_csi_enum_framesizes 205
[  251.888816] ar0135 ar0135_power_off 360
[  251.888824] ar0135 2-0018: ar0135_power_off: power off
[  251.888828] ar0135 ar0135_power_off 374
[  251.962988] nvmap_alloc_handle: PID 1754: camera_recordin: WARNING: All NvMap Allocations must have a tag to identify the subsystem allocating memory.Please pass the tag to the API call NvRmMemHanldeAllocAttr() or relevant.
[  251.997432] ar0135 ar0135_power_on 302
[  251.997443] ar0135 2-0018: ar0135_power_on: power on
[  251.997446] ar0135 ar0135_power_on 314
[  252.000925] ar0135 ar0135_s_ctrl 936
[  252.000931] ar0135 ar0135_s_ctrl 936
[  252.000941] ar0135 ar0135_s_ctrl 936
[  252.000943] ar0135 ar0135_set_coarse_time 828
[  252.000946] ar0135 ar0135_s_ctrl 936
[  252.002235] vi4_init
[  252.002242] vi4_write ffffff8005280000 48 3f0000f9
[  252.002245] vi4_write ffffff8005280000 44 3f000001
[  252.002262] tegra-vi4 15700000.vi: master error
[  252.006819] tegra-vi4 15700000.vi: master error
[  252.006891] vi4_write ffffff8005280000 6020 1
[  252.006893] vi4_write ffffff8005280000 6000 e39c08e3
[  252.006899] csi tegra_csi_get_format 364
[  252.006901] ar0135 ar0135_s_stream 508
[  252.006905] ar0135 2-0018: ar0135_s_stream++
[  252.006906] ar0135 ar0135_write_table 274
[  252.006907] regmap_util_write_table_16_as_8
[  252.161786] ar0135 ar0135_set_coarse_time 828
[  252.161806] ar0135 ar0135_set_coarse_time_short 863
[  252.161821] ar0135 ar0135_write_table 274
[  252.161835] regmap_util_write_table_16_as_8
[  252.162774] ar0135 2-0018: ar0135_s_stream--
[  252.162790] tegra_csi_s_stream
[  252.164417] nvcsi 150c0000.nvcsi: csi port:2
[  252.164434] csi4_phy_write 1 20 700000
[  252.164813] tegra_csi_s_stream bypass
[  252.165017] ar0135 ar0135_s_ctrl 936
[  252.165034] ar0135 ar0135_s_ctrl 936
[  252.165070] ar0135 ar0135_s_ctrl 936
[  252.165082] ar0135 ar0135_set_coarse_time 828
[  252.165095] ar0135 ar0135_s_ctrl 936
[  252.168333] ar0135 ar0135_s_ctrl 936
[  252.168357] ar0135 ar0135_s_ctrl 936
[  252.169845] ar0135 ar0135_s_ctrl 936
[  252.169867] ar0135 ar0135_s_ctrl 936
[  253.665191] fence timeout on [ffffffc05ed73300] after 1500ms
[  253.665193] fence timeout on [ffffffc05ed73a00] after 1500ms
[  253.665195] fence timeout on [ffffffc05ed73400] after 1500ms

After that I’ve checked using v4l2-ctl which was suggested on some of the other threads.

sudo v4l2-ctl -d /dev/video0 --set-fmt-video=width=1280,height=960,pixelformat=RAW12 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=10 --stream-to=1080.raw --verbose

VIDIOC_QUERYCAP: ok
VIDIOC_S_EXT_CTRLS: ok
VIDIOC_G_FMT: ok
VIDIOC_ENUM_FMT: ok
VIDIOC_S_FMT: ok
Format Video Capture:
        Width/Height      : 1280/960
        Pixel Format      : 'RGGB'
        Field             : None
        Bytes per Line    : 1280
        Size Image        : 1228800
        Colorspace        : sRGB
        Transfer Function : Default
        YCbCr Encoding    : Default
        Quantization      : Default
        Flags             :
VIDIOC_REQBUFS: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_STREAMON: ok
        Index    : 0
        Type     : Video Capture
        Flags    : mapped, done
        Field    : None
        Sequence : 0
        Length   : 1228800
        Bytesused: 1228800
        Timestamp: 18446743798846512952.18446744073443089241s (Monotonic, End-of-Frame)

        Index    : 1
        Type     : Video Capture
        Flags    : mapped, done
        Field    : None
        Sequence : 1
        Length   : 1228800
        Bytesused: 1228800
        Timestamp: 18446743798846512952.18446744073443089241s (Monotonic, End-of-Frame)

 1.00 fps
        Index    : 2
        Type     : Video Capture
        Flags    : mapped, done
        Field    : None
        Sequence : 2
        Length   : 1228800
        Bytesused: 1228800
        Timestamp: 18446743798846512952.18446744073443089241s (Monotonic, End-of-Frame)

 1.00 fps
        Index    : 3
        Type     : Video Capture
        Flags    : mapped, done
        Field    : None
        Sequence : 3
        Length   : 1228800
        Bytesused: 1228800
        Timestamp: 18446743798846512952.18446744073443089241s (Monotonic, End-of-Frame)

 0.99 fps
        Index    : 0
        Type     : Video Capture
        Flags    : mapped, done
        Field    : None
        Sequence : 4
        Length   : 1228800
        Bytesused: 1228800
        Timestamp: 18446743798846512952.18446744073443089241s (Monotonic, End-of-Frame)

 0.99 fps
        Index    : 1
        Type     : Video Capture
        Flags    : mapped, done
        Field    : None
        Sequence : 5
        Length   : 1228800
        Bytesused: 1228800
        Timestamp: 18446743798846512952.18446744073443089241s (Monotonic, End-of-Frame)

 0.99 fps
        Index    : 2
        Type     : Video Capture
        Flags    : mapped, done
        Field    : None
        Sequence : 6
        Length   : 1228800
        Bytesused: 1228800
        Timestamp: 18446743798846512952.18446744073443089241s (Monotonic, End-of-Frame)

 0.99 fps
        Index    : 3
        Type     : Video Capture
        Flags    : mapped, done
        Field    : None
        Sequence : 7
        Length   : 1228800
        Bytesused: 1228800
        Timestamp: 18446743798846512952.18446744073443089241s (Monotonic, End-of-Frame)

 0.99 fps
        Index    : 0
        Type     : Video Capture
        Flags    : mapped, done
        Field    : None
        Sequence : 8
        Length   : 1228800
        Bytesused: 1228800
        Timestamp: 18446743798846512952.18446744073443089241s (Monotonic, End-of-Frame)

 0.99 fps
        Index    : 1
        Type     : Video Capture
        Flags    : mapped, done
        Field    : None
        Sequence : 9
        Length   : 1228800
        Bytesused: 1228800
        Timestamp: 18446743798846512952.18446744073443089241s (Monotonic, End-of-Frame)

 0.99 fps
VIDIOC_STREAMOFF: ok

This time the kernel log gives me much more information and actual writings to the vi4 registers are performed.

[  565.545566] ar0135 ar0135_power_on 302
[  565.545577] ar0135 2-0018: ar0135_power_on: power on
[  565.545580] ar0135 ar0135_power_on 314
[  565.551573] vi4_init
[  565.551583] vi4_write ffffff8005280000 48 3f0000f9
[  565.551588] vi4_write ffffff8005280000 44 3f000001
[  565.551593] vi4_write ffffff8005280000 6020 1
[  565.551597] vi4_write ffffff8005280000 6000 e39c08e3
[  565.551606] csi tegra_csi_get_format 364
[  565.551612] tegra_channel_capture_setup 357
[  565.551615] tegra_channel_notify_enable 242
[  565.552035] vi4_write ffffff8005280000 42c 1
[  565.552041] vi4_channel_write 0 20 13f1f
[  565.552046] vi4_channel_write 0 24 abf
[  565.552051] vi4_channel_write 0 2c 0
[  565.552055] vi4_channel_write 0 28 0
[  565.552059] vi4_channel_write 0 30 500
[  565.552064] vi4_channel_write 0 34 3c0
[  565.552068] vi4_channel_write 0 48 0
[  565.552072] vi4_channel_write 0 4c 500
[  565.552076] vi4_channel_write 0 58 500
[  565.552080] vi4_channel_write 0 50 0
[  565.552083] vi4_channel_write 0 54 3c0
[  565.552087] vi4_channel_write 0 5c 3c0
[  565.552090] vi4_channel_write 0 80 1
[  565.552094] vi4_channel_write 0 88 0
[  565.552097] vi4_channel_write 0 84 10
[  565.552100] vi4_channel_write 0 b8 0
[  565.552104] vi4_channel_write 0 ec 0
[  565.552107] vi4_channel_write 0 134 0
[  565.552110] vi4_channel_write 0 44 1000000
[  565.552114] vi4_channel_write 0 38 0
[  565.552117] vi4_channel_write 0 3c 0
[  565.552120] vi4_channel_write 0 120 0
[  565.552864] vi4_channel_write 0 108 0
[  565.552872] vi4_channel_write 0 10c 0
[  565.552879] vi4_channel_write 0 110 0
[  565.552885] vi4_channel_write 0 e0 5a600000
[  565.552890] vi4_channel_write 0 e8 500
[  565.552895] vi4_channel_write 0 e4 0
[  565.552899] vi4_channel_write 0 f0 0
[  565.552903] vi4_channel_write 0 f4 0
[  565.552908] vi4_channel_write 0 f8 0
[  565.552912] vi4_channel_write 0 fc 0
[  565.552916] vi4_channel_write 0 100 0
[  565.552920] vi4_channel_write 0 104 0
[  565.552926] ar0135 ar0135_s_stream 508
[  565.552935] ar0135 2-0018: ar0135_s_stream++
[  565.552940] ar0135 ar0135_write_table 274
[  565.552944] regmap_util_write_table_16_as_8
[  565.707851] ar0135 ar0135_write_table 274
[  565.707869] regmap_util_write_table_16_as_8
[  565.708805] ar0135 2-0018: ar0135_s_stream--
[  565.708822] tegra_csi_s_stream
[  565.710676] nvcsi 150c0000.nvcsi: csi port:2
[  565.710698] csi4_phy_write 1 20 700000
[  565.711107] tegra_csi_start_streaming
[  565.711137] nvcsi 150c0000.nvcsi: csi4_start_streaming ports index=2, lanes=2
[  565.711157] nvcsi 150c0000.nvcsi: csi4_stream_init
[  565.711172] csi4_stream_write 2 400 ffffffff
[  565.711187] csi4_stream_write 2 408 ffffffff
[  565.711199] csi4_stream_write 2 404 ffffffff
[  565.711211] csi4_stream_write 2 40c ffffffff
[  565.711222] csi4_stream_write 2 a4 3ffff
[  565.711234] csi4_stream_write 2 ac 7ffff
[  565.711245] csi4_stream_write 2 90 0
[  565.711256] csi4_stream_write 2 a8 0
[  565.711267] csi4_stream_write 2 b0 0
[  565.711283] nvcsi 150c0000.nvcsi: csi4_stream_config
[  565.711295] csi4_stream_write 2 6c 0
[  565.711306] csi4_stream_write 2 70 3
[  565.711317] csi4_stream_write 2 74 0
[  565.711328] csi4_stream_write 2 20 0
[  565.711349] nvcsi 150c0000.nvcsi: csi4_stream_config (2) read VC0_DPCM_CTRL = 00000000
[  565.711367] nvcsi 150c0000.nvcsi: csi4_phy_config
[  565.711379] csi4_phy_write 1 0 0
[  565.711398] nvcsi 150c0000.nvcsi: NVCSI_CIL_CONFIG = 00000000
[  565.711410] csi4_phy_write 1 18 3
[  565.711422] csi4_phy_write 1 4 0
[  565.711433] csi4_phy_write 1 20 70011
[  565.711445] csi4_phy_write 1 c 200
[  565.711457] csi4_phy_write 1 c 0
[  565.711468] csi4_phy_write 1 4 2
[  565.711479] csi4_phy_write 1 20 700000
[  565.711491] csi4_phy_write 1 5c 462194
[  565.711502] csi4_phy_write 1 18 0
[  565.711514] csi4_stream_write 2 8 1
[  565.711525] tegra_csi_s_stream exit with 0
[  565.711538] vi4_channel_write 0 4 1
[  565.711551] vi4_channel_write 0 1c 3
[  566.709297] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  566.715732] vi4_channel_write 0 108 0
[  566.715750] vi4_channel_write 0 10c 0
[  566.715764] vi4_channel_write 0 110 0
[  566.715779] vi4_channel_write 0 e0 5a800000
[  566.715791] vi4_channel_write 0 e8 500
[  566.715803] vi4_channel_write 0 e4 0
[  566.715814] vi4_channel_write 0 f0 0
[  566.715825] vi4_channel_write 0 f4 0
[  566.715836] vi4_channel_write 0 f8 0
[  566.715846] vi4_channel_write 0 fc 0
[  566.715857] vi4_channel_write 0 100 0
[  566.715868] vi4_channel_write 0 104 0
[  566.715879] vi4_channel_write 0 4 1
[  566.715890] vi4_channel_write 0 1c 3
[  567.713270] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  567.719686] vi4_channel_write 0 108 0
[  567.719703] vi4_channel_write 0 10c 0
[  567.719716] vi4_channel_write 0 110 0
[  567.719730] vi4_channel_write 0 e0 5aa00000
[  567.719743] vi4_channel_write 0 e8 500
[  567.719755] vi4_channel_write 0 e4 0
[  567.719767] vi4_channel_write 0 f0 0
[  567.719778] vi4_channel_write 0 f4 0
[  567.719788] vi4_channel_write 0 f8 0
[  567.719799] vi4_channel_write 0 fc 0
[  567.719810] vi4_channel_write 0 100 0
[  567.719821] vi4_channel_write 0 104 0
[  567.719832] vi4_channel_write 0 4 1
[  567.719843] vi4_channel_write 0 1c 3
[  568.717260] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  568.723690] vi4_channel_write 0 108 0
[  568.723708] vi4_channel_write 0 10c 0
[  568.723722] vi4_channel_write 0 110 0
[  568.723736] vi4_channel_write 0 e0 5ac00000
[  568.723748] vi4_channel_write 0 e8 500
[  568.723760] vi4_channel_write 0 e4 0
[  568.723772] vi4_channel_write 0 f0 0
[  568.723782] vi4_channel_write 0 f4 0
[  568.723793] vi4_channel_write 0 f8 0
[  568.723804] vi4_channel_write 0 fc 0
[  568.723814] vi4_channel_write 0 100 0
[  568.723825] vi4_channel_write 0 104 0
[  568.723837] vi4_channel_write 0 4 1
[  568.723848] vi4_channel_write 0 1c 3
[  569.721267] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  569.727725] vi4_channel_write 0 108 0
[  569.727743] vi4_channel_write 0 10c 0
[  569.727758] vi4_channel_write 0 110 0
[  569.727772] vi4_channel_write 0 e0 5a600000
[  569.727784] vi4_channel_write 0 e8 500
[  569.727797] vi4_channel_write 0 e4 0
[  569.727809] vi4_channel_write 0 f0 0
[  569.727820] vi4_channel_write 0 f4 0
[  569.727830] vi4_channel_write 0 f8 0
[  569.727841] vi4_channel_write 0 fc 0
[  569.727852] vi4_channel_write 0 100 0
[  569.727863] vi4_channel_write 0 104 0
[  569.727875] vi4_channel_write 0 4 1
[  569.727886] vi4_channel_write 0 1c 3
[  570.725268] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  570.731724] vi4_channel_write 0 108 0
[  570.731742] vi4_channel_write 0 10c 0
[  570.731756] vi4_channel_write 0 110 0
[  570.731789] vi4_channel_write 0 e0 5a800000
[  570.731802] vi4_channel_write 0 e8 500
[  570.731815] vi4_channel_write 0 e4 0
[  570.731826] vi4_channel_write 0 f0 0
[  570.731837] vi4_channel_write 0 f4 0
[  570.731848] vi4_channel_write 0 f8 0
[  570.731858] vi4_channel_write 0 fc 0
[  570.731886] vi4_channel_write 0 100 0
[  570.731897] vi4_channel_write 0 104 0
[  570.731909] vi4_channel_write 0 4 1
[  570.731920] vi4_channel_write 0 1c 3
[  571.729287] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  571.735737] vi4_channel_write 0 108 0
[  571.735755] vi4_channel_write 0 10c 0
[  571.735770] vi4_channel_write 0 110 0
[  571.735784] vi4_channel_write 0 e0 5aa00000
[  571.735796] vi4_channel_write 0 e8 500
[  571.735807] vi4_channel_write 0 e4 0
[  571.735819] vi4_channel_write 0 f0 0
[  571.735830] vi4_channel_write 0 f4 0
[  571.735841] vi4_channel_write 0 f8 0
[  571.735851] vi4_channel_write 0 fc 0
[  571.735862] vi4_channel_write 0 100 0
[  571.735874] vi4_channel_write 0 104 0
[  571.735885] vi4_channel_write 0 4 1
[  571.735896] vi4_channel_write 0 1c 3
[  572.733283] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  572.739719] vi4_channel_write 0 108 0
[  572.739735] vi4_channel_write 0 10c 0
[  572.739748] vi4_channel_write 0 110 0
[  572.739762] vi4_channel_write 0 e0 5ac00000
[  572.739773] vi4_channel_write 0 e8 500
[  572.739785] vi4_channel_write 0 e4 0
[  572.739796] vi4_channel_write 0 f0 0
[  572.739807] vi4_channel_write 0 f4 0
[  572.739818] vi4_channel_write 0 f8 0
[  572.739828] vi4_channel_write 0 fc 0
[  572.739839] vi4_channel_write 0 100 0
[  572.739850] vi4_channel_write 0 104 0
[  572.739861] vi4_channel_write 0 4 1
[  572.739872] vi4_channel_write 0 1c 3
[  573.737266] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  573.743701] vi4_channel_write 0 108 0
[  573.743718] vi4_channel_write 0 10c 0
[  573.743732] vi4_channel_write 0 110 0
[  573.743745] vi4_channel_write 0 e0 5a600000
[  573.743757] vi4_channel_write 0 e8 500
[  573.743768] vi4_channel_write 0 e4 0
[  573.743780] vi4_channel_write 0 f0 0
[  573.743791] vi4_channel_write 0 f4 0
[  573.743801] vi4_channel_write 0 f8 0
[  573.743812] vi4_channel_write 0 fc 0
[  573.743823] vi4_channel_write 0 100 0
[  573.743833] vi4_channel_write 0 104 0
[  573.743844] vi4_channel_write 0 4 1
[  573.743855] vi4_channel_write 0 1c 3
[  574.741265] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  574.747668] vi4_channel_write 0 108 0
[  574.747680] vi4_channel_write 0 10c 0
[  574.747690] vi4_channel_write 0 110 0
[  574.747700] vi4_channel_write 0 e0 5a800000
[  574.747708] vi4_channel_write 0 e8 500
[  574.747716] vi4_channel_write 0 e4 0
[  574.747725] vi4_channel_write 0 f0 0
[  574.747732] vi4_channel_write 0 f4 0
[  574.747739] vi4_channel_write 0 f8 0
[  574.747747] vi4_channel_write 0 fc 0
[  574.747754] vi4_channel_write 0 100 0
[  574.747762] vi4_channel_write 0 104 0
[  574.747769] vi4_channel_write 0 4 1
[  574.747777] vi4_channel_write 0 1c 3
[  575.745271] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  575.751719] vi4_channel_write 0 108 0
[  575.751739] vi4_channel_write 0 10c 0
[  575.751754] vi4_channel_write 0 110 0
[  575.751768] vi4_channel_write 0 e0 5aa00000
[  575.751780] vi4_channel_write 0 e8 500
[  575.751791] vi4_channel_write 0 e4 0
[  575.751803] vi4_channel_write 0 f0 0
[  575.751813] vi4_channel_write 0 f4 0
[  575.751824] vi4_channel_write 0 f8 0
[  575.751835] vi4_channel_write 0 fc 0
[  575.751846] vi4_channel_write 0 100 0
[  575.751856] vi4_channel_write 0 104 0
[  575.751867] vi4_channel_write 0 4 1
[  575.751878] vi4_channel_write 0 1c 3
[  576.749272] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  576.755697] vi4_channel_write 0 108 0
[  576.755713] vi4_channel_write 0 10c 0
[  576.755726] vi4_channel_write 0 110 0
[  576.755738] vi4_channel_write 0 e0 5ac00000
[  576.755750] vi4_channel_write 0 e8 500
[  576.755761] vi4_channel_write 0 e4 0
[  576.755773] vi4_channel_write 0 f0 0
[  576.755783] vi4_channel_write 0 f4 0
[  576.755794] vi4_channel_write 0 f8 0
[  576.755805] vi4_channel_write 0 fc 0
[  576.755815] vi4_channel_write 0 100 0
[  576.755826] vi4_channel_write 0 104 0
[  576.755837] vi4_channel_write 0 4 1
[  576.755848] vi4_channel_write 0 1c 3
[  577.753287] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  577.759713] vi4_channel_write 0 108 0
[  577.759728] vi4_channel_write 0 10c 0
[  577.759741] vi4_channel_write 0 110 0
[  577.759753] vi4_channel_write 0 e0 5a600000
[  577.759765] vi4_channel_write 0 e8 500
[  577.759776] vi4_channel_write 0 e4 0
[  577.759787] vi4_channel_write 0 f0 0
[  577.759799] vi4_channel_write 0 f4 0
[  577.759809] vi4_channel_write 0 f8 0
[  577.759820] vi4_channel_write 0 fc 0
[  577.759831] vi4_channel_write 0 100 0
[  577.759841] vi4_channel_write 0 104 0
[  577.759852] vi4_channel_write 0 4 1
[  577.759863] vi4_channel_write 0 1c 3
[  578.757275] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  578.763731] vi4_channel_write 0 108 0
[  578.763754] vi4_channel_write 0 10c 0
[  578.763771] vi4_channel_write 0 110 0
[  578.763786] vi4_channel_write 0 e0 5a800000
[  578.763799] vi4_channel_write 0 e8 500
[  578.763811] vi4_channel_write 0 e4 0
[  578.763824] vi4_channel_write 0 f0 0
[  578.763834] vi4_channel_write 0 f4 0
[  578.763845] vi4_channel_write 0 f8 0
[  578.763856] vi4_channel_write 0 fc 0
[  578.763867] vi4_channel_write 0 100 0
[  578.763878] vi4_channel_write 0 104 0
[  578.763889] vi4_channel_write 0 4 1
[  578.763900] vi4_channel_write 0 1c 3
[  579.761286] tegra-vi4 15700000.vi: ATOMP_FE syncpt timeout!
[  579.767299] ar0135 ar0135_s_stream 508
[  579.767327] ar0135 2-0018: ar0135_s_stream++
[  579.767341] ar0135 ar0135_update_ctrl_range 753
[  579.767390] ar0135 ar0135_write_table 274
[  579.767403] regmap_util_write_table_16_as_8
[  579.767415] tegra_csi_s_stream
[  579.768096] tegra_csi_stop_streaming
[  579.768136] nvcsi 150c0000.nvcsi: csi4_stop_streaming ports index=0, lanes=2
[  579.768162] nvcsi 150c0000.nvcsi: csi4_phy_config
[  579.768179] csi4_phy_write 1 0 0
[  579.768203] nvcsi 150c0000.nvcsi: NVCSI_CIL_CONFIG = 00000002
[  579.768217] csi4_phy_write 1 18 3
[  579.768229] csi4_phy_write 1 4 0
[  579.768241] csi4_phy_write 1 20 70011
[  579.768253] csi4_phy_write 1 c 200
[  579.768271] nvcsi 150c0000.nvcsi: csi4_stream_check_status
[  579.768296] nvcsi 150c0000.nvcsi: csi4_stream_check_status (2) INTR_STATUS 0x00000008
[  579.776292] nvcsi 150c0000.nvcsi: csi4_stream_check_status (2) ERR_INTR_STATUS 0x00000008
[  579.784546] nvcsi 150c0000.nvcsi: csi4_cil_check_status 276
[  579.784586] tegra_csi_s_stream exit with 0
[  579.800721] ar0135 ar0135_power_off 360
[  579.800743] ar0135 2-0018: ar0135_power_off: power off
[  579.800754] ar0135 ar0135_power_off 374

I’ll get these PXL_SOF syncpt timeout and I’ll ask myself if maybe the configuration of the CSI registers are not correct.
Sadly no Reference Manual for the TX2 is currently available so I can’t interpret them though.

I’ve found this thread:
https://devtalk.nvidia.com/default/topic/1007058/jetson-tx2/porting-tx1-camera-driver-to-kernel-4-4-15-pxl_sof-syncpt-timeout/3
Seems to be similar to mine. I have adapted the patches attached to post #38.
No change however.

Inside this thread someone suggested checking the clocks.

Currently our evaluation hardware allows two solder options for selecting the reference clock for our LVDS deserializer:
Connection to CAM1_MCLK and a persistent connection to an 25 MHz oscillator.
Currently I use the 25 MHz. But I assume that CAM1_MCLK would be the better option.

But I’m a little bit confused here about the names.
How is any CAMx_MCLK connected to the device tree.

clocks = <&tegra_car TEGRA186_CLK_EXTPERIPH2>,
 <&tegra_car TEGRA186_CLK_PLLP_OUT0>;
 clock-names = "extperiph2", "pllp_grtba";
 mclk = "extperiph2";

I can’t see the relation.
How do I select which pad?

Ok,

I’ve figured out that extperiph2 / EXTPERIPH2 leads to CAM1_MCLK while
extperiph1 / EXTPERIPH1 connects to CAM0_MCLK.

But even with connecting CAMx_MCLK to the reference clock of our LVDS deserializer I have no change.

@azeps
Does sensor output RAW12? The Bytes per line seems not correct.

Format Video Capture:
Width/Height : 1280/960
Pixel Format : ‘RGGB’
Field : None
Bytes per Line : 1280
Size Image : 1228800
Colorspace : sRGB
Transfer Function : Default
YCbCr Encoding : Default
Quantization : Default
Flags :

Thank you for pointing that out. I haven’t noticed that before. But the first execution of v4l2-ctl after rebooting is correct.
I also use SRGGB12 now as I still have configured the Bayer Pattern inside the device tree even if it doesn’t exist on our Imager. It’s more compatible to the ov5693 that way.

I don’t get it yet why the format changes.

nvidia@tegra-ubuntu-8991:~$ sudo v4l2-ctl -d /dev/video0 --set-fmt-video=width=1280,height=960,pixelformat=SRGGB12 --set-ctrl bypass_mode=0 --stream-mmap --stream-c                                                                        ount=10 --stream-to=1080.raw --verbose
VIDIOC_QUERYCAP: ok
VIDIOC_S_EXT_CTRLS: ok
VIDIOC_G_FMT: ok
VIDIOC_ENUM_FMT: ok
VIDIOC_S_FMT: ok
Format Video Capture:
        Width/Height      : 1280/960
        Pixel Format      : 'RGGB'
        Field             : None
        Bytes per Line    : 2560
        Size Image        : 2457600
        Colorspace        : sRGB
        Transfer Function : Default
        YCbCr Encoding    : Default
        Quantization      : Default
        Flags             :
VIDIOC_REQBUFS: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF:

........

nvidia@tegra-ubuntu-8991:~$ sudo v4l2-ctl -d /dev/video0 --set-fmt-video=width=1280,height=960,pixelformat=SRGGB12 --set-ctrl bypass_mode=0 --stream-mmap --stream-c                                                                        ount=10 --stream-to=1080.raw --verbose
VIDIOC_QUERYCAP: ok
VIDIOC_S_EXT_CTRLS: ok
VIDIOC_G_FMT: ok
VIDIOC_ENUM_FMT: ok
VIDIOC_S_FMT: ok
Format Video Capture:
        Width/Height      : 1280/960
        Pixel Format      : 'RGGB'
        Field             : None
        Bytes per Line    : 1280
        Size Image        : 1228800
        Colorspace        : sRGB
        Transfer Function : Default
        YCbCr Encoding    : Default
        Quantization      : Default
        Flags             :
VIDIOC_REQBUFS: ok
VIDIOC_QUERYBUF: ok

........

Wow!

I guess I’ve made a misconfiguration of v4l2-ctl.
RG12 seems to be the right pixelformat.

nvidia@tegra-ubuntu-8991:~$ sudo v4l2-ctl -d /dev/video0 --set-fmt-video=width=1280,height=960,pixelformat=RG12 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=1 --stream-to=1080.raw --verbose
VIDIOC_QUERYCAP: ok
VIDIOC_S_EXT_CTRLS: ok
VIDIOC_G_FMT: ok
VIDIOC_S_FMT: ok
Format Video Capture:
        Width/Height      : 1280/960
        Pixel Format      : 'RG12'
        Field             : None
        Bytes per Line    : 2560
        Size Image        : 2457600
        Colorspace        : sRGB
        Transfer Function : Default
        YCbCr Encoding    : Default
        Quantization      : Default
        Flags             :
VIDIOC_REQBUFS: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_STREAMON: ok
VIDIOC_DQBUF: failed: Input/output error
VIDIOC_STREAMOFF: ok

The kernel log has changed and this popped out.

[  721.731011] tegra_channel_notify_status_callback 223
[  721.731045] tegra-vi4 15700000.vi: Status:  7 channel:00 frame:0002
[  721.737422] tegra-vi4 15700000.vi:          timestamp sof 1385830822 eof 1386390795 data 0x00000001
[  721.746539] tegra-vi4 15700000.vi:          capture_id 0 stream  2 vchan  0
[  721.753568] tegra_channel_handle_error 205
[  722.712657] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11

This is a new development. I still don’t get any frames. But we’ve moved into the right direction.

The logs “PXL_SOF syncpt timeout!” means didn’t receive start of frame. You may need to probe the mipi signal to make sure it’s MIPI compatible.

Yes, this might be true. But something has changed. There is a Status message made available by tegra_channel_notify_status which didn’t happen before. But I can’t really interpret it.
I assume that the first frame was actually accepted and after that the SOF timeout occured.
Is this possible?

Our possibilities to probe MIPI signals are very limited as no logic analyzer with this frequency is available for us.

@azeps
You can try to increase the timeout time, current the value is 250ms I believe it should be enough for most of sensor.

@ShaneCCC:
Where do i find this timeout?
250ms indeed seems like a lot of time. I can’t imagine that this is really the flaw.

I do have new development nonetheless:

I do get calls to tegra_ivc_vi_notify_report(const struct vi_notify_msg_ex *msg).
These tell me about the status of the VI as it seems.

In my posts above i had a status value of 7.
It’s not official but this is my reverse engineering attempt:
The value of 7 means VI_CAPTURE_STATUS_CHANSEL_SHORT_FRAME.
These are “status” values according to vi-notfiy.h

/* This must match libnvvi API header and vi-notifier enum in FW */
enum {
	VI_CAPTURE_STATUS_NONE,
	VI_CAPTURE_STATUS_SUCCESS,
	VI_CAPTURE_STATUS_CSIMUX_FRAME,
	VI_CAPTURE_STATUS_CSIMUX_STREAM,
	VI_CAPTURE_STATUS_CHANSEL_FAULT,          //<-- 4
	VI_CAPTURE_STATUS_CHANSEL_FAULT_FE,
	VI_CAPTURE_STATUS_CHANSEL_COLLISION,
	VI_CAPTURE_STATUS_CHANSEL_SHORT_FRAME,    //<-- 7
	VI_CAPTURE_STATUS_ATOMP_PACKER_OVERFLOW,
	VI_CAPTURE_STATUS_ATOMP_FRAME_TRUNCATED,
	VI_CAPTURE_STATUS_ATOMP_FRAME_TOSSED,
	VI_CAPTURE_STATUS_ISPBUF_FIFO_OVERFLOW,
	VI_CAPTURE_STATUS_SYNC_FAILURE,
	VI_CAPTURE_STATUS_NOTIFIER_BACKEND_DOWN,
};

We had some issues with our deserializer. But now the resolution should be correct as the status had changed.

[  486.762193] tegra-vi4 15700000.vi: Status:  4 channel:00 frame:0002
[  486.768567] tegra-vi4 15700000.vi:          timestamp sof 2633521242 eof 2633521539 data 0x00000200
[  486.777671] tegra-vi4 15700000.vi:          capture_id 0 stream  2 vchan  0

The status changed to 4.
So I now have VI_CAPTURE_STATUS_CHANSEL_FAULT. What does that mean?
Could you please provide us with documentation about the VI and the CSI. It would be so helpful.
Is there a incomplete version of the yet missing Reference Manual that we could acquire?

@azeps
Please see the attached image.

External Media

It’s working !!!
We have frames.

The VI is very picky about its input frames. We had issues with our ds90ub964 as the documentation was kinda flawed.
The following steps helped us to reach our goals. I hope they solve similar problems discovered by other people:

  • The virtual channel needs to be 0 (we use CSI input C).
  • The datatype needs to be correct
  • Enable debugging support in the kernel. You get so much messages helping to solve that.
  • The LINE_LEN register of the ds90ub964 does has given us a length of 1920 at a resolution of 1280. I assumed that this is an configuration issue and lowered the resolution to at least get some frames into the TX2. This was wrong as the LINE_LEN of this deserializer is the word count. According to CSI2 specification the word count for RAW12 is 1.5 times the pixels as 2 pixels are packed together as 3 bytes. I know this has NOTHING to do with the TX2 and Nvidia. But maybe someone else falls into this trap.
  • The FV_MIN_TIME register of the ds90ub964 lead to the first line being discarded. As one line was missing from the expected resolution the VI ignored that frame completly.

This topic can be closed now as we have successfully grabbed frames with the following command:

sudo v4l2-ctl -d /dev/video0 --set-fmt-video=width=1280,height=960,pixelformat=RG12 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=1000 --stream-to=1080.raw --verbose

RG12 is ignored that way. The resulting images were greyscale as we wanted them.

in the camera dtsi, is maintaining "pixel_t = “bayer_bggr” the final solution for attaining RAW12 support?

Also in the modified driver, what was necessary to change the probing such that the interface comes up?
After that, where’s the injection point for adding the new source into a makefile?

Thanks in advance

pixel_t is the kinda RGB partner order totally different with RAW12, You can check the kernel sensor driver to know how to report it as RAW10/RAW12.

Shortly after getting frames we have discovered that the frames itself are incorrect.
Our 12 bit pixel values are wrong. They are shifted 2 bits to the left and the lower 2 bits are mirrored versions of the two most significant bits of the 12 bit value.

Otherwise explained:
00BB AAAA AAAA AABB
A are the 10 least significat bits of our 12 bit raw value.
B are the 2 most significant bits of our 12 bit raw value.
So essentially the 2 most significant bits are mirroed on the low ones.
I would expect something like:
0000 BBAA AAAA AAAA

So we are kinda not finished here.
14 bits of the 16 bits are set which is impossible considering the CSI data type is RAW12. So I assume a mistake on the side of the TX2.

Well but now to your question. In fact i had it set to bayer_bggr in the first place. But now I’ve altered the kernel to actually have support for raw12 which I didn’t find inside the kernel.
But it still has the same issue

Here is my device tree part of relevance:

mode0 { // AR0135_MODE_1280X960
	mclk_khz = "25000";
	num_lanes = "2";
	tegra_sinterface = "serial_c";
	discontinuous_clk = "no";
	dpcm_enable = "false";
	cil_settletime = "0";

	active_w = "1280";
	//active_w = "1920"; //FIXME
	active_h = "960";
	pixel_t = "raw12";
	readout_orientation = "90";
	line_length = "1388";
	inherent_gain = "1";
	mclk_multiplier = "6.67";
	//pix_clk_hz = "74250000";
	pix_clk_hz = "160000000";

	min_gain_val = "1.0";
	max_gain_val = "16";
	min_hdr_ratio = "1";
	max_hdr_ratio = "64";
	min_framerate = "1.816577";
	max_framerate = "54";
	min_exp_time = "34";
	max_exp_time = "550385";
	embedded_metadata_height = "0";
};

But this is not supported inside the official kernel.
I’ve added to core.c

{
		TEGRA_VF_RAW12,
		12,
		MEDIA_BUS_FMT_SBGGR12_1X12,
		2,
		TEGRA_IMAGE_FORMAT_T_R16_I,
		TEGRA_IMAGE_DT_RAW12,
		V4L2_PIX_FMT_SBGGR12,
		"BGBG.. GRGR..",
	},

        /* RAW12 - This is new */
	{
		TEGRA_VF_RAW12,
		12,
		MEDIA_BUS_FMT_Y12_1X12,
		2,
		TEGRA_IMAGE_FORMAT_T_R16_I,
		TEGRA_IMAGE_DT_RAW12,
		V4L2_PIX_FMT_Y12,
		"RAW 12 Bit Grey",
	},

	/* RGB888 */
	{
		TEGRA_VF_RGB888,
		24,
		MEDIA_BUS_FMT_RGB888_1X24,
		4,
		TEGRA_IMAGE_FORMAT_T_A8R8G8B8,
		TEGRA_IMAGE_DT_RGB888,
		V4L2_PIX_FMT_ABGR32,
		"BGRA-8-8-8-8",
	},

And also in camera_common.c

static const struct camera_common_colorfmt camera_common_color_fmts[] = {
	{
		MEDIA_BUS_FMT_SRGGB12_1X12,
		V4L2_COLORSPACE_SRGB,
		V4L2_PIX_FMT_SRGGB12,
	},
	{
		MEDIA_BUS_FMT_SRGGB10_1X10,
		V4L2_COLORSPACE_SRGB,
		V4L2_PIX_FMT_SRGGB10,
	},
	{
		MEDIA_BUS_FMT_SRGGB8_1X8,
		V4L2_COLORSPACE_SRGB,
		V4L2_PIX_FMT_SRGGB8,
	},
	{ /* This is new */
		MEDIA_BUS_FMT_Y12_1X12,
		V4L2_COLORSPACE_RAW,
		V4L2_PIX_FMT_Y12,
	},
};

It was strange that only SRGGB was supported in the function camera_common_try_fmt(*). So I’ve modded this at the end:

if (mf->code != MEDIA_BUS_FMT_SRGGB8_1X8 &&
		mf->code != MEDIA_BUS_FMT_SRGGB10_1X10 &&
		mf->code != MEDIA_BUS_FMT_SRGGB12_1X12 &&
		mf->code != MEDIA_BUS_FMT_Y12_1X12)
	{
		mf->code = MEDIA_BUS_FMT_SRGGB10_1X10;
		err = -EINVAL;
	}

	mf->field = V4L2_FIELD_NONE;
	mf->colorspace = V4L2_COLORSPACE_SRGB;
	printk("camera_common_try_fmt wants to have SRGB.\n");

	if (mf->code == MEDIA_BUS_FMT_Y12_1X12)
	{
		printk("camera_common_try_fmt - But I will force RAW at this point FIXME\n");
		mf->colorspace = V4L2_COLORSPACE_RAW;
	}

	return err;
}

I’ve also added a type to soc_mediabus.c as only Y10 was supported here. But no Y12.

.code = MEDIA_BUS_FMT_Y10_1X10,
	.fmt = {
		.fourcc			= V4L2_PIX_FMT_Y10,
		.name			= "Grey 10bit",
		.bits_per_sample	= 10,
		.packing		= SOC_MBUS_PACKING_EXTEND16,
		.order			= SOC_MBUS_ORDER_LE,
		.layout			= SOC_MBUS_LAYOUT_PACKED,
	},
}, {
	.code = MEDIA_BUS_FMT_Y12_1X12, /* This is new */
	.fmt = {
		.fourcc			= V4L2_PIX_FMT_Y12,
		.name			= "Grey 12bit",
		.bits_per_sample	= 12,
		.packing		= SOC_MBUS_PACKING_EXTEND16,
		.order			= SOC_MBUS_ORDER_LE,
		.layout			= SOC_MBUS_LAYOUT_PACKED,
	},
}, {
	.code = MEDIA_BUS_FMT_SBGGR10_2X8_PADHI_LE,
	.fmt = {
		.fourcc			= V4L2_PIX_FMT_SBGGR10,
		.name			= "Bayer 10 BGGR",
		.bits_per_sample	= 8,
		.packing		= SOC_MBUS_PACKING_2X8_PADHI,
		.order			= SOC_MBUS_ORDER_LE,
		.layout			= SOC_MBUS_LAYOUT_PACKED,
	},
}, {

Then I changed the support for the device tree in sensor_common.c:

static int extract_pixel_format(
	const char *pixel_t, u32 *format)
{
	size_t size = strnlen(pixel_t, OF_MAX_STR_LEN);

	if (strncmp(pixel_t, "bayer_bggr10", size) == 0)
		*format = V4L2_PIX_FMT_SBGGR10;
	else if (strncmp(pixel_t, "bayer_rggb10", size) == 0)
		*format = V4L2_PIX_FMT_SRGGB10;
	else if (strncmp(pixel_t, "bayer_bggr12", size) == 0)
		*format = V4L2_PIX_FMT_SBGGR12;
	else if (strncmp(pixel_t, "bayer_rggb12", size) == 0)
		*format = V4L2_PIX_FMT_SRGGB12;
	else if (strncmp(pixel_t, "bayer_wdr_pwl_rggb12", size) == 0)
		*format = V4L2_PIX_FMT_SRGGB12;
	else if (strncmp(pixel_t, "bayer_xbggr10p", size) == 0)
		*format = V4L2_PIX_FMT_XBGGR10P;
	else if (strncmp(pixel_t, "bayer_xrggb10p", size) == 0)
		*format = V4L2_PIX_FMT_XRGGB10P;
	else if (strncmp(pixel_t, "raw12", size) == 0)
	{
		*format = V4L2_PIX_FMT_Y12;
		printk("extract_pixel_format V4L2_PIX_FMT_Y12 *********************\n");
	}
	else {
		pr_err("%s: Need to extend format%s\n", __func__, pixel_t);
		return -EINVAL;
	}

	return 0;
}

Adding the driver to the kernel compile was done like this:

Adding

CONFIG_VIDEO_AR0135=y

to tegra18_defconfig.

Adding this to drivers/media/i2c/Kconfig:

config VIDEO_AR0135
        tristate "AR0135 camera sensor support"
        depends on I2C && VIDEO_V4L2 && VIDEO_V4L2_SUBDEV_API
        ---help---
          This is a Video4Linux2 sensor-level driver for the Aptina
          AR0135 imager.

Adding this to the Makefile next to that Kconfig:

obj-$(CONFIG_VIDEO_AR0135) += ar0135.o

hello azeps,

you could find the document from Embedded Download Center
[url]https://developer.nvidia.com/embedded/downloads[/url]

please refer to the Parker Series SoC Technical Reference Manual, in the CHAPTER 27: VIDEO INPUT (VI) about the raw memory format description as attach image.
External Media
thanks

I didn’t even knew this document existed. :-(
So this is how it is.
I can’t even find a way to deactivate this “feature” inside this document.
So I guess we have to adapt here.

Hello Azeps,

did you have to pull any special tricks in the device tree definitions to get the system properly probing / mounting to video0?
I’ve been following the nvidia guide and your previously posted snippets as well, but for some reason it’s causing some pretty bad results.

→ remove camera plugin manager from base
→ replace original ov5693 dtsi, with a renamed version for custom device
→ change compatibility string, to reflect new driver name
→ compile new driver, add to kernel as module.

Unfortunately, it bricks my system so hard that my screen goes black even if I comment out all all the camera dtsi includes, and just use a basic DTB. Insights?

A few updates for anyone else who’s following the same process. It turns out that even if all the configs and drivers are correct, the system doesn’t mount video0 until the driver is compiled into the kernel itself. Thankfully, the new 28.1 documentation mentioned this, because I was working with it as a module before to prevent bricking the kernel at boot.

I followed Azep’s clear instructions from above, and also activated monochrome RAW12 support. I initially couldn’t get the pixelformat to trigger, until I noticed that there was a space in the Y12 pixelformat when I ran a list-formats on v4l. Therefore, the command needed a quotation mark to capture a frame properly:

v4l2-ctl --set-fmt-video=width=1920,height=1080,pixelformat='Y12 '  --stream-mmap --stream-count=1 -d /dev/video0 --stream-to=csi_dump.raw  --verbose

Now, I’m having a similar but different problem for the bitwise formatting. I put my deserializer into pattern generation mode, putting out 8 bars.

Their programmed values are:
[‘AA’ ‘33’ ‘F0’ ‘7F’ ‘55’ ‘CC’ ‘0F’ ‘80’]
However, the raw stream coming in are:
[‘AA2A’ ‘CC0C’ ‘3F3C’ ‘DD1F’ ‘5515’ ‘3333’ ‘C003’ ‘2220’]
The closest that I can get is to read it in as little-endian, followed by the above instructions to bitmask out the first/last 2 bits, and bitshift 2 2:
[‘AAA’ ‘333’ ‘F0F’ ‘7F7’ ‘555’ ‘CCC’ ‘F0’ ‘808’]

This gets me close, but the 4 LSB are still wrong and the datatype wouldn’t be able to carry a full 12-bit magnitude. Any advice on something to try?

To debug, I’ve tried overriding the driver’s set_fmt function to explicitly call the RAW12 format in my driver.c file. It feels like something upstream in how the stream is unpacked.

static int ov5693_set_fmt(struct v4l2_subdev *sd,
struct v4l2_subdev_pad_config *cfg,
struct v4l2_subdev_format *format)
{
int ret;

printk("Requesting format: %u\n", format->format.code);
printk("Overriding format to RAW12 1080p: %u\n", MEDIA_BUS_FMT_Y12_1X12);
format->format.code = MEDIA_BUS_FMT_Y12_1X12; 
format->format.colorspace = V4L2_COLORSPACE_RAW; 
format->format.width = 1920;
format->format.height = 1080;
format->format.field = V4L2_FIELD_NONE; 


if (format->which == V4L2_SUBDEV_FORMAT_TRY){
ret = camera_common_try_fmt(sd, &format->format);
}
else{
ret = camera_common_s_fmt(sd, &format->format);
}

return ret;
}