How to grab pre-configured csi video stream without I2C?

Hello Forum!

Is there an way to grab a pre-configured CSI-2 video stream?

we are using a TX2 dev board with R28.1.

The input video stream is generated by a SDI-to-CSI2 adapter, it’s a 4-lane, 1920x1080@60, YUV422 signal. And I connected the input MIPI-CSI data and clk lines directly to the CSI port of tx2.

What should I do to grab the video stream through the /dev/video0 device? Should I need to update device tree or modify the csi-camera drivers to register a camera device? what is the correct procedure?

We’ve tried to modify the /kernel/drivers/media/i2c/ov5693.c to ignore the i2c part and recompiled the kernel source. But still cannot get the /dev/video0, should I modify dts file? which dts or dtsi file should be used? because there are a few dts files and dtsi files, i am a little confused here.

I’ve been stucked here by this problem for quite some time.

Thanks a lot!

@DaLT
Did you check the sensor programing guide? Please check it. There’s tip you need to disable the plugin manager.

Thanks for your replay!

I read the sensor programming guide, and found that
" Because Tegra uses Plugin Manager by default, you must first unregister Plugin Manager support first and then add your device information to the main device tree DTSI file."

So I need to modify “kernel/arch/arm64/boot/dts/tegra210-plugin-manager/tegra210-jetson-cv-plugin-manager.dtsi” to disbale plugin manager and add a new device in “tegra210-platforms/tegra210-jetson-cv-camera-modules.dtsi”, Is this right?

And do I need to modify driver source “i2c/ov5693.c”?

You can follow ov5693.c as template to implement your sensor driver.

Thanks for your patience.

I still have no clue to skip i2c config just directly grab frame data from csi port.

could you please point me the direction I should focus on?

@DaLT
You can just keep the i2c struct and just don’t access the i2c bus.

@ShaneCCC

Thanks.

In the R28.1 user guide, there is a tip says:
“Use tegra210-jetson-cv-camera-modules.dtsi or tegra18x-quill-cv-camera-modules.dtsi as a model for generating your DTSI. In your file, change status = disable to status = okay.”

But I cannot find those dtsi files in R28.1 kernel source.

Is there another model file we can use?

@DaLT
They move the …/hardware/nvidia/…/…

@ShaneCCC, Yes, I look into the directory : hardware/nvidia/platform/t18x/common/kernel-dts/t18x-common-platforms/ , cannot find tegra18x-quill-cv-camera-modules.dtsi file.

Do you reference to the l4t-documentation-28.1 ??
The file should be “tegra186-quill-camera-modules.dtsi” not tegra18x-quill-cv-camera-modules.dtsi

Using Main Platform Device Tree File
Register your new device by updating main platform DTSI file to include your new device DTSI file. Because Tegra uses Plugin Manager by default, you must first unregister Plugin Manager support, then add your device information to the main device tree DTSI file.
Prerequisites
• You have obtained the kernel source files. For more information, see Synchronizing the Kernel in the Development Guide for the release.
To register a device using main-platform device tree files
1. Locate and edit the following file:
• TX1:
hardware/nvidia/platform/t210/jetson/kernel-dts/tegra210-jetson-cv-base-p25970-2180-a00.dts
• TX2:
hardware/nvidia/platform/t18x/quill/kernel-dts/tegra186-quill-p3310-1000-a00-00-base.dtsi
2. In the DTSI, remove the following line:
• TX1:
#include "jetson-plugin-manager/tegra210-jetson-cv-camera-plugin-manager.dtsi"
• TX2:
#include <t18x-common-plugin-manager/tegra186-quill-camera-plugin-manager.dtsi"
3. Locate and edit the following file:
TX1:
hardware/nvidia/platform/t210/jetson/kernel-dts/tegra210-jetson-cv-base-p2597-2180-a00.dts
TX2:
hardware/nvidia/platform/t18x/quill/kernel-dts/tegra186-quill-p3310-1000-a00-00-base.dts
4. In the DTS file, replace the following line:
• TX1:
#include "jetson-platforms/tegra210-jetson-cv-camera-modules.dtsi"
• TX2:
#include <t18x-common-platforms/tegra186-quill-camera-modules.dtsi>
With an #include statement specifying the DTSI file for your new device.

@ShaneCCC

Yes, I reference to the l4t-documentation-28.1.

But after the lines above, there is a “Note”:

Note:
Use tegra210-jetson-cv-camera-modules.dtsi or tegra18x-quill-cv-camera-modules.dtsi as a model for generating your DTSI. In your file, change status = disable to status = okay.

But I cannot find this model file.

@ShaneCCC

I reference to jetsonHacks’ build kernel scripts:

and find that the scripts does not make dtbs.
So do I need to add “make dtbs” in the makeKernel.sh and after compiling, copy arch/arm64/boot/dtb/tegra18xxxxx.dtb to /boot/?

And I also find that in /boot/extlinux/extlinux/extlinux.conf, no “FDT xxxxx.dtb” line. So if I want to modify device tree, should I need to add this line and which dtb should be used?

Quite confused. Thanks for your reply.

I’m following you guys discussion. If I understand correctly, the references to the discussion are in “NVIDIA Tegra Linux Driver Package, Development Guide, 28.1 Release”. The documentation can be downloaded by:

  1. Go to “Linux For Tegra R28.1” site. https://developer.nvidia.com/embedded/linux-tegra
  2. Under the section “28.1 Driver Package”, click on “Documentation” and download the tar file.
  3. Untar the package, check out Start_L4T_Docs.html
  4. The topics are in the section “Camera Development, Sensor Driver Programming Guide”

Hope it helps for those who are interested in the discussion.

@ytmx3, Thanks for your supplementary information. Your reply makes this discussion much clearer.

@ShaneCCC, and I just found that R28.1 no longer use /boot/extlinux/extlinux.conf to FDT .dtb. So cross-compilation and flashing from host to device is the only way to modify and update the device tree, am I right?

Yes, correct please use flash command instead of replace the DTB file.

Hi, ShaneCCC, thanks for your replay.

I already updated the device tree blobs and made the ov5693_probe function be called successfully, but still cannot make /dev/video0 show. Is there something wrong? I just skip the i2c register configuration. Here is my modified probe:

static int ov5693_probe(struct i2c_client *client,
			const struct i2c_device_id *id)
{
	struct camera_common_data *common_data;
	struct device_node *node = client->dev.of_node;
	struct ov5693 *priv;
	char debugfs_name[10];
	int err;

	pr_info("[OV5693]: ---probing v4l2 sensor.---\n");

	if (!IS_ENABLED(CONFIG_OF) || !node)
		return -EINVAL;

	common_data = devm_kzalloc(&client->dev,
			    sizeof(struct camera_common_data), GFP_KERNEL);
	if (!common_data)
		return -ENOMEM;
	pr_info("[OV5693]: devm_kzalloc(&client->dev,.\n");
	priv = devm_kzalloc(&client->dev,
			    sizeof(struct ov5693) + sizeof(struct v4l2_ctrl *) *
			    ARRAY_SIZE(ctrl_config_list),
			    GFP_KERNEL);
	if (!priv)
		return -ENOMEM;
	pr_info("[OV5693]: devm_regmap_init_i2c.\n");
	priv->regmap = devm_regmap_init_i2c(client, &ov5693_regmap_config);
	if (IS_ERR(priv->regmap)) {
		dev_err(&client->dev,
			"regmap init failed: %ld\n", PTR_ERR(priv->regmap));
		return -ENODEV;
	}
	pr_info("[OV5693]: ov5693_parse_dt.\n");
	priv->pdata = ov5693_parse_dt(client);
	if (PTR_ERR(priv->pdata) == -EPROBE_DEFER)
		return -EPROBE_DEFER;
	if (!priv->pdata) {
		dev_err(&client->dev, "unable to get platform data\n");
		return -EFAULT;
	}

	common_data->ops		= &ov5693_common_ops;
	common_data->ctrl_handler	= &priv->ctrl_handler;
	common_data->i2c_client		= client;
	common_data->frmfmt		= ov5693_frmfmt;
	common_data->colorfmt		= camera_common_find_datafmt(
					  OV5693_DEFAULT_DATAFMT);
	common_data->power		= &priv->power;
	common_data->ctrls		= priv->ctrls;
	common_data->priv		= (void *)priv;
	common_data->numctrls		= ARRAY_SIZE(ctrl_config_list);
	common_data->numfmts		= ARRAY_SIZE(ov5693_frmfmt);
	common_data->def_mode		= OV5693_DEFAULT_MODE;
	common_data->def_width		= OV5693_DEFAULT_WIDTH;
	common_data->def_height		= OV5693_DEFAULT_HEIGHT;
	common_data->fmt_width		= common_data->def_width;
	common_data->fmt_height		= common_data->def_height;
	common_data->def_clk_freq	= OV5693_DEFAULT_CLK_FREQ;

	priv->i2c_client = client;
	priv->s_data			= common_data;
	priv->subdev			= &common_data->subdev;
	priv->subdev->dev		= &client->dev;
	priv->s_data->dev		= &client->dev;
	pr_info("[OV5693]: ov5693_power_get.\n");
	err = ov5693_power_get(priv);
	if (err)
		return err;
	pr_info("[OV5693]: camera_common_parse_ports.\n");
	err = camera_common_parse_ports(client, common_data);
	if (err) {
		dev_err(&client->dev, "Failed to find port info\n");
		return err;
	}
	sprintf(debugfs_name, "ov5693_%c", common_data->csi_port + 'a');
	dev_dbg(&client->dev, "%s: name %s\n", __func__, debugfs_name);
	camera_common_create_debugfs(common_data, debugfs_name);
	pr_info("[OV5693]: v4l2_i2c_subdev_init.\n");
	v4l2_i2c_subdev_init(priv->subdev, client, &ov5693_subdev_ops);
	       
	/* eeprom interface */
	err = ov5693_eeprom_device_init(priv);
	if (err && priv->pdata->has_eeprom)
		dev_err(&client->dev,
			"Failed to allocate eeprom reg map: %d\n", err);
	pr_info("[OV5693]: ov5693_ctrls_init.\n");
	err = ov5693_ctrls_init(priv, !err);
	if (err) {
		pr_info("[OV5693]: ov5693_ctrls_init failed.\n");
		return err;
	}

	priv->subdev->internal_ops = &ov5693_subdev_internal_ops;
	priv->subdev->flags |= V4L2_SUBDEV_FL_HAS_DEVNODE |
			       V4L2_SUBDEV_FL_HAS_EVENTS;

#if defined(CONFIG_MEDIA_CONTROLLER)
	pr_info("[OV5693]: CONFIG_MEDIA_CONTROLLER.\n");
	priv->pad.flags = MEDIA_PAD_FL_SOURCE;
	priv->subdev->entity.type = MEDIA_ENT_T_V4L2_SUBDEV_SENSOR;
	priv->subdev->entity.ops = &ov5693_media_ops;
	err = media_entity_init(&priv->subdev->entity, 1, &priv->pad, 0);
	if (err < 0) {
		dev_err(&client->dev, "unablvsse to init media entity\n");
		pr_info("[OV5693]: unable to init media entity.\n");
		return err;
	}
#endif
	pr_info("[OV5693]: v4l2_async_register_subdev.\n");
	err = v4l2_async_register_subdev(priv->subdev);
		
	
	if (err) {
		pr_info("[OV5693]: unable to v4l2_async_register_subdev.\n");
		return err;
	}

	dev_dbg(&client->dev, "Detected OV5693 sensor\n");
	pr_info("[OV5693]: Detected OV5693 sensor.\n");

	return 0;
}

This probe function can be called successfully and run with no err, why there is still no /dev/video0?lease use flash command instead of replace the DTB file.

[/quote]

Please check the trace the video_register_device() in channel.c

int tegra_vi_channels_register(struct tegra_mc_vi *vi)
{
        int ret = 0;
        struct tegra_channel *it;
        int count = 0;

        list_for_each_entry(it, &vi->vi_chans, list) {
                struct v4l2_subdev *sd = it->subdev_on_csi;
                bool is_csi = false;

                if (sd) {
                        /*
                         * If subdevice on csi is csi itself,
                         * then sensor subdevice is not connected
                         */
                        is_csi = strstr(sd->name, "nvcsi") != NULL;

                        if (is_csi)
                                continue;
                } else
                        continue;

                if (!it->init_done)
                        continue;
                ret = video_register_device(&it->video, VFL_TYPE_GRABBER, -1);
                if (ret < 0) {
                        dev_err(&it->video.dev, "failed to register %s\n",
                                it->video.name);
                        continue;
                }
                count++;
        }

        if (count == 0) {

@ShaneCCC,thanks for your help.

I’ve solved that problem and can get /dev/video0 now.

But there is another one that the input is a 1080p@60, YV12 format stream signal, what format params should be used in device tree node and how to preview the input stream?

Thanks again for your help!

Try below command and check the test.raw is validate first.

v4l2-ctl -d /dev/video0 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=1 --stream-to=test.raw