custom camera driver with gstreamer v4l2src

Hello

I made driver for ar0132, that can work with yavta.
I can capture one frame in raw10 format and transform it to bmp. All looks fine. Now i am trying to capture avi file with gstreamer. For beginning i try to see video on screen:

gst-launch-1.0 v4l2src ! xvimagesink

i see only black screen in window. If i set export GST_DEBUG=6, i see a lot of debug messages: LOG, DEBUG and INFO types. Only that is strange for me is strings:

0:00:01.253140733  7254       0x518940 DEBUG                   v4l2 gstv4l2object.c:2474:gst_v4l2_object_probe_caps_for_format:<v4l2src0> Enumerating frame sizes for RGGB
0:00:01.253181672  7254       0x518940 LOG                     v4l2 gstv4l2object.c:2482:gst_v4l2_object_probe_caps_for_format:<v4l2src0> got discrete frame size 1280x960
0:00:01.253210318  7254       0x518940 LOG                     v4l2 gstv4l2object.c:2210:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2src0> get frame interval for 1280x960, RGGB
0:00:01.253252663  7254       0x518940 LOG                     v4l2 gstv4l2object.c:2234:gst_v4l2_object_probe_caps_for_format_and_size:<v4l2src0> adding discrete framerate: 45/1
0:00:01.277707008  7254       0x518940 DEBUG                   v4l2 gstv4l2object.c:1955:gst_v4l2_object_get_colorspace: Unknown enum v4l2_colorspace 0

Unknown enum v4l2_colorspace 0 is message in string 2010 (GStreamer (Good Plugins): sys/v4l2/gstv4l2object.c | Fossies) and i dont understand why colorspace is zero.

my dmesg log:

[95107.468336] ar0132 6-0010: camera_common_s_fmt(4106) size 1280 x 960
[95107.474712] camera_common_colorfmt i = 5 
[95107.478765] ar0132 6-0010: camera_common_try_fmt: size 1280 x 960
[95107.499489] ar0132 6-0010: camera_common_s_fmt: ret = 0
[95107.504710] camera_common_s_fmt:0
[95107.508031] ar0132_set_format:0
[95107.522312] ar0132_get_format
[95107.525311] ar0132 6-0010: camera_common_g_fmt++
[95107.529947] ar0132 6-0010: 1 fmt->code = 4106
[95107.534644] ar0132 6-0010: 2 fmt->colorspace = 8
[95107.539285] ar0132 6-0010: 3 s_data->fmt_width = 1280
[95107.544347] ar0132 6-0010: 4 s_data->fmt_height = 960
[95107.561572] vi 54080000.vi: Calibrate csi port 2
[95107.566401] ar0132 6-0010: ar0132_s_stream++ enable 1
[95107.580919] ar0132 6-0010: [ar0132]: write i2c-dev:0x10 addr:0x3088= 0x8000 - OK
[95107.600953] ar0132 6-0010: [ar0132]: write i2c-dev:0x10 addr:0x3086= 0x225 - OK
...
[95109.760966] ar0132 6-0010: [ar0132]: write i2c-dev:0x10 addr:0x3012= 0x2a0 - OK
[95109.768292] [ar0132] ar0132_linear_mode_setup ret=0
[95109.785230] ar0132 6-0010: [ar0132]: write i2c-dev:0x10 addr:0x3032= 0x0 - OK
[95109.805234] ar0132 6-0010: [ar0132]: write i2c-dev:0x10 addr:0x3002= 0x2 - OK
...
[95110.349221] ar0132 6-0010: [ar0132]: write i2c-dev:0x10 addr:0x3126= 0x80 - OK
[95110.369004] ar0132 6-0010: [ar0132]: write i2c-dev:0x10 addr:0x311c= 0x3dd - OK
[95110.392875] ar0132 6-0010: [ar0132]: write i2c-dev:0x10 addr:0x311e= 0x2 - OK
[95110.400038] [ar0132] ar0132_set_autoexposure ret=0
[95110.421243] ar0132 6-0010: [ar0132]: write i2c-dev:0x10 addr:0x3070= 0x0 - OK
[95110.428392] ar0132 6-0010: ret: 00
/* Find a data format by a pixel code in an array */
const struct camera_common_colorfmt *camera_common_find_datafmt(
		unsigned int code)
{
	int i;
	
	for (i = 0; i < ARRAY_SIZE(camera_common_color_fmts); i++)
		if (camera_common_color_fmts[i].code == code) {
			pr_info("camera_common_colorfmt i = %d \n", i);
			return camera_common_color_fmts + i;
		}

	pr_info("camera_common_colorfmt return NULL code = %d\n", code);
	return NULL;
}
EXPORT_SYMBOL(camera_common_find_datafmt);
...
int camera_common_s_fmt(struct v4l2_subdev *sd, struct v4l2_mbus_framefmt *mf)
{
	struct i2c_client *client = v4l2_get_subdevdata(sd);
	struct camera_common_data *s_data = to_camera_common_data(client);
	int ret;

	dev_err(&client->dev, "%s(%u) size %i x %i\n", __func__,
			mf->code, mf->width, mf->height);

	/* MIPI CSI could have changed the format, double-check */
	if (!camera_common_find_datafmt(mf->code)) {
		dev_err(&client->dev, "%s: !camera_common_find_datafmt(mf->code)\n", __func__);	
		return -EINVAL;
	}

	ret = camera_common_try_fmt(sd, mf);

	s_data->colorfmt = camera_common_find_datafmt(mf->code);

	dev_err(&client->dev, "%s: ret = %d\n", __func__, ret);
	
	return ret;
}

int camera_common_g_fmt(struct v4l2_subdev *sd, struct v4l2_mbus_framefmt *mf)
{
	struct i2c_client *client = v4l2_get_subdevdata(sd);
	struct camera_common_data *s_data = to_camera_common_data(client);
	const struct camera_common_colorfmt *fmt = s_data->colorfmt;

	dev_err(&client->dev, "%s++\n", __func__);

	mf->code	= fmt->code;
	dev_err(&client->dev, "1 fmt->code = %d\n", fmt->code);
	mf->colorspace	= fmt->colorspace;
	dev_err(&client->dev, "2 fmt->colorspace = %d\n", fmt->colorspace);
	mf->width	= s_data->fmt_width;
	dev_err(&client->dev, "3 s_data->fmt_width = %d\n", s_data->fmt_width);
	mf->height	= s_data->fmt_height;
	dev_err(&client->dev, "4 s_data->fmt_height = %d\n", s_data->fmt_height);
	mf->field	= V4L2_FIELD_NONE;
	mf->xfer_func = fmt->xfer_func;
	mf->ycbcr_enc = fmt->ycbcr_enc;
	mf->quantization = fmt->quantization;

	return 0;
}

What i doing wrong? I have not ideas how debug this error, may be somebody can help me?
thanks!

What format provides your camera driver ?

v4l2-ctl --list-formats

If your driver only outputs bayer 10 bits, then you cannot use v4l2src, it only supports outputing 8bits bayer. Check the src caps with:

gst-inspect-1.0 v4l2src

What format provides your camera driver ?

nvidia@tegra-ubuntu:~$ v4l2-ctl --list-formats
ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'RGGB'
	Name        : 8-bit Bayer RGRG/GBGB

	Index       : 1
	Type        : Video Capture
	Pixel Format: 'RG10'
	Name        : 10-bit Bayer RGRG/GBGB

	Index       : 2
	Type        : Video Capture
	Pixel Format: 'BA10'
	Name        : 10-bit Bayer GRGR/BGBG

	Index       : 3
	Type        : Video Capture
	Pixel Format: 'BG10'
	Name        : 10-bit Bayer BGBG/GRGR

	Index       : 4
	Type        : Video Capture
	Pixel Format: 'RG12'
	Name        : 12-bit Bayer RGRG/GBGB

	Index       : 5
	Type        : Video Capture
	Pixel Format: 'AR24'
	Name        : 32-bit BGRA 8-8-8-8

Check the src caps with:

nvidia@tegra-ubuntu:~$ gst-inspect-1.0 v4l2src
Factory Details:
  Rank                     primary (256)
  Long-name                Video (video4linux2) Source
  Klass                    Source/Video
  Description              Reads frames from a Video4Linux2 device
  Author                   Edgard Lima <edgard.lima@indt.org.br>, Stefan Kost <ensonic@users.sf.net>

Plugin Details:
  Name                     video4linux2
  Description              elements for Video 4 Linux
  Filename                 /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstvideo4linux2.so
  Version                  1.8.3
  License                  LGPL
  Source module            gst-plugins-good
  Source release date      2016-08-19
  Binary package           GStreamer Good Plugins (Ubuntu)
  Origin URL               https://launchpad.net/distros/ubuntu/+source/gst-plugins-good1.0

GObject
 +----GInitiallyUnowned
       +----GstObject
             +----GstElement
                   +----GstBaseSrc
                         +----GstPushSrc
                               +----GstV4l2Src

Implemented Interfaces:
  GstURIHandler
  GstTuner
  GstColorBalance
  GstVideoOrientation

Pad Templates:
  SRC template: 'src'
    Availability: Always
    Capabilities:
      image/jpeg
      video/mpeg
            mpegversion: 4
           systemstream: false
      video/mpeg
            mpegversion: 2
      video/mpegts
           systemstream: true
      video/x-bayer
                 format: { bggr, gbrg, grbg, rggb }
                  width: [ 1, 32768 ]
                 height: [ 1, 32768 ]
              framerate: [ 0/1, 2147483647/1 ]
      video/x-dv
           systemstream: true
      video/x-h263
                variant: itu
      video/x-h264
          stream-format: { byte-stream, avc }
              alignment: au
      video/x-pwc1
                  width: [ 1, 32768 ]
                 height: [ 1, 32768 ]
              framerate: [ 0/1, 2147483647/1 ]
      video/x-pwc2
                  width: [ 1, 32768 ]
                 height: [ 1, 32768 ]
              framerate: [ 0/1, 2147483647/1 ]
      video/x-raw
                 format: { RGB16, BGR, RGB, GRAY8, GRAY16_LE, GRAY16_BE, YVU9, YV12, YUY2, YVYU, UYVY, Y42B, Y41B, YUV9, NV12_64Z32, NV24, NV61, NV16, NV21, NV12, I420, BGRA, BGRx, ARGB, xRGB, BGR15, RGB15 }
                  width: [ 1, 32768 ]
                 height: [ 1, 32768 ]
              framerate: [ 0/1, 2147483647/1 ]
      video/x-sonix
                  width: [ 1, 32768 ]
                 height: [ 1, 32768 ]
              framerate: [ 0/1, 2147483647/1 ]
      video/x-vp8
      video/x-wmv
             wmvversion: 3
                 format: WVC1


Element Flags:
  no flags set

Element Implementation:
  Has change_state() function: 0x7f92f18708

Element has no clocking capabilities.

URI handling capabilities:
  Element can act as source.
  Supported URI protocols:
    v4l2

Pads:
  SRC: 'src'
    Pad Template: 'src'

Element Properties:
  name                : The name of the object
                        flags: readable, writable
                        String. Default: "v4l2src0"
  parent              : The parent of the object
                        flags: readable, writable
                        Object of type "GstObject"
  blocksize           : Size in bytes to read per buffer (-1 = default)
                        flags: readable, writable
                        Unsigned Integer. Range: 0 - 4294967295 Default: 4096 
  num-buffers         : Number of buffers to output before sending EOS (-1 = unlimited)
                        flags: readable, writable
                        Integer. Range: -1 - 2147483647 Default: -1 
  typefind            : Run typefind before negotiating
                        flags: readable, writable
                        Boolean. Default: false
  do-timestamp        : Apply current stream time to buffers
                        flags: readable, writable
                        Boolean. Default: false
  device              : Device location
                        flags: readable, writable
                        String. Default: "/dev/video0"
  device-name         : Name of the device
                        flags: readable
                        String. Default: "vi-output, ar0132 6-0010"
  device-fd           : File descriptor of the device
                        flags: readable
                        Integer. Range: -1 - 2147483647 Default: -1 
  flags               : Device type flags
                        flags: readable
                        Flags "GstV4l2DeviceTypeFlags" Default: 0x00000000, "(none)"
                           (0x00000001): capture          - Device supports video capture
                           (0x00000002): output           - Device supports video playback
                           (0x00000004): overlay          - Device supports video overlay
                           (0x00000010): vbi-capture      - Device supports the VBI capture
                           (0x00000020): vbi-output       - Device supports the VBI output
                           (0x00010000): tuner            - Device has a tuner or modulator
                           (0x00020000): audio            - Device has audio inputs or outputs
  brightness          : Picture brightness, or more precisely, the black level
                        flags: readable, writable, controllable
                        Integer. Range: -2147483648 - 2147483647 Default: 0 
  contrast            : Picture contrast or luma gain
                        flags: readable, writable, controllable
                        Integer. Range: -2147483648 - 2147483647 Default: 0 
  saturation          : Picture color saturation or chroma gain
                        flags: readable, writable, controllable
                        Integer. Range: -2147483648 - 2147483647 Default: 0 
  hue                 : Hue or color balance
                        flags: readable, writable, controllable
                        Integer. Range: -2147483648 - 2147483647 Default: 0 
  norm                : video standard
                        flags: readable, writable
                        Enum "V4L2_TV_norms" Default: 0, "none"
                           (0): none             - none
                           (45056): NTSC             - NTSC
                           (4096): NTSC-M           - NTSC-M
                           (8192): NTSC-M-JP        - NTSC-M-JP
                           (32768): NTSC-M-KR        - NTSC-M-KR
                           (16384): NTSC-443         - NTSC-443
                           (255): PAL              - PAL
                           (7): PAL-BG           - PAL-BG
                           (1): PAL-B            - PAL-B
                           (2): PAL-B1           - PAL-B1
                           (4): PAL-G            - PAL-G
                           (8): PAL-H            - PAL-H
                           (16): PAL-I            - PAL-I
                           (224): PAL-DK           - PAL-DK
                           (32): PAL-D            - PAL-D
                           (64): PAL-D1           - PAL-D1
                           (128): PAL-K            - PAL-K
                           (256): PAL-M            - PAL-M
                           (512): PAL-N            - PAL-N
                           (1024): PAL-Nc           - PAL-Nc
                           (2048): PAL-60           - PAL-60
                           (16711680): SECAM            - SECAM
                           (65536): SECAM-B          - SECAM-B
                           (262144): SECAM-G          - SECAM-G
                           (524288): SECAM-H          - SECAM-H
                           (3276800): SECAM-DK         - SECAM-DK
                           (131072): SECAM-D          - SECAM-D
                           (1048576): SECAM-K          - SECAM-K
                           (2097152): SECAM-K1         - SECAM-K1
                           (4194304): SECAM-L          - SECAM-L
                           (8388608): SECAM-Lc         - SECAM-Lc
  io-mode             : I/O mode
                        flags: readable, writable
                        Enum "GstV4l2IOMode" Default: 0, "auto"
                           (0): auto             - GST_V4L2_IO_AUTO
                           (1): rw               - GST_V4L2_IO_RW
                           (2): mmap             - GST_V4L2_IO_MMAP
                           (3): userptr          - GST_V4L2_IO_USERPTR
                           (4): dmabuf           - GST_V4L2_IO_DMABUF
                           (5): dmabuf-import    - GST_V4L2_IO_DMABUF_IMPORT
  extra-controls      : Extra v4l2 controls (CIDs) for the device
                        flags: readable, writable
                        Boxed pointer of type "GstStructure"
  pixel-aspect-ratio  : Overwrite the pixel aspect ratio of the device
                        flags: readable, writable
                        String. Default: null
  force-aspect-ratio  : When enabled, the pixel aspect ratio will be enforced
                        flags: readable, writable
                        Boolean. Default: true

Element Signals:
  "prepare-format" :  void user_function (GstElement* object,
                                          gint arg0,
                                          GstCaps* arg1,
                                          gpointer user_data);

Thanks a lot! I cannot see where are the information about support (not support) 10bit or 8bit. I can change driver to RAW12, it will not works with v4l2src too?

So you have a 8 bits mode rggb, this is the one you’ll have to use with v4l2src.

These four formats are all 8 bits. For example, 10 bits would be named RG10, BG10…

You may also check for available resolutions and framerates with:

v4l2-ctl --list-formats-ext

I assume your camera can provide 1280x960 @ 45fps in 8 bits format RGGB.
I’d suggest to try this:

v4l2-ctl -d /dev/video0 --set-fmt-video=width=1280,height=960,pixelformat=RGGB  --set-ctrl bypass_mode=0
gst-launch-1.0 v4l2src ! 'video/x-bayer, format=rggb, width=1280, height=960, framerate=45/1' ! bayer2rgb ! videoconvert ! xvimagesink

.

Note: plugin bayer2rgb is part of package gstreamer1.0-plugins-bad. You may install it with apt.

v4l2-ctl -d /dev/video0 --set-fmt-video=width=1280,height=960,pixelformat=RGGB  --set-ctrl bypass_mode=0
gst-launch-1.0 v4l2src ! 'video/x-bayer, format=rggb, width=1280, height=960, framerate=45/1' ! bayer2rgb ! videoconvert ! xvimagesink

This command make the output:

nvidia@tegra-ubuntu:~$ gst-launch-1.0 v4l2src ! 'video/x-bayer, format=rggb, width=1280, height=960, framerate=45/1' ! bayer2rgb ! videoconvert ! xvimagesink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
WARNING: from element /GstPipeline:pipeline0/GstXvImageSink:xvimagesink0: A lot of buffers are being dropped.
Additional debug info:
gstbasesink.c(2854): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstXvImageSink:xvimagesink0:
There may be a timestamping problem, or this computer is too slow.
WARNING: from element /GstPipeline:pipeline0/GstXvImageSink:xvimagesink0: A lot of buffers are being dropped.
Additional debug info:
gstbasesink.c(2854): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstXvImageSink:xvimagesink0:
There may be a timestamping problem, or this computer is too slow.
These four formats are all 8 bits. For example, 10 bits would be named RG10, BG10...

Thank you, my camera uses ds90ub964 + da90ub913 for transmit data. This ics works with raw10 or raw12 only. May be i made some stupid things in driver code, i dont understand what raw8 and ar24 doing in list-formats.

struct camera_common_data *common_data;
...
common_data = devm_kzalloc(&client->dev,sizeof(struct camera_common_data), GFP_KERNEL);
...
common_data->colorfmt = camera_common_find_datafmt(MEDIA_BUS_FMT_SRGGB10_1X10);
static const struct camera_common_colorfmt camera_common_color_fmts[] = {
	{
		MEDIA_BUS_FMT_SRGGB12_1X12,
		V4L2_COLORSPACE_SRGB,
		V4L2_PIX_FMT_SRGGB12,
	},
	{
		MEDIA_BUS_FMT_SRGGB10_1X10,
		V4L2_COLORSPACE_RAW,
		V4L2_PIX_FMT_SRGGB10,
	},
...

May be it is possible to recompile v4l2src from source with support rg10 for example? Or i can only use another plugin?

For using the onboard camera module which is a OV5693 sensor providing 10 bits, the standard way with gstreamer is to use plugin nvcamerasrc:

gst-launch-1.0 nvcamerasrc ! nvvidconv ! xvimagesink

I’ve also read about a new plugin nvarguscamera, but I haven’t used it, so I cannot tell more.

Thank, Honey_Patouceul

With nvcamerasrc my driver dont works, i try to find alternatives for nvcamerasrc.

https://devtalk.nvidia.com/default/topic/1036285/nvcamerasrc-error-freezes-linux-kernel/