Viewing Raw Bayer data

Hi All,

I am using R24.2. I followed below steps to capture Raw Bayer data

Step1: v4l2-ctl --set-fmt-video=width=1920,height=1080,pixelformat=RG10 --stream-mmap --stream-count=100 -d /dev/video0

Step2: yavta /dev/video0 -c1 -n1 -s1920x1080 -fSRGGB10 -Fcam.raw

(one thing I noted is even though We configure size as 1920x1080… it is taking it is as 2592x1944)

Can someone guide me on how to verify this cam.raw is properly recorded or not?

Thanks
Palani

I believe it’s the same topic with below one. Please see the detail information by below link.

https://devtalk.nvidia.com/default/topic/968475

maybe something like:

gst-launch-1.0 -v v4l2src device=/dev/video0 num-buffers=10 ! ‘video/x-bayer,format=rggb,width=1280,height=720’ ! identity silent=false ! filesink location=test.bayer

gst-launch-1.0 -v filesrc blocksize=5038848 num-buffers=4 location=test.bayer ! ‘video/x-bayer,format=rggb,width=2592,height=1944,framerate=30/1’ ! bayer2rgb ! filesink location=test.rgb

and use something like rawpixels to see the frame

Hi David,

In R24.2, Will gstreamer v4l2src work for Bayer sensor ?

I have read that in previous releases this is not possible. Please Let me know.

Thanks
Palani

In R24.2 v4l2 works correctly, however, you would still get the frame in bayer and not in YUV. We are working to look for alternatives to use the ISP, not sure if openkcam allows you to use it or not. Also we are checking if we could do the conversion in CUDA.

I am asking about it here: https://devtalk.nvidia.com/default/topic/965351/l4t-24-2-isp-configuration-options/?offset=3#4994137

Hi David,

In R24.2, I tried with above gstreamer commands (gst-launch-1.0 -v v4l2src device=/dev/video0 num-buffers=10 ! ‘video/x-bayer,format=rggb,width=1280,height=720’ ! identity silent=false ! filesink location=test.bayer) You have provided for raw bayer capture. Image is recorded without any gstreamer error. But it is an invalid file (all are zeros). In one post I read like, there are issues with Yavta also in R24.2.

Please Let me know Your suggestion.

Hi Palani,

We had tested with R24.2 and other camera sensor. We just ran the test with the camera that comes in the Jetson board and it seems that the problem is that default kernel configuration in the input is for RAW10 and v4l2src doesn’t have support for RAW10 but RAW8. The patch below was created by one of our engineers (JJ) and would add support for it. With this pipeline:

gst-launch-1.0 -v v4l2src device=/dev/video0 ! 'video/x-bayer,format=rggb,width=2592,height=1944' ! bayer2rgb ! nveglglessink

You will see that the image is captured and displayed, however, the colors look wrong, we believe it is because the bayer2rgb works with RAW8 and not with RAW10.

https://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/gst/bayer/gstbayer2rgb.c

So options are:

a) Add support to the kernel to use RAW8 and use everything on RAW8
b) Convert the RAW10 images to other format that could be displayed properly, maybe extend bayer2rgb?

We have a patch for a) however, it needs to be tested since it was used in other customer project. Hope this helps.

Index: gst_1.9.1/gst-plugins-good-1.9.1/sys/v4l2/gstv4l2object.c
===================================================================
--- gst_1.9.1.orig/gst-plugins-good-1.9.1/sys/v4l2/gstv4l2object.c	2016-10-11 16:07:17.427961922 -0600
+++ gst_1.9.1/gst-plugins-good-1.9.1/sys/v4l2/gstv4l2object.c	2016-10-11 16:08:29.099961393 -0600
@@ -160,10 +160,10 @@
   {V4L2_PIX_FMT_NV42, TRUE, GST_V4L2_RAW},
 
   /* Bayer formats - see http://www.siliconimaging.com/RGB%20Bayer.htm */
-  {V4L2_PIX_FMT_SBGGR8, TRUE, GST_V4L2_CODEC},
-  {V4L2_PIX_FMT_SGBRG8, TRUE, GST_V4L2_CODEC},
-  {V4L2_PIX_FMT_SGRBG8, TRUE, GST_V4L2_CODEC},
-  {V4L2_PIX_FMT_SRGGB8, TRUE, GST_V4L2_CODEC},
+  {V4L2_PIX_FMT_SBGGR10, TRUE, GST_V4L2_CODEC},
+  {V4L2_PIX_FMT_SGBRG10, TRUE, GST_V4L2_CODEC},
+  {V4L2_PIX_FMT_SGRBG10, TRUE, GST_V4L2_CODEC},
+  {V4L2_PIX_FMT_SRGGB10, TRUE, GST_V4L2_CODEC},
 
   /* compressed formats */
   {V4L2_PIX_FMT_MJPEG, FALSE, GST_V4L2_CODEC},
@@ -1072,10 +1072,10 @@
       rank = 0;
       break;
 
-    case V4L2_PIX_FMT_SBGGR8:
-    case V4L2_PIX_FMT_SGBRG8:
-    case V4L2_PIX_FMT_SGRBG8:
-    case V4L2_PIX_FMT_SRGGB8:
+    case V4L2_PIX_FMT_SBGGR10:
+    case V4L2_PIX_FMT_SGBRG10:
+    case V4L2_PIX_FMT_SGRBG10:
+    case V4L2_PIX_FMT_SRGGB10:
       rank = BAYER_BASE_RANK;
       break;
 
@@ -1461,15 +1461,15 @@
       break;
     case V4L2_PIX_FMT_WNVA:    /* Winnov hw compres */
       break;
-    case V4L2_PIX_FMT_SBGGR8:
-    case V4L2_PIX_FMT_SGBRG8:
-    case V4L2_PIX_FMT_SGRBG8:
-    case V4L2_PIX_FMT_SRGGB8:
+    case V4L2_PIX_FMT_SBGGR10:
+    case V4L2_PIX_FMT_SGBRG10:
+    case V4L2_PIX_FMT_SGRBG10:
+    case V4L2_PIX_FMT_SRGGB10:
       structure = gst_structure_new ("video/x-bayer", "format", G_TYPE_STRING,
-          fourcc == V4L2_PIX_FMT_SBGGR8 ? "bggr" :
-          fourcc == V4L2_PIX_FMT_SGBRG8 ? "gbrg" :
-          fourcc == V4L2_PIX_FMT_SGRBG8 ? "grbg" :
-          /* fourcc == V4L2_PIX_FMT_SRGGB8 ? */ "rggb", NULL);
+          fourcc == V4L2_PIX_FMT_SBGGR10 ? "bggr" :
+          fourcc == V4L2_PIX_FMT_SGBRG10 ? "gbrg" :
+          fourcc == V4L2_PIX_FMT_SGRBG10 ? "grbg" :
+          fourcc == V4L2_PIX_FMT_SRGGB10 ? "rggb" : "rggb" , NULL);
       break;
     case V4L2_PIX_FMT_SN9C10X:
       structure = gst_structure_new_empty ("video/x-sonix");
@@ -1769,13 +1769,13 @@
       const gchar *format = gst_structure_get_string (structure, "format");
       if (format) {
         if (!g_ascii_strcasecmp (format, "bggr"))
-          fourcc = V4L2_PIX_FMT_SBGGR8;
+          fourcc = V4L2_PIX_FMT_SBGGR10;
         else if (!g_ascii_strcasecmp (format, "gbrg"))
-          fourcc = V4L2_PIX_FMT_SGBRG8;
+          fourcc = V4L2_PIX_FMT_SGBRG10;
         else if (!g_ascii_strcasecmp (format, "grbg"))
-          fourcc = V4L2_PIX_FMT_SGRBG8;
+          fourcc = V4L2_PIX_FMT_SGRBG10;
         else if (!g_ascii_strcasecmp (format, "rggb"))
-          fourcc = V4L2_PIX_FMT_SRGGB8;
+          fourcc = V4L2_PIX_FMT_SRGGB10;
       }
     } else if (g_str_equal (mimetype, "video/x-sonix")) {
       fourcc = V4L2_PIX_FMT_SN9C10X;

Here are some instructions to rebuild gstreamer:

How do I get bayer2rgb installed?

1 Like

{requesting email notifications…}

sudo apt-get install gstreamer1.0-plugins-bad

The lists is here.

https://gstreamer.freedesktop.org/documentation/plugins.html