Hi,
I set a sensor to provide RAW8 and I’m able to get the frame properly using v4l2src. Since I need to perform the debayering using the ISP, I want to use nvcamerasrc for capturing. Checking the device-tree settings for another sensor that I found on the kernel that provided RAW12, I concluded that in order to correctly set the mode on nvcamerasrc I needed to set the following values on the particular mode on the sensor that has RAW8:
mode0 {
+ mode_type = "bayer";
+ pixel_phase = "rggb";
+ dynamic_pixel_bit_depth = "8";
+ csi_pixel_bit_depth = "8";
+ pixel_t = "bayer_rggb8";
+
...
}
When I launch a gstreamer pipeline using that mode, I’m able to see that nvcamerasrc recognizes properly that the sensor is going to provide RAW8
1640 x 1232 FR=30.000000 CF=0xf09208a10 SensorModeType=4 CSIPixelBitDepth=8 DynPixelBitDepth=8
However, I’m only seeing a black frame at the output from nvcamerasrc. In the past as stated before, I have been able to capture and debayer RAW12, and it seems like the setting above are the ones that configures the mode properly. I got the following questions:
1- Does Nvcamerasrc supports debayering RAW8?
2- Am I missing some extra setting for enabling debayering of RAW8 in Nvcamerasrc ?
Kind regards,
JJ
hello josejich,
your modification to change the device tree bit depth may not the correct method to configure as a RAW8 output.
since sensor output format was determine by the sensor side; device tree make the correct settings to correspond that.
may i know why you need RAW8 format?
thanks
Hi JerryChang,
It seems like nvcamerasrc it was using more resources that I would like to when the sensor provided RAW10 and nvcamerasrc was debayering to I420, so I would like to test the performance when the camera provides RAW8 and nvcamerasrc performs the debayering to I420.
I checked that the sensor was providing RAW8 properly using the v4l2src path, so the sensor settings seems to be correct. I’m wondering if the settings I set on the device-tree are the correct to get nvcamerasrc to perform the RAW8 to I420 conversion.
Thanks again!,
JJ
hello DavidSoto,
our reference camera driver is RAW10 output but we’re able to get 30-fps result.
could you help analysis with v4l2-ctl
v4l2-ctl --list-formats-ext
and also argus_camera
./argus_camera -i --sensormode=0 --outputsize=2592x1944 --framerate=30 --kpi
Hi JerryChang,
Thanks for your response, sorry, maybe I confused you with my previous answer. When using RAW10 or RAW12 and only nvcamerasrc with the 5 or 6 cameras it is possible to get 30fps. Something like:
nvcamerasrc ! fakesink in 5 or 6 cameras
So nvcamerasrc is fine. But when adding the algorithm of the customer, it is not possible to reach 30fps, I know that this indicates that maybe the customer algorithm is not efficient enough to keep the framerate up, something like:
nvcamerasrc ! customAlg ! fakesink in 5 or 6 cameras
However, as part of the optimization process the customer wants to reduce the amount of data per pixel, and instead of capturing RAW10 or RAW12 they want to capture RAW8. For this, we added the support in the camera and verified that something like this is working at 30fps:
v4l2src ! customAlg ! fakesink in 5 or 6 cameras
where now customAlg is also doing the debayering in CUDA and some other functions, all in RAW8. But I keep thinking that if we are able to get nvcamerasrc working with RAW8, it will be even more efficient because the ISP is pretty powerful.
Is there any limitation on nvcamerasrc causing to not work when the input sensor gives RAW8?
Thanks,
-David
Hi,
I also set a sensor to provide Bayer RAW8 and I’m able to get the frame properly using v4l2src. But when I have tried to use nvcamerasrc for capturing, I am getting a black frame.
I am using the same device tree ISP configuration described previously.
In the past I have configured Bayer RAW12 for another sensor and for this case, the configuration applied was:
mode0 {
. . .
dynamic_pixel_bit_depth = "12";
csi_pixel_bit_depth = "12";
pixel_t = "bayer_grbg12";
. . .
This change was enough to configure the ISP properly to process RAW12.
Do you know if I need to apply some extra change to be able to process Bayer RAW8 format?
Regards,
-Manuel
https://developer.ridgerun.com/wiki/index.php?title=RidgeRun_Professional_Services
hello manuel.leiva,
thanks for bringing this issue up.
could you run the camera daemon manually and share the background message to us.
execute below command at the terminal before launching camera for capture,
sudo /usr/sbin/nvcamera-daemon
also, please share both capture commands with v4l2src and nvcamerasrc
thanks
Hello JerryChang,
Thanks for your response, These are the messages displayed.
nvidia@tegra-ubuntu:~$ sudo /usr/sbin/nvcamera-daemon
NvPclHwGetModuleList: No module data found
NvPclHwGetModuleList: No module data found
NvPclHwGetModuleList: No module data found
PCLHW_DTParser
LoadOverridesFile: looking for override file [/Calib/camera_override.isp] 1/16LoadOverridesFile: looking for override file [/data/nvcam/settings/camera_overrides.isp] 2/16LoadOverridesFile: looking for override file [/opt/nvidia/nvcam/settings/camera_overrides.isp] 3/16LoadOverridesFile: looking for override file [/var/nvidia/nvcam/settings/camera_overrides.isp] 4/16---- imager: Found override file [/var/nvidia/nvcam/settings/camera_overrides.isp]. ----
NvCameraIspConfigFileLoad: Config file "common.cfg" Line 594: Error: Invalid isp config attribute: "//Thisistheextradelayintroducedwhileprogrammingthegroupholdsettingapartfromdefault//totaldelayof3framesi.e.2framesingroupholdsetttingstoeffectand1frametocapture.//Settingdefaultvalueaszeroasitisdisabledbydefault.ae.MaxGroupHoldDelayFrameCount=0"
NvCameraIspConfigFileLoad: Config file "common.cfg" Line 3177: Error: Invalid isp config attribute element: "fd.Available=TRUE"
NvCameraIspConfigFileLoad: Config file "common.cfg" Line 3178: Error: Invalid isp config attribute element: "fd.ForceEnable=FALSE"
NvCameraIspConfigFileLoad: Config file "camera_overrides.isp" Line 447: Error: Invalid isp config attribute element: "mwbCCT.sunlight=6100"
NvCameraIspConfigFileLoad: Config file "camera_overrides.isp" Line 450: Error: Invalid isp config attribute element: "mwbCCT.tungsten=2950"
NvCameraIspConfigFileLoad: Config file "camera_overrides.isp" Line 453: Error: Invalid isp config attribute element: "mwbCCT.flash=7300"
NvCameraIspConfigFileLoad: Config file "camera_overrides.isp" Line 454: Error: Invalid isp config attribute element: "mwbCCT.horizon=3000"
LSC: LSC surface is not based on full res!
LoadOverridesFile: looking for override file [/Calib/camera_override.isp] 1/16LoadOverridesFile: looking for override file [/data/nvcam/settings/camera_overrides.isp] 2/16LoadOverridesFile: looking for override file [/opt/nvidia/nvcam/settings/camera_overrides.isp] 3/16LoadOverridesFile: looking for override file [/var/nvidia/nvcam/settings/camera_overrides.isp] 4/16---- imager: Found override file [/var/nvidia/nvcam/settings/camera_overrides.isp]. ----
NvCameraIspConfigFileLoad: Config file "common.cfg" Line 594: Error: Invalid isp config attribute: "//Thisistheextradelayintroducedwhileprogrammingthegroupholdsettingapartfromdefault//totaldelayof3framesi.e.2framesingroupholdsetttingstoeffectand1frametocapture.//Settingdefaultvalueaszeroasitisdisabledbydefault.ae.MaxGroupHoldDelayFrameCount=0"
NvCameraIspConfigFileLoad: Config file "common.cfg" Line 3177: Error: Invalid isp config attribute element: "fd.Available=TRUE"
NvCameraIspConfigFileLoad: Config file "common.cfg" Line 3178: Error: Invalid isp config attribute element: "fd.ForceEnable=FALSE"
NvCameraIspConfigFileLoad: Config file "camera_overrides.isp" Line 447: Error: Invalid isp config attribute element: "mwbCCT.sunlight=6100"
NvCameraIspConfigFileLoad: Config file "camera_overrides.isp" Line 450: Error: Invalid isp config attribute element: "mwbCCT.tungsten=2950"
NvCameraIspConfigFileLoad: Config file "camera_overrides.isp" Line 453: Error: Invalid isp config attribute element: "mwbCCT.flash=7300"
NvCameraIspConfigFileLoad: Config file "camera_overrides.isp" Line 454: Error: Invalid isp config attribute element: "mwbCCT.horizon=3000"
LSC: LSC surface is not based on full res!
LoadOverridesFile: looking for override file [/Calib/camera_override.isp] 1/16LoadOverridesFile: looking for override file [/data/nvcam/settings/camera_overrides.isp] 2/16LoadOverridesFile: looking for override file [/opt/nvidia/nvcam/settings/camera_overrides.isp] 3/16LoadOverridesFile: looking for override file [/var/nvidia/nvcam/settings/camera_overrides.isp] 4/16---- imager: Found override file [/var/nvidia/nvcam/settings/camera_overrides.isp]. ----
NvCameraIspConfigFileLoad: Config file "common.cfg" Line 594: Error: Invalid isp config attribute: "//Thisistheextradelayintroducedwhileprogrammingthegroupholdsettingapartfromdefault//totaldelayof3framesi.e.2framesingroupholdsetttingstoeffectand1frametocapture.//Settingdefaultvalueaszeroasitisdisabledbydefault.ae.MaxGroupHoldDelayFrameCount=0"
NvCameraIspConfigFileLoad: Config file "common.cfg" Line 3177: Error: Invalid isp config attribute element: "fd.Available=TRUE"
NvCameraIspConfigFileLoad: Config file "common.cfg" Line 3178: Error: Invalid isp config attribute element: "fd.ForceEnable=FALSE"
NvCameraIspConfigFileLoad: Config file "camera_overrides.isp" Line 447: Error: Invalid isp config attribute element: "mwbCCT.sunlight=6100"
NvCameraIspConfigFileLoad: Config file "camera_overrides.isp" Line 450: Error: Invalid isp config attribute element: "mwbCCT.tungsten=2950"
NvCameraIspConfigFileLoad: Config file "camera_overrides.isp" Line 453: Error: Invalid isp config attribute element: "mwbCCT.flash=7300"
NvCameraIspConfigFileLoad: Config file "camera_overrides.isp" Line 454: Error: Invalid isp config attribute element: "mwbCCT.horizon=3000"
LSC: LSC surface is not based on full res!
NvCameraIspConfigFileLoad: Config file "common.cfg" Line 594: Error: Invalid isp config attribute: "//Thisistheextradelayintroducedwhileprogrammingthegroupholdsettingapartfromdefault//totaldelayof3framesi.e.2framesingroupholdsetttingstoeffectand1frametocapture.//Settingdefaultvalueaszeroasitisdisabledbydefault.ae.MaxGroupHoldDelayFrameCount=0"
NvCameraIspConfigFileLoad: Config file "common.cfg" Line 3177: Error: Invalid isp config attribute element: "fd.Available=TRUE"
NvCameraIspConfigFileLoad: Config file "common.cfg" Line 3178: Error: Invalid isp config attribute element: "fd.ForceEnable=FALSE"
NvCameraIspConfigFileLoad: Config file "camera_overrides.isp" Line 447: Error: Invalid isp config attribute element: "mwbCCT.sunlight=6100"
NvCameraIspConfigFileLoad: Config file "camera_overrides.isp" Line 450: Error: Invalid isp config attribute element: "mwbCCT.tungsten=2950"
NvCameraIspConfigFileLoad: Config file "camera_overrides.isp" Line 453: Error: Invalid isp config attribute element: "mwbCCT.flash=7300"
NvCameraIspConfigFileLoad: Config file "camera_overrides.isp" Line 454: Error: Invalid isp config attribute element: "mwbCCT.horizon=3000"
LSC: LSC surface is not based on full res!
PowerServiceCore:handleRequests: timePassed = 1594
This is the command used to capture from nvcamerasrc
DISPLAY=:0 gst-launch-1.0 nvcamerasrc sensor-id=0 auto-exposure=off fpsRange="30 30" ! 'video/x-raw(memory:NVMM),width=(int)4656,height=(int)3496,format=(string)I420,framerate=(fraction)21/1' ! nvvidconv ! 'video/x-raw, width=640, height=480' ! xvimagesink
This is the command used to capture with v4l2src
v4l2-ctl -d /dev/video0 --set-fmt-video=width=4656,height=3496, --set-ctrl bypass_mode=0 --stream-mmap --stream-count=1 --stream-to image_8bit.raw
This is the information provided by v4l2-ctl related to the video format:
nvidia@tegra-ubuntu:~$ v4l2-ctl --all | grep "Format Video" -A 10
Format Video Capture:
Width/Height : 4656/3496
Pixel Format : 'RGGB'
Field : None
Bytes per Line : 4864
Size Image : 17004544
Colorspace : sRGB
Transfer Function : Default
YCbCr Encoding : Default
Quantization : Default
Flags :
Thanks.
hello manuel.leiva,
thanks for sharing the information.
since you had dump the raw8 files with v4l2-ctl.
could you also analysis the raw files with 3rdparty tools, such as 7yuv.
let’s confirmed the raw file is correct by converting to an image or reading its hex values.
thanks
JerryChang,
I have tried to convert the image captured with 7yuv, but at least with the evaluation version, bayer 8bit/pixel is not supported (only was supported 16bit/pixel) , Then I have tried with another tool called Bayer2rgb (see RAW Image Compression | Color Space Conversion | RidgeRun)
Using this command:
./bayer2rgb --input=image_8bit.raw --output=pic.tiff --width=4864 --height=3496 --bpp=8 --first=RGGB --method=BILINEAR --tiff
I got this image:
https://drive.google.com/file/d/1KvkWILva6WQJB_cv4IptTqbbjNIoloX0/view?usp=sharing
The image has a black area on the right, since v4l2-core is assigning ‘Bytes per Line: 4864’ but the image width is 4656.
Thanks.
hello manuel.leiva/ DavidSoto,
thanks for the confirmation,
we’re going to investigate this internally.
beside gstreamer pipeline, did you also got black preview frames with libargus?
moreover,
since there are different drivers between TX1 and TX2.
do you have any chances to confirm the status for TX2.
thanks
Hi JerryChang,
I have tried to capture using libargus (TX1) but I am getting the same black frame.
Also I was checking the “Sensor Driver Programming Guide” and I have found that there is not a definition for bayer 8bpp.
pixel_t: Specifies the readout pixel pattern of the sensor.
The following examples shows the values for Bayer sensors with the “bggr” pixel pattern:
• 10 bit: bayer_bggr
• 12 bit: bayer_bggr12
• 14 bit: bayer_bggr14
Based on this documentation I think that there is not support for Bayer8.
Thanks.
hi all,
we would like to investigate this internally.
could you share the raw files dumped by RAW8 sensors.
for example, please dump raw files as below command.
v4l2-ctl -d /dev/camera/video0 --set-fmt-video=width=1920,height=1080,maxframerate=30,pixelformat=RG8 --stream-mmap --stream-count=300 --stream-to=/test.raw8
thanks!
Hi JerryChang,
You can download example images from these links:
Image: one frame: https://drive.google.com/file/d/1yXwrHO14njxm3nEzQ3Vq6MZ99-1kLn9J/view?usp=sharing
Size: 17M
Command:
v4l2-ctl -d /dev/video0 --set-fmt-video=width=4656,height=3496, --set-ctrl bypass_mode=0 --stream-mmap --stream-count=1 --stream-to image_8bit.raw
Image 10 frames: image_8bit.raw - Google Drive
size: 163M
Command:
v4l2-ctl -d /dev/video0 --set-fmt-video=width=4656,height=3496,pixelformat=RG8 --stream-mmap --stream-count=10 --stream-to=image_8bit_10_frames.raw
Thanks.
I also tried switching my sensor to RAW8 and just get black from the output. Capturing a frame with Yavta shows data, however. Interested to hear how this develops.
Any updates on the Bayer8 front?
I was also wondering if we need to change settings in tegra-camera-platform. Specifically the min_bits_per_pixel and vi_peak_byte_per_pixel, and isp_peak_byte_per_pixel.
Here’s the dmesg output when I try to use bayer_rggb8:
[ 142.819129] daxc02 1-0010: daxc02_power_on
[ 143.065486] extract_pixel_format: Need to extend formatbayer_rggb8
[ 143.071779] daxc02 1-0010: Unsupported pixel format
[ 143.076743] daxc02 1-0010: Failed to read image properties
[ 143.084002] daxc02 1-0010: daxc02_power_off
and the corresponding function in sensor_common.c:
static int extract_pixel_format(
const char *pixel_t, u32 *format)
{
size_t size = strnlen(pixel_t, OF_MAX_STR_LEN);
if (strncmp(pixel_t, "bayer_bggr10", size) == 0)
*format = V4L2_PIX_FMT_SBGGR10;
else if (strncmp(pixel_t, "bayer_rggb10", size) == 0)
*format = V4L2_PIX_FMT_SRGGB10;
else if (strncmp(pixel_t, "bayer_bggr12", size) == 0)
*format = V4L2_PIX_FMT_SBGGR12;
else if (strncmp(pixel_t, "bayer_rggb12", size) == 0)
*format = V4L2_PIX_FMT_SRGGB12;
else if (strncmp(pixel_t, "bayer_wdr_pwl_rggb12", size) == 0)
*format = V4L2_PIX_FMT_SRGGB12;
else if (strncmp(pixel_t, "bayer_xbggr10p", size) == 0)
*format = V4L2_PIX_FMT_XBGGR10P;
else if (strncmp(pixel_t, "bayer_xrggb10p", size) == 0)
*format = V4L2_PIX_FMT_XRGGB10P;
else {
pr_err("%s: Need to extend format%s\n", __func__, pixel_t);
return -EINVAL;
}
return 0;
}
Doesn’t look like 8 bit is supported (or at least not broken out).