YOLO with onboard camera, pixel format error

Hi,

I am trying to run the YOLO with the onboard camera. https://github.com/pjreddie/darknet

It is working nicely with RGB camera of intel Realsense but I have problem with onboard camera. The problem is in this part of the code in C:

static CvCapture * cap;
cap = cvCaptureFromCAM(0);

The above code result in the following error:

HIGHGUI ERROR: V4L2: Pixel format of incoming image is unsupported by OpenCV
Couldn't connect to webcam.

I have tried the following witout any knowlefge, just seen on the topics and did not understand why. But it is still faulty

const char *gst = "nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)I420, framerate=(fraction)30/1 ! \
	           nvvidconv flip-method=6 ! video/x-raw, format=(string)I420 ! \
		   videoconvert ! video/x-raw, format=(string)BGR ! \
		   appsink";
cap = cvCaptureFromFile(gst);

I am getting this error

Couldn't connect to webcam.
: No such file or directory

I have opencv_version 3.2.0-dev

I think you have to build OpenCV with GStreamer.

I’m having the same error message under python. I’ve been using OpenCV4Tegra. I’m trying to figure out how to set Gstreamer in CV4Tegra

UPDATE: From the looks of other comment threads, Nvidia doesn’t provide Gstreamer with CV4Tegra. I’m installing regular old 3.2 for now using the Jetsonhacks script as a starter, with GStreamer enabled (in the script it’s disabled by default). But from the looks of this thread, https://devtalk.nvidia.com/default/topic/936870/jetson-tx1/opencv4tegra-vs-opencv3-1-w-cuda/ , the Tegra optimised CV is 10x faster.

UPDATE2: There was a longstanding bug, https://devtalk.nvidia.com/default/topic/929483/jetson-tx1/opencv-videocapture-usb-camera/?offset=14#5196536 , and I’m not clear yet if it was resolved or not. This post is turning into a notebook for me as I try to solve this problem.

UPDATE3: I’m seeing current chatter about the bug, so it’s still outstanding. It looks like I’m going to have to work on my prototype under CV3.2 until I can order in a camera that works with the Tegra optimized opencv.

Incidentally, the program I’m trying to get running on my TX2 worked on my Ubuntu 16.04 desktop with Opencv3.2.

Hi,

Default Opencv4Tegra doesn’t enable GStreamer support, but you can build one from source:
This page will guide you to do this: http://dev.t7.ai/jetson/opencv/

2. Please remember to open onboard camera with nvcamerasrc. V4l2 is for USB-Camera.

Thanks.

Thanks, is the opencv4tegra is really that much faster?

I prefer to use other usb camera for now then.

Hi,

We upstream our acceleration to OpenCV3.x.
Please use OpenCV3.x to get better acceleration.

Thanks.

You can see my repository for running YOLO on Jetson TX2 and TX1 (Jetpack 3.1)

Thanks for sharing this.
However, even after boosting my TX2 with jetson_clocks, I only get (2.5 - 2.6 fps) with yolo.weights model (mostly GPU limited as showed by tegrastats).
I have also added cuda arch 6.2 in Makefile and I am using opencv-3.3.0 compiled for TX2.
Do you have higher rates ?

Dup removed…

I also only get 2~3 fps on my TX2 with the regular yolo model/weights. And I can get 16~17 fps if I run the tiny yolo. I’ve shared my experience in this post: [url]https://jkjung-avt.github.io/yolov2/[/url]

I try to run the basic Yolo Darknet Demo with

./darknet detector demo cfg/coco.data cfg/yolo.cfg yolo.weights

but it gives me a video signal error (pixel format).

Does anybody know how to add the video signal of jetsons onboard cam?

I havea jetson tx2 with jetpack3.2 flashed.

Thank you!

Hi gustavvz,

Have you checked the post of jkjung13 in comment #10.
The user runs YOLO correctly with a USB webcam (/dev/video1).

Thanks.

Yes i know. But i want to run it with the onboard cam.
But as i was told OpenCV 3.3.1 that comes with JetPack 3.2 does not support gstream. Is that correct?

Please take a look https://github.com/AlexanderRobles21/YOLO-darknet-on-Jetson-TX2

Try to run this command:

$ ./darknet detector demo cfg/coco.data cfg/yolo.cfg yolo.weights “nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720,format=(string)I420, framerate=(fraction)30/1 ! nvvidconv flip-method=0 ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink”

Write back if this was useful

This worked for me.

Jetson TX1 with YOLO and tiny-YOLO.

If you are facing flipped video, change flip-method to 2:

$ ./darknet detector demo cfg/coco.data cfg/yolo.cfg yolo.weights "nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720,format=(string)I420, framerate=(fraction)30/1 ! nvvidconv flip-method=2 ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink"