Fail to open TX2 onboard camera with OpenCV C++ via Nsight - GStreamer issue

Autonomous Machines Jetson & Embedded Systems Jetson TX2
1

Hello everyone,

I am trying to opening Jetson TX2 onboard camera from Nsight. I have installed and build OpenCV with Jetpack 4.2 on Jetson TX2. My code is written in C++.

#include <iostream>
#include <opencv2/core.hpp>
#include <opencv2/videoio.hpp>
#include <opencv2/highgui.hpp>

using namespace cv;

int main(void)
{
	Mat img;
	cv::VideoCapture cap;

	std::cout<<cv::getBuildInformation()<<std::endl;
	const char* gst = "nvarguscamerasrc ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)NV12, framerate=(fraction)30/1  ! nvvidconv  video/x-raw(memory:NVMM),format=(string)I420  ! appsink";

	cap.open(gst);

	if(!cap.isOpened())
	{
		std::cout<<"Camera is not opened"<<std::endl;
	}
    unsigned int width = cap.get(CV_CAP_PROP_FRAME_WIDTH);
    unsigned int height = cap.get(CV_CAP_PROP_FRAME_HEIGHT);
    unsigned int pixels = width*height;
    std::cout <<"Frame size : "<<width<<" x "<<height<<", "<<pixels<<" Pixels "<<std::endl;

    cv::namedWindow("MyCameraPreview", CV_WINDOW_AUTOSIZE);
    cv::Mat frame_in(width, height, CV_8UC3);

    while(1)
    {
    	if (!cap.read(frame_in))
    	{
    		std::cout<<"Capture read error"<<std::endl;
    		break;
    	}
    	else
    	{
    		std::cout<<"open window"<<std::endl;
    		cv::imshow("MyCameraPreview",frame_in);
    		cv::waitKey(1000/120); // let imshow draw and wait for next frame 8 ms for 120 fps
    	}
    }

    cap.release();

return 0;
}

I am able to build my code successfully. However, when I run my code remotely on Jetson TX2 from Nsight, I receive following error:

(DetectNet_v1:8932): GStreamer-CRITICAL **: 16:25:02.183: gst_element_make_from_uri: assertion 'gst_uri_is_valid (uri)' failed
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 2592 x 1944 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 16.000000; Exposure Range min 34000, max 550385000;

GST_ARGUS: 2592 x 1458 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 16.000000; Exposure Range min 34000, max 550385000;

GST_ARGUS: 1280 x 720 FR = 120.000005 fps Duration = 8333333 ; Analog Gain range min 1.000000, max 16.000000; Exposure Range min 22000, max 358733000;

GST_ARGUS: Running with following settings:
   Camera index = 0
   Camera mode  = 2
   Output Stream W = 1280 H = 720
   seconds to Run    = 0
   Frame Rate = 120.000005
GST_ARGUS: PowerService: requested_clock_Hz=12096000
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
GST_ARGUS: Cleaning up
GST_ARGUS:
PowerServiceHwVic::cleanupResources
CONSUMER: Done Success
GST_ARGUS: Done Success
OpenCV Error: Unspecified error (GStreamer: unable to start pipeline
) in cvCaptureFromCAM_GStreamer, file /home/nvidia/build_opencv/opencv/modules/videoio/src/cap_gstreamer.cpp, line 887
VIDEOIO(cvCreateCapture_GStreamer (CV_CAP_GSTREAMER_FILE, filename)): raised OpenCV exception:

/home/nvidia/build_opencv/opencv/modules/videoio/src/cap_gstreamer.cpp:887: error: (-2) GStreamer: unable to start pipeline
 in function cvCaptureFromCAM_GStreamer

Camera is not opened
Frame size : 0 x 0, 0 Pixels

(DetectNet_v1:8932): Gtk-WARNING **: 16:25:02.800: cannot open display:
logout

The problem is about GStreamer pipeline. However, when I checked opencv build settings, GStreamer is enabled. Here is the output:

General configuration for OpenCV 3.3.1 =====================================
  Version control:               3.3.1-2-g31ccdfe11

  Platform:
    Timestamp:                   2019-02-06T09:45:58Z
    Host:                        Linux 4.9.140-tegra aarch64
    CMake:                       2.8.12.2
    CMake generator:             Unix Makefiles
    CMake build tool:            /usr/bin/make
    Configuration:               Release

  CPU/HW features:
    Baseline:                    NEON FP16
      required:                  NEON
      disabled:                  VFPV3

  C/C++:
    Built as dynamic libs?:      YES
    C++11:                       YES
    C++ Compiler:                /usr/bin/c++  (ver 7.3.0)
    C++ flags (Release):         -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Winit-self -Wno-narrowing -Wno-delete-non-virtual-dtor -Wno-comment -Wno-implicit-fallthrough -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections    -fvisibility=hidden -fvisibility-inlines-hidden -O3 -DNDEBUG  -DNDEBUG
    C++ flags (Debug):           -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wundef -Winit-self -Wpointer-arith -Wshadow -Wsign-promo -Wuninitialized -Winit-self -Wno-narrowing -Wno-delete-non-virtual-dtor -Wno-comment -Wno-implicit-fallthrough -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections    -fvisibility=hidden -fvisibility-inlines-hidden -g  -O0 -DDEBUG -D_DEBUG
    C Compiler:                  /usr/bin/cc
    C flags (Release):           -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Winit-self -Wno-narrowing -Wno-comment -Wno-implicit-fallthrough -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections    -fvisibility=hidden -O3 -DNDEBUG  -DNDEBUG
    C flags (Debug):             -fsigned-char -W -Wall -Werror=return-type -Werror=non-virtual-dtor -Werror=address -Werror=sequence-point -Wformat -Werror=format-security -Wmissing-declarations -Wmissing-prototypes -Wstrict-prototypes -Wundef -Winit-self -Wpointer-arith -Wshadow -Wuninitialized -Winit-self -Wno-narrowing -Wno-comment -Wno-implicit-fallthrough -fdiagnostics-show-option -pthread -fomit-frame-pointer -ffunction-sections    -fvisibility=hidden -g  -O0 -DDEBUG -D_DEBUG
    Linker flags (Release):     
    Linker flags (Debug):       
    ccache:                      NO
    Precompiled headers:         NO
    Extra dependencies:          dl m pthread rt /usr/lib/aarch64-linux-gnu/libtbb.so
    3rdparty dependencies:

  OpenCV modules:
    To be built:                 core flann imgproc ml objdetect photo video dnn imgcodecs shape videoio highgui superres ts features2d calib3d stitching videostab python2 python3
    Disabled:                    js world
    Disabled by dependency:      -
    Unavailable:                 cudaarithm cudabgsegm cudacodec cudafeatures2d cudafilters cudaimgproc cudalegacy cudaobjdetect cudaoptflow cudastereo cudawarping cudev java viz

  GUI:
    QT:                          NO
    GTK+ 2.x:                    YES (ver 2.24.32)
    GThread :                    YES (ver 2.56.3)
    GtkGlExt:                    NO
    OpenGL support:              NO
    VTK support:                 NO

  Media I/O:
    ZLib:                        /usr/lib/aarch64-linux-gnu/libz.so (ver 1.2.11)
    JPEG:                        /usr/lib/aarch64-linux-gnu/libjpeg.so (ver )
    WEBP:                        build (ver encoder: 0x020e)
    PNG:                         /usr/lib/aarch64-linux-gnu/libpng.so (ver 1.6.34)
    TIFF:                        /usr/lib/aarch64-linux-gnu/libtiff.so (ver 42 - 4.0.9)
    JPEG 2000:                   build (ver 1.900.1)
    OpenEXR:                     NO
    GDAL:                        NO
    GDCM:                        NO

  Video I/O:
    DC1394 1.x:                  NO
    DC1394 2.x:                  NO
    FFMPEG:                      YES
      avcodec:                   YES (ver 57.107.100)
      avformat:                  YES (ver 57.83.100)
      avutil:                    YES (ver 55.78.100)
      swscale:                   YES (ver 4.8.100)
      avresample:                NO
    GStreamer:                  
      base:                      YES (ver 1.14.1)
      video:                     YES (ver 1.14.1)
      app:                       YES (ver 1.14.1)
      riff:                      YES (ver 1.14.1)
      pbutils:                   YES (ver 1.14.1)
    OpenNI:                      NO
    OpenNI PrimeSensor Modules:  NO
    OpenNI2:                     NO
    PvAPI:                       NO
    GigEVisionSDK:               NO
    Aravis SDK:                  NO
    UniCap:                      NO
    UniCap ucil:                 NO
    V4L/V4L2:                    NO/YES
    XIMEA:                       NO
    Xine:                        NO
    Intel Media SDK:             NO
    gPhoto2:                     NO

  Parallel framework:            TBB (ver 2017.0 interface 9107)

  Trace:                         YES ()

  Other third-party libraries:
    Use Intel IPP:               NO
    Use Intel IPP IW:            NO
    Use VA:                      NO
    Use Intel VA-API/OpenCL:     NO
    Use Lapack:                  NO
    Use Eigen:                   YES (ver 3.3.4)
    Use Cuda:                    NO
    Use OpenCL:                  NO
    Use OpenVX:                  NO
    Use custom HAL:              YES (carotene (ver 0.0.1))

  Python 2:
    Interpreter:                 /usr/bin/python2.7 (ver 2.7.15)
    Libraries:                   /usr/lib/aarch64-linux-gnu/libpython2.7.so (ver 2.7.15rc1)
    numpy:                       /usr/lib/python2.7/dist-packages/numpy/core/include (ver 1.13.3)
    packages path:               lib/python2.7/dist-packages

  Python 3:
    Interpreter:                 /usr/bin/python3 (ver 3.6.7)
    Libraries:                   /usr/lib/aarch64-linux-gnu/libpython3.6m.so (ver 3.6.7)
    numpy:                       /usr/lib/python3/dist-packages/numpy/core/include (ver 1.13.3)
    packages path:               lib/python3.6/dist-packages

  Python (for build):            /usr/bin/python2.7

  Java:
    ant:                         NO
    JNI:                         NO
    Java wrappers:               NO
    Java tests:                  NO

  Matlab:                        Matlab not found or implicitly disabled

  Documentation:
    Doxygen:                     NO

  Tests and samples:
    Tests:                       YES
    Performance tests:           YES
    C/C++ Examples:              YES

  Install path:                  /usr

  cvconfig.h is in:              /home/nvidia/build_opencv/build

Do you have any idea, why I am not able to open on-board camera?

Thank you very much

2

Hi,

Could you open the camera on the device directly?
The error looks is not relevant to Nsight.

Thanks.

3

Hi,
Please try

const char* gst = "nvarguscamerasrc ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)NV12, framerate=(fraction)30/1  ! nvvidconv ! video/x-raw,format=(string)I420  ! appsink";

We have verified feeding I420 to appsink in
https://devtalk.nvidia.com/default/topic/1024245/jetson-tx2/opencv-3-3-and-integrated-camera-problems-/post/5210735/#5210735

4

Hello DaneLLL,

Thank you for your answer. I was already trying with I420. The problem is that when I run it, it doesn’t give error; but also nothing happens. Program seems stall.

I have also tried the code in the link provided:
I observe same result as before.

#include <stdio.h>
#include <opencv2/opencv.hpp>

using namespace cv;
using namespace std;

int main(int argc, char** argv)
{
  VideoCapture cap("nvcamerasrc ! video/x-raw(memory:NVMM), width=1280, height=720,format=NV12, framerate=30/1 ! nvvidconv ! video/x-raw,format=I420 ! appsink");

  if (!cap.isOpened())
    {
      cout << "Failed to open camera." << endl;
      return -1;
    }

  for(;;)
    {
      Mat frame;
      cap >> frame;
      Mat bgr;
      cvtColor(frame, bgr, CV_YUV2BGR_I420);
      imshow("original", bgr);
      waitKey(1);
    }

  cap.release();
}

Any idea why program becomes unresponsive?

Thanks

5

Hi,
Suggest you re-install OpenCV by running the script:
https://github.com/AastaNV/JEP/blob/master/script/install_opencv3.4.0_TX2.sh

And run

VideoCapture cap("nvarguscamerasrc ! video/x-raw(memory:NVMM), width=1280, height=720,format=NV12, framerate=30/1 ! nvvidconv ! video/x-raw,format=I420 ! appsink");

It is what we have verified on r32.1/TX2 with camera board ov5693.

1 Like
6

Steps for 32.1/TX2 are in
[url]https://devtalk.nvidia.com/default/topic/1057460/jetson-tx2/custom-built-opencv-3-4-0-following-nvidia-advice-not-working/post/5363355/#5363355[/url]

7

Hello, I also cant open the built in camera. I’m using cpp, building The VideoCapture object with the arguments you’re telling us to pass.

It did always open with this setting, 1 month without any problem, but it suddenly stopped and didn’t capture images anymore. Opencv is properly installed, and I also tried to open the camera with Python which was also a fail.

It seems like the camera was improperly closed and then I couldn’t use it anymore.

The thing is, in short, that whenever I try to open the camera it returns:

GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected…
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 2592 x 1944 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 16.000000; Exposure Range min 34000, max 550385000;

GST_ARGUS: 2592 x 1458 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 16.000000; Exposure Range min 34000, max 550385000;

GST_ARGUS: 1280 x 720 FR = 120.000005 fps Duration = 8333333 ; Analog Gain range min 1.000000, max 16.000000; Exposure Range min 22000, max 358733000;

GST_ARGUS: Running with following settings:
Camera index = 0
Camera mode = 2
Output Stream W = 1280 H = 720
seconds to Run = 0
Frame Rate = 120.000005
GST_ARGUS: PowerService: requested_clock_Hz=12096000
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.

So, what did just happen?

1 Like