[Python] How to use TX1 Camera with OpenCV

Hello,

I am trying to use TX1 Camera to capture video in openCV.
But even when I am trying a basic example like one below I do not get any output - not even an error message.
Could you let me know if I am missing anything?

Thanks
(how do I put the codes separately?)

import numpy as np
import cv2

cap = cv2.VideoCapture(0)

while(True):
# Capture frame-by-frame
ret, frame = cap.read()

# Our operations on the frame come here
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)

# Display the resulting frame
cv2.imshow('frame',gray)
if cv2.waitKey(1) & 0xFF == ord('q'):
    break

When everything done, release the capture

cap.release()
cv2.destroyAllWindows()

Thanks for reporting the issue, we are currently investigating the case and we’ll let you know when we have an update.
In the meantime, you can try using the TX1 camera module through gstreamer and/or the nvgstcapture-1.0 sample.
(see [url]https://devtalk.nvidia.com/default/topic/898129/jetson-tx1/enabling-camera-on-jetson-tx1-board/post/4758217/#4758217[/url] for example pipeline)

@dusty_nv,

 I tried what you suggested, and using OpenCV C++, I compiled some code from 

[url]https://devtalk.nvidia.com/default/topic/904949/jetson-tx1/how-to-get-tx1-camera-in-opencv/?offset=5#4774306[/url]

with the corrections noted further down, and it compiles just fine, but when I try to run it from terminal, nothing happens. Any advice? Thanks!

You might need this recent OpenCV bug fix applied: [url]https://github.com/Itseez/opencv/pull/5984[/url]

Okay, thanks! I haven’t done much with Github before, so I am not sure how to apply that fix to OpenCV4tegra.

Did you figure how to apply the bug fix on opencv4tegra?

This seems to be a recurring theme:

  • opencv4tegra has important speed optimizations for jetson
  • opencv4tegra can’t be built with optimizations ‘in the field’ (is this true?)
  • opencv source has some fixes for our issues, but when we build it ourselves we get an unoptimized build.

This means that we either get the slow, non-optimized, fixed version or it means that we get the fast, buggy, behind-the-times version.

Dusty_nv, can you comment?

Thanks!

@kalvik, No, the page just looks like a bunch of information about what was changed, not on how to actually apply the fix. I am hoping they just have a .deb file or something. Not sure though. Thanks!

The above patch was referencing upstream OpenCV. The CV camera interface tries using V4L2 device. With R23.2, V4L2 driver is provided for the TX1 devkit onboard CSI camera — see the Video for Linux User Guide included in the L4T Driver Documentation: [url]http://developer.nvidia.com/embedded/dlc/l4t-documentation-23-2[/url]
In lieu of V4L2 support, you can alternatively configure OpenCV to use gstreamer instead.

@dusty_nv
Hi dusty_nv,
I am trying open the camera on the TX1 by OpenCV and I have flashed it with tegra 23.2.It compiles fine but when I run it,I got nothing.can you tell me how to configure OpenCV to use gstreamer?
Thanks a lot!

I am also having troubles using my TX1 for anything useful because of the issues mentioned here (and a few other places) when trying to use opencv4tegra with python and a USB web camera. It looks like the problems have been going on for some time with the occasional peep from an NVidia developer that “fix coming soon!”. I understand that everyone’s under pressure but really, this must be the most common configuration that people get started with and I really think NVidia should sort this out asap.
Get the next version out now please!

Any news on when next version of opencv4tegra is due out?

Ultimately, I need to run a CSI camera from OpenCV inside Python. I have a gstreamer pipe that almost works, but need to migrate to python. I would like to split the video stream into a high res stream for cuda processing and a low res stream for preview. The following command works, but slow (seems to run on the ARM cores) - thought is that videocrop and videoscale are not GPU enabled.

gst-launch -v nvcamerasrc fpsRange=“30.0 30.0” ! ‘video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1’ ! nvvidconv flip-method=2 tnr=6 ! ‘video/x-raw, width=(int)1920, height=(int)1080, format=(string)BGRx’ ! tee name=tp tp. ! queue max-size-buffers=4 ! ximagesink name=highres tp. ! queue ! videocrop left=240 right=240 ! videoscale ! ‘video/x-raw, width=(int)640, height=(int)480, format=(string)BGRx, pixel-aspect-ratio=(fraction)1/1’ ! ximagesink name=lowres

And I get the following warnings:
WARNING: from element /GstPipeline:pipeline0/GstXImageSink:lowres: A lot of buffers are being dropped.
Additional debug info:
gstbasesink.c(2854): gst_base_sink_is_too_late (): /GstPipeline:pipeline0/GstXImageSink:lowres:
There may be a timestamping problem, or this computer is too slow.

The following almost works but high res image is blank. Speed looks reasonable.

gst-launch -v nvcamerasrc fpsRange=“30.0 30.0” ! ‘video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1’ ! nvtee name=tp tp. ! nvvidconv flip-method=2 ! queue ! ‘video/x-raw, width=(int)640, height=(int)480, format=(string)BGRx, pixel-aspect-ratio=(fraction)1/1’ ! ximagesink name=lowres tp. ! nvvidconv flip-method=2 ! queue ! ximagesink name=highres

Can anyone give me an example the syntax to integrate either of the above into cv2 and python? The following fails to open the camera

mycam = cv2.VideoCapture(“gst-launch -v nvcamerasrc name=devkitCamera fpsRange="30.0 30.0" ! ‘video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1’ ! nvvidconv flip-method=2”)

Hello,
As to Jetson TX1 on-board camera in openCV, would you please refer to [url]https://devtalk.nvidia.com/default/topic/943129/jetson-tx1/highgui-error-v4l-v4l2-while-opening-camera-with-opencv4tegra-l4t-r24/post/4921383/#4921383[/url]

If there’s still any problem, please set GST_DEBUG=3 and run your program to capture some GST log.

br
ChenJian

@jachen
Hi jachen,
I have a camera pluged in TX1,how can I get the camera worked successfully with opencv3.1 (not the default camera)?could you please show me the codes then?
thank you very much!

Hello, soga_zh:
What kind of camera you are using?
For usb camera, there are a lot of samples in internet, like: Reading and Writing Video — OpenCV 3.0.0-dev documentation

br
ChenJian

@jachen
Hi jachen,
I want to install QT5 on TX1 board . Though many times did I try , it failed in the end . Do u have some instructions?
Thank you very much!

Hello, soga_zh:
So far, QT has no official support from NV.
You can search forum and get some information from other threads.

br
Chenjian

@jachen
Hi jachen,
thank u for your information!

hi anyone know how to use on-board camera with python opencv 3.1.0

thanks in advance

Hi danie,
Please follow

https://devtalk.nvidia.com/default/topic/987537/?comment=5062661