Opencv Face Detection Poor Performance with jetson nano

Autonomous Machines Jetson & Embedded Systems Jetson Nano
1

I recently performed opencv 4 face detection using DNN model res10_300x300_ssd_iter_140000.caffemodel and found that it managed terrible performance 1 frame/5 seconds at its best

Can you please suggest a solution to improve the frame rate or does Nvidia provides any tested face detection models like you do for object detection?

Thanks

2

Hi,

Do you use cap() to read camera frame?
OpenCV by default use ffmpeg to read/write camera.
Since FFmpeg is a CPU implementation on Jetson, the performance won’t be good.

For better performance, you can check our detection sample here:
[url]https://github.com/dusty-nv/jetson-inference/blob/master/docs/detectnet-training.md[/url]

Thanks.

3

Hi
I tried this example ./detectnet-camera facenet from jetson inference

to be honest. detection is too bad, it cant even detect with my glasses or minor face turns

Opencv DNN was extremely good at its detection even under occlusion

Please suggest me a better option to detect face, Nvidia is it not optimised for Opencv, i do remember when compiling opencv i used CUDA flags?

Thanks

4

I even see that from this link

opencv does work with non Intel gpu with opencl support and not sure why opencv only uses cpu?

5

The Facenet provided by jetson-inference does not outperform OpenCV_dnn. Is there no better way? Is it necessary to use DIGIST to learn Facenet in order to directly learn Facenet?

6

I can confirm that its not slowing down because of reading image from camera but when opencv dnn detect is used, see below code

import sys
import cv2
import gi
gi.require_version('Gst', '1.0')
from gi.repository import Gst

from imutils.video import VideoStream
import numpy as np
import argparse
import imutils
import time
import cv2

# construct the argument parse and parse the arguments
ap = argparse.ArgumentParser()
ap.add_argument("-p", "--prototxt", required=False, default = "opencv-face-recognition/face_detection_model/deploy.prototxt",
	help="path to Caffe 'deploy' prototxt file")
ap.add_argument("-m", "--model", required=False, default = "opencv-face-recognition/face_detection_model/res10_300x300_ssd_iter_140000.caffemodel",
	help="path to Caffe pre-trained model")
ap.add_argument("-c", "--confidence", type=float, default=0.5,
	help="minimum probability to filter weak detections")
args = vars(ap.parse_args())

# load our serialized model from disk
print("[INFO] loading model...")
net = cv2.dnn.readNetFromCaffe(args["prototxt"], args["model"])



def read_cam():
        cap = cv2.VideoCapture("nvarguscamerasrc ! video/x-raw(memory:NVMM), width=(int)640, height=(int)480,format=(string)NV12, framerate=(fraction)30/1 ! nvvidconv ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink")
        if cap.isOpened():
                cv2.namedWindow("demo", cv2.WINDOW_AUTOSIZE)
                while True:
                        ret_val, frame = cap.read();
                        cv2.imshow('demo',frame)



#SLOWS DOWN HERE
                        # grab the frame dimensions and convert it to a blob
                        (h, w) = frame.shape[:2]
                        blob = cv2.dnn.blobFromImage(cv2.resize(frame, (300, 300)), 1.0,
                                (300, 300), (104.0, 177.0, 123.0))
                 
                        # pass the blob through the network and obtain the detections and
                        # predictions
                        net.setInput(blob)
                        detections = net.forward()
                        print("detections ", detections)



                        
                        if cv2.waitKey(1) == ord('q'):
                              break
        else:
                print ("camera open failed")

        cv2.destroyAllWindows()


if __name__ == '__main__':
        print(cv2.getBuildInformation())
        Gst.debug_set_active(True)
        Gst.debug_set_default_threshold(3)
        read_cam()
7

Nvidia Please respond, bcs the only reason for me to use Jetson nano is its image processing performance but looks like its not even supporting opencv, please help or recommend a better approach using opencv as most of the stuff is based on opencv

8

Thank you. I will turn your source on my jetson nano

(args) Error timeout: can not create camera provider (in src / rpc / socket / client / socketClientDispatch.cpp, Could you tell me the cause of this error?

I am using a logitech USB camera.

9

Hi,

Sorry for the late reply.

You can customize the model with your own dataset to get a better accuracy.
Here is our tutorial: https://github.com/dusty-nv/jetson-inference/blob/master/docs/detectnet-training.md

For camera use case, it’s not recommended to use third-party library, ex. openCV.
Please check our MMAPI or Deepstream(available in Q2) for a better performance.

Thanks.

1 Like
10

It is very unfortunate that opencv is not recommended

The idea is to not reinvent a wheel and I dont think jetson nano never reach the accuracy of a DNN trained with millions of images

[b]Nvidia should clearly mention that OpenCV not recommended or supported

than advertising it like this

[/b]

Starting to loose hope with Nvidia Jetson Nano… :(

12

opencv works fine

not as fast as the demo for the jetson (prob because that’s written in C).
on a macbook, the above code webcam runs the face detection at 12 fps (but does not use gstreamer) so i’m not surprised it’s abit choppy on the tegra.

13

The main concern is what they claim vs reality

You can not compare MacBook cpu with you CUDA, GPUs should perform better unfortunately it’s not

After all NVIDIA claims they support OpenCV and CUDA optimised but it’s untrue

14

Hi,

OpenCV is supported and worked on Jetson.
But if you want a high-performance multimedia solution, it’s recommended to use MMAPI or Deepstream.

OpenCV is widely used and have lots of vision based features.
So we add it into the default package to ease user’s overhead.

But we also have some multimedia solution and we do a lot to optimize it on the Jetson platform.
This is why we recommend using our solution rather than OpenCV.

Thanks.

3 Likes
15

Please put this statement clear for users than using OpenCV in many official videos & docs as if it’s very much performant and recommended by nvidia

Also it will be better if there is any comparison benchmarks between OpenCV vs MMAPI. To make better decisions

16

How can I use MMAPI or deepstream in jetson nano. Please point me to any documentation/links

17

Hi,

Deepstream will be released in Q2 2019.
You can register a notification here: https://developer.nvidia.com/deepstream-sdk/jetson-nano-notify-me

MMAPI is installed with the SDKmanager.
You can find the sample at /usr/src/tegra_multimedia_api/.
Document is L4T Multimedia API Reference: Data Structures

Thanks.

18

Hi,

I found that NVIDIA has optimized the OpenCV, but when I started Nano with SD card system, How do I use optimized OpenCV？
I built and installed OpenCV manually as so far and it really took a long time.

https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%2520Linux%2520Driver%2520Package%2520Development%2520Guide%2Foverview.html%23

Provides OpenCV that NVIDIA optimized specifically for the Tegra platform. It contains Tegra-specific optimizations that enable it to run faster than the OpenCV implementation.

Thanks

19

Both should be installed side by side. The Nvidia provided OpenCV is installed in /usr (/usr/bin for executables, /usr/include for headers, /usr/lib for libraries, etc). All build scripts I’ve seen posted here, by default, configure the prefix to be /usr/local (so /usr/local/bin, /usr/local/include, /usr/local/lib, etc).

Are you looking to use OpenCV with Python or C++ (or some other bindings)?

1 Like
20

Ok, So does recently released Jetpack 4.2.1 support opencv to run on CUDA cores real time operation as stated here?

https://devtalk.nvidia.com/default/topic/1057580/jetson-nano/jetpack-4-2-1-l4t-r32-2-release-for-jetson-nano-jetson-tx1-tx2-and-jetson-agx-xavier/5

"OpenCV is the leading open source library for computer vision, image processing and machine learning, and now features GPU acceleration for real-time operation."

Thanks

21

It is true, though. OpenCV has CUDA acceleration support for some things, but not everything is GPU accelerated. and most algorithms run purely on the CPU. Not all functions have GPU versions and the conversion is not automatic. In other words; a program written for CPU will have to be rewritten before it uses the GPU.

A macbook CPU is much more powerful than the Nano’s for sure, but the Nano’s CPU is not supposed to be powerful. It only needs to be powerful enough to keep up with the GPU and that it does well. You offload what you can onto the GPU to make things go fast.

1 Like