Is there any demos available for python jetson inference

Hi

I have successfully run the demos available in jetson inference. As I have been devloping all my code in python so I am bit comfortable in python instead of C++. But all the demo in jetson inference are with C++ only.

Just wanted to know if there is any python demo available. ?
Any link/tutorials/documentation.

Thanks

The C++ is very well commented. My preferred language is not C++ but I was able to follow. You might give it a shot anyway and post any problems you run into here.

Hi guys, thanks for the feedback - I do plan to add Python support and samples to jetson-inference.

For the time being, TensorRT also has a Python API that you could use directly.

Any link/tutorial for tensorRT python for jetson nano. Thanks

You should see them on your Jetson under /usr/src/tensorrt

Here’s also a link to the TensorRT Python API documentation:

[url]https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html#python_topics[/url]

are you looking for object detection or classification? I have some tfrt object detection code that works ok, although I found the pytorch is faster.

Do you have any examples on github or anywhere else. I need some examples for reference so that I can later work it on my own.

Thanks

Not yet, i plan on putting on a zoo but currently still too busy exploring. If you let me know what are you looking for and on what framework i can send you something to play with

I am looking for tensorflow with basic object detection in python. If you can send any examples for this, that would be really helpful. Thanks

this should start you up - it requires no special setup IIRC. Remember that the one off stages are notoriously slow so don’t cntrl-c… If you improve on that, please post here:

from PIL import Image
from tf_trt_models.detection import download_detection_model
import numpy as np
import tensorflow as tf
import time
from tf_trt_models.detection import download_detection_model, build_detection_graph
import tensorflow.contrib.tensorrt as trt
import graphsurgeon as gs
import sys
from tf_trt_models.classification import download_classification_checkpoint
from tf_trt_models.classification import build_classification_graph

if len(sys.argv) > 1 and sys.argv[1] == "download":
    config_path, checkpoint_path = download_detection_model('ssd_mobilenet_v1_coco')
    print(config_path, checkpoint_path)
    print("building frozen graph....")
    frozen_graph, input_names, output_names = build_detection_graph(
        config=config_path,
        checkpoint=checkpoint_path
    )
    print(input_names,output_names)
    print("building inference graph....")
    trt_graph = trt.create_inference_graph(
        input_graph_def=frozen_graph,
        outputs=output_names,
        max_batch_size=1,
        max_workspace_size_bytes=1 << 25,
        precision_mode='FP16',
        minimum_segment_size=50
    )

    print("saving...")
    with open('./ssd_mobilenet_v1_coco_trt.pb', 'wb') as f:
        f.write(trt_graph.SerializeToString())
    load_file = False
else:
    load_file = True
    trt_graph= tf.GraphDef()
    with tf.gfile.GFile('./ssd_mobilenet_v1_coco_trt.pb', 'rb') as pf:
       trt_graph.ParseFromString(pf.read())
       input_names = ['image_tensor']
       output_names = ['detection_boxes', 'detection_classes', 'detection_scores', 'num_detections']


print("creating session...")
tf_config = tf.ConfigProto()
tf_config.gpu_options.allow_growth = True

tf_sess = tf.Session(config=tf_config)

tf.import_graph_def(trt_graph, name='')

tf_input = tf_sess.graph.get_tensor_by_name(input_names[0] + ':0')
tf_scores = tf_sess.graph.get_tensor_by_name('detection_scores:0')
tf_boxes = tf_sess.graph.get_tensor_by_name('detection_boxes:0')
tf_classes = tf_sess.graph.get_tensor_by_name('detection_classes:0')
tf_num_detections = tf_sess.graph.get_tensor_by_name('num_detections:0')

image = Image.open("t.jpg")

image_resized = np.array(image.resize((300, 300)))
image = np.array(image)

print("inference...")

for _ in range(10):
    s = time.time()
    scores, boxes, classes, num_detections = tf_sess.run([tf_scores, tf_boxes, tf_classes, tf_num_detections], feed_dict={
        tf_input: image_resized[None, ...]
    })
    print(time.time()-s)

boxes = boxes[0] # index by 0 to remove batch dimension
scores = scores[0]
classes = classes[0]
num_detections = num_detections[0]

for i in range(int(num_detections)):
    print('%d (%0.2f) ' % (classes[i], scores[i]))

oh, sorry, it does require the tf_trt_models from GitHub - NVIDIA-AI-IOT/tf_trt_models: TensorFlow models accelerated with NVIDIA TensorRT

did it work for you?

UPDATE (7/9): Python support has been merged into the master branch. No longer use the Python branch. See here for more info: https://devtalk.nvidia.com/default/topic/1057006/jetson-nano/hello-ai-world-now-supports-python-and-onboard-training-with-pytorch-/

OK, I’ve checked-in initial support for Python in Hello AI World here:

https://github.com/dusty-nv/jetson-inference/

If you could test it and let me know if it works for you, that would be helpful - thanks.

Here is the test to run:

# build the python branch
$ git clone -b python https://github.com/dusty-nv/jetson-inference jetson-inference-python
$ cd jetson-inference-python
$ git submodule update --init
$ mkdir build
$ cd build
$ cmake ../
$ sudo make install

# run the python recognition sample
$ cd aarch64/bin
$ python my-recognition.py --network=googlenet black_bear.jpg

And here is what the Python code looks like:

import jetson.inference
import jetson.utils

import argparse

# parse the command line
parser = argparse.ArgumentParser()
parser.add_argument("filename", type=str, help="filename of the image to process")
parser.add_argument("--network", type=str, default="googlenet", help="model to use, can be:  alexnet, googlenet, googlenet-12")
opt = parser.parse_args()

print(opt)

# load an image (into shared CPU/GPU memory)
img, width, height = jetson.utils.loadImageRGBA(opt.filename)

# load the recognition network
net = jetson.inference.imageNet(opt.network)

# classify the image
class_idx, confidence = net.Classify(img, width, height)

# find the object description
class_desc = net.GetClassDesc(class_idx)

# print out the result
print("image is recognized as '{:s}' (class #{:d}) with {:f}% confidence".format(class_desc, class_idx, confidence * 100))

Underneath it is using the Python C API to call the C++ objects from jetson-inference using TensorRT, so the performance should still be good.
Currently I have the jetson.inference.imageNet ready, next I will work on the bindings for detectNet. I also plan to add support for numpy arrays.

Thank you dusty,

some comments:

  1. you need to add “git submodule update --init” after "
    cd jetson-inference-python" for the cmake to work.
    1. it carries with it the overwhelming weight of qt4. If it can be make into something cleaner that does not require graphics it would be better as a library.

    2. model downloads: It would be nice to be selective about what you want to download - it takes a while and take a lot of disk space. Also, the progress creates so many lines that you can’t go back and see what actually happened before.

    while trying to run your example i got:

    m@m-jetson-tests:/mnt/usbdrive/jetson-inference-python/build/aarch64/bin$ python3 my-recognition.py --network=googlenet black_bear.jpg 
    jetson.inference.__init__.py
    Traceback (most recent call last):
      File "my-recognition.py", line 23, in <module>
        import jetson.inference
      File "/usr/lib/python3.6/dist-packages/jetson/inference/__init__.py", line 4, in <module>
        from jetson_inference_python import *
    ImportError: libjetson-utils.so: cannot open shared object file: No such file or directory
    

Thanks, forgot that! Added that step above.

I will have to look into this, in the future perhaps it makes more sense to just use X11/XGL directly. The GUI is used for the camera applications.

That’s another good idea, perhaps I could make a curses-like interface to select the models. And maybe I can change wget to be less noisy. Thanks.

Can you try running “sudo ldconfig” and see if that helps?

Also are you able to run the existing C++ applications from jetson-inference-python/build/aarch64/bin (like imagenet-console)?

Thanks for beta testing!

sudo ldconfig did the trick!
it works well, around 20ms/image on my 10w nano (I know, I know, the power supply is just around the corner)
now for object detection…

OK thanks for letting me know, I did previously add this as an install step that runs during “sudo make install”, but I will take another look at why it didn’t seem to work initially for you (presumably related to a system running this branch for the first time).

I’m working for the bindings for detectNet now, and also camera. Will let you know when that branch of the repo is ready to try again.

Hi guys, ok the Python bindings for detectNet are ready to try, along with camera streaming and OpenGL display.

Re-clone the python branch of jetson-inference repo, and after building/installing it, try running these:

$ cd jetson-inference-python/build/aarch64/bin
$ ./camera-viewer.py   # test camera streaming
$ ./detectnet-console.py --network=coco-dog dog_1.jpg test_dog_1.jpg
$ ./detectnet-console.py --network=pednet peds-004.jpg test_peds_4.jpg
$ ./detectnet-camera.py --network=facenet

And here is the Python code for realtime object detection with camera streaming, pretty simple!

import jetson.inference
import jetson.utils

import argparse

# parse the command line
parser = argparse.ArgumentParser()

parser.add_argument("--network", type=str, default="pednet", help="model to use, can be:  pednet, multiped, facenet, coco-dog, coco-bottle, coco-chair, coco-airplane")
parser.add_argument("--threshold", type=float, default=0.5, help="minimum detection threshold to use")
parser.add_argument("--width", type=int, default=1280, help="desired width of camera stream (default is 1280 pixels)")
parser.add_argument("--height", type=int, default=720, help="desired height of camera stream (default is 720 pixels)")
parser.add_argument("--v4l2_device", type=int, default=-1, help="if using VL42 camera, index of the desired /dev/video node")

opt, argv = parser.parse_known_args()

# load the object detection network
net = jetson.inference.detectNet(opt.network, argv, opt.threshold)

# create the camera and display
camera = jetson.utils.gstCamera(opt.width, opt.height, opt.v4l2_device)
display = jetson.utils.glDisplay()

# process frames until user exits
while display.IsOpen():
	# capture the image
	img, width, height = camera.CaptureRGBA()

	# detect objects in the image (with overlay)
	detections = net.Detect(img, width, height)

	# print the detections
	print("detected {:d} objects in image".format(len(detections)))

	for detection in detections:
		print(detection)

	# render the image
	display.RenderOnce(img, width, height)

	# update the title bar
	display.SetTitle("{:s} | {:.0f} FPS".format(opt.network, display.GetFPS()))

Next I am going to add support for importing arrays into memory from numpy.

Hi dusty,

Thank you again for all the amazing work you both are doing. Couple of questions:

  1. can this be used headless?
  2. can the gstCamera open rtsp stream?
  3. by “using numpy” did you mean that it can’t accept opencv acquired images yet?
  4. do you have performance benchmarks?

Thank again,
Moshe

Hi Moshe, thanks for your interest, here are my responses to your questions above:

The example camera programs (imagenet-camera.py and detectnet-camera.py) are setup to render the video with OpenGL, but if you remove the few lines of code in them for OpenGL display they will work headless. The imagenet-console.py and detectnet-console.py programs already work headless.

Not by default, but you can add RTSP to the GStreamer pipeline here:
https://github.com/dusty-nv/jetson-utils/blob/00fd5284b7a10dd9c9e6dbe12dc18b5736c61fb4/camera/gstCamera.cpp#L427

Right now it passes around CUDA memory using opaque Python capsule containers, so I am adding numpy as a way of getting data into and out of those containers. The capsules are used to achieve the best performance so that Python isn’t needlessly touching and copying around the GPU memory, which brings me to your question #4 below…

In comparing the C++ versions of the jetson-inference programs to their new Python counterparts, I see the same performance from both, which was my goal. To do this I wrote the Python bindings so they called into the C++ classes for image recognition and object detection that do the heavy lifting, and using the memory capsule containers mentioned above to route the buffers around without any extra memory copies.