NVIDIA Webinar — Deep Reinforcement Learning in Robotics with NVIDIA Jetson

dusty_nv · May 14, 2018, 4:07pm

Deep Reinforcement Learning in Robotics with NVIDIA Jetson

Register today for a free NVIDIA Jetson webinar from yours truly to learn about the creation of autonomous reinforcement learning agents for robotics.

Join this engaging, informative webinar to learn:

Modern approaches in deep reinforcement learning for implementing flexible tasks and behaviors in robots, like pick-and-place and path planning
Deep reinforcement algorithm verification in PyTorch and 2D environments in the OpenAI Gym
3D robotic simulation with Gazebo and C++/CUDA and leveraging online transfer learning to transition to real-world platforms
Explore the use of NVIDIA's embedded Jetson TX1/TX2 and JetPack for seamless deployment throughout the reinforcement learning workflow built on a portable, open-source AI software stack.

Also, join us after the presentation for a live Q&A session.

Date & Time — Thursday, June 14 2018, 11:00am - 12:00pm Pacific
Registration — https://nvda.ws/2KdbtaO

Don’t forget to watch our previous series of webinars from the On Demand recordings below:

MartinSnaith · June 17, 2018, 1:07pm

Hello Dusty,

Thanks for the interesting webinar.
It’s great to see reinforcement learning getting so much attention.
In the early 90’s my research was centred around using Temporal Difference (Q) learning on walking and maze learning mobile robots. In fact we were probably the first to demonstrate the techniques on real robots. (Several international conferences and papers. Blah, blah, blah)
Having started up full time research again three years ago I’m fascinated to see the amazing hardware and software we now have easy access to.
Nvidia support libraries and demos are particularly useful now as I have three working research platforms (2 drones and a tracked rover) which are ready to start learning unstructured real environments.
I had intended using an A3C based on the ray_framework example from Berkley but yours looks more fun.
With that in mind I’ve cloned your Git repository and tried a build on one of my Ubuntu 16.04 machines.
Unfortunately building Gazebo has proved impossible due to a gstreamer library mismatch.

libgstreamer-plugins-base1.0-dev_1.8.0-1ubuntu1_amd64.deb

Won’t install as my underlying libraries are slightly newer. These are there to support openVX so I’m wary of regressing them as that machine supports my CUDA optical flow work.
The make fails at:
cpp:27:32: fatal error: gst/app/gstappsink.h: No such file or directory #include <gst/app/gstappsink.h>

After a day or so trying to patch up the dependencies including installing the required gst files, I’ve even tried to install without Gazebo but still no luck.

Any way. I do have other CUDA dev boxes available.
Do you have a suggested build for gstreamer and Gazebo I can put on another box to get the demos up?
And is it possible to build the pytorch, c and CUDNN stuff on the current box without Gazebo by editing your setup scripts?

Thanks again.

Regards,

Martin

linuxdev · June 17, 2018, 7:47pm

That file is on my system, and dpkg says it is from package “libgstreamer-plugins-base1.0-dev”:

ubuntu@x2:~$ dpkg -S /usr/include/gstreamer-1.0/gst/app/gstappsink.h 
libgstreamer-plugins-base1.0-dev: /usr/include/gstreamer-1.0/gst/app/gstappsink.h

Beware that if your program wants this:

#include <gst/app/gstappsink.h>

…then you have to add “/usr/include/gstreamer-1.0” to your ~~linker~~header search path (~~e.g., “-L/usr/include/gstreamer-1.0”~~ eeps! EDIT, that was for linking…“-I/usr/include/gstreamer-1.0” is correct…capital of “-i”).

dusty_nv · June 17, 2018, 9:27pm

If you are still unable to get the gstreamer plugins installed, you could try removing gstCamera.h/cpp from jetson-reinforcement/utils/camera and removing references to gstapp-1.0 from utils/CMakeLists.txt. That camera wrapper isn’t actually used by the Gazebo part of the tutorial.

MartinSnaith · June 26, 2018, 12:31pm

Thanks for the advice.
Unfortunately no luck with the install on my OF machine.

So starting with a fresh Ubuntu 16.04 LTS install on an identical machine and a fresh clone from github.
After adding the old-releases repos and installing the CUDA toolkit using Jetpack installer
The cmake works but make fails at link with an -lcuda not found error:

[ 48%] Linking CXX shared library x86_64/lib/libjetson-reinforcement.so
/usr/bin/ld: cannot find -lcuda
collect2: error: ld returned 1 exit status
CMakeFiles/jetson-reinforcement.dir/build.make:188: recipe for target ‘x86_64/lib/libjetson-reinforcement.so’ failed
make[2]: *** [x86_64/lib/libjetson-reinforcement.so] Error 1
CMakeFiles/Makefile2:67: recipe for target ‘CMakeFiles/jetson-reinforcement.dir/all’ failed
make[1]: *** [CMakeFiles/jetson-reinforcement.dir/all] Error 2
Makefile:83: recipe for target ‘all’ failed
make: *** [all] Error 2

Ok, so I checked the Cuda install and it looks ok:

jetson-reinforcement/build$ grep CUDA_TOOLKIT_ROOT_DIR *
CMakeCache.txt:CUDA_TOOLKIT_ROOT_DIR:PATH=/usr/local/cuda-9.0
CMakeCache.txt://This is the value of the last time CUDA_TOOLKIT_ROOT_DIR was
CMakeCache.txt:CUDA_TOOLKIT_ROOT_DIR_INTERNAL:INTERNAL=/usr/local/cuda-9.0

jetson-reinforcement/build$ which nvcc
/usr/local/cuda-9.0/bin/nvcc
jetson-reinforcement/build$ ls -l /usr/local/cuda
lrwxrwxrwx 1 root root 8 Jun 25 23:38 /usr/local/cuda → cuda-9.0
jetson-reinforcement/build$ ls /usr/local/cuda/bin/nvcc
/usr/local/cuda/bin/nvcc

jetson-reinforcement/build$ ldconfig -p | grep cuda
libnvrtc.so.9.0 (libc6,x86-64) => /usr/local/cuda-9.0/targets/x86_64-linux/lib/libnvrtc.so.9.0
libnvrtc.so (libc6,x86-64) => /usr/local/cuda-9.0/targets/x86_64-linux/lib/libnvrtc.so
libnvrtc-builtins.so.9.0 (libc6,x86-64) => /usr/local/cuda-9.0/targets/x86_64-linux/lib/libnvrtc-builtins.so.9.0

…

libaccinj64.so (libc6,x86-64) => /usr/local/cuda-9.0/targets/x86_64-linux/lib/libaccinj64.so
libOpenCL.so.1 (libc6,x86-64) => /usr/local/cuda-9.0/targets/x86_64-linux/lib/libOpenCL.so.1
libOpenCL.so (libc6,x86-64) => /usr/local/cuda-9.0/targets/x86_64-linux/lib/libOpenCL.so

and finally:

echo $LD_LIBRARY_PATH
/usr/local/cuda-9.0/lib64:
echo $PATH
/usr/local/cuda-9.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin

Which all looks OK to me. Have I missed something obvious?
Any advice most welcome.

Thanks,

Martin

dusty_nv · June 26, 2018, 1:55pm

Hi Martin, have you been able to locate the libcuda driver API on your system?

MartinSnaith · June 29, 2018, 8:30am

Hello Dusty,

find /usr -type f -name “libcuda*”
/usr/local/cuda-9.0/targets/aarch64-linux/lib/stubs/libcuda.so
/usr/local/cuda-9.0/targets/aarch64-linux/lib/libcudart.so.9.0.252
/usr/local/cuda-9.0/targets/aarch64-linux/lib/libcudadevrt.a
/usr/local/cuda-9.0/targets/aarch64-linux/lib/libcudart_static.a
/usr/local/cuda-9.0/targets/x86_64-linux/lib/stubs/libcuda.so
/usr/local/cuda-9.0/targets/x86_64-linux/lib/libcudart.so.9.0.252
/usr/local/cuda-9.0/targets/x86_64-linux/lib/libcudadevrt.a
/usr/local/cuda-9.0/targets/x86_64-linux/lib/libcudart_static.a
/usr/local/cuda-9.0/doc/man/man7/libcuda.7
/usr/local/cuda-9.0/doc/man/man7/libcudart.so.7
/usr/local/cuda-9.0/doc/man/man7/libcudart.7
/usr/local/cuda-9.0/doc/man/man7/libcuda.so.7
/usr/local/cuda-9.0/extras/Debugger/include/libcudacore.h
/usr/local/cuda-9.0/extras/Debugger/lib64/libcudacore.a

nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2017 NVIDIA Corporation
Built on Sun_Nov_19_03:10:15_CST_2017
Cuda compilation tools, release 9.0, V9.0.252

echo $PATH
/usr/local/cuda-9.0/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin

So I guess that’s a yes. The libs/stubs directory is not in LD_LIBRARY_PATH is that correct?

Slow replies from me due to Client activity.

Regards,

Martin

dusty_nv · June 29, 2018, 2:36pm

Hi Martin, add the path to your LD_LIBRARY_PATH, and if that doesn’t work, to CMake’s link_directories().
This normally shouldn’t be required — was your Jetson setup successfully with JetPack?

MartinSnaith · July 2, 2018, 12:43pm

Hello Dusty,

Tried both of LD_LIBRARY_PATH and link_directories() suggestions.
Still got the same make error.

Tried a Cuda toolkit re-install with JetPack which went in OK.
Did a reboot, removed the build directory, re ran cmake, re ran make.

Still the same error.

Tried a manual Cuda re-install using Cuda API from CUDA Toolkit 11.7 Update 1 Downloads | NVIDIA Developer
Did a reboot, removed the build directory, re ran cmake, re ran make.

Oh dear, still the same make error.

Boy am I running out of ideas now.

I’m installing on a laptop with this release:

$ uname -m && cat /etc/*release
x86_64
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=16.04
DISTRIB_CODENAME=xenial
DISTRIB_DESCRIPTION=“Ubuntu 16.04.4 LTS”
NAME=“Ubuntu”
VERSION=“16.04.4 LTS (Xenial Xerus)”
ID=ubuntu
ID_LIKE=debian
PRETTY_NAME=“Ubuntu 16.04.4 LTS”
VERSION_ID=“16.04”
HOME_URL=“http://www.ubuntu.com/”
SUPPORT_URL=“http://help.ubuntu.com/”
BUG_REPORT_URL=“http://bugs.launchpad.net/ubuntu/”
VERSION_CODENAME=xenial
UBUNTU_CODENAME=xenial

As ever any suggestions gratefully received.

Regards

Martin

MartinSnaith · July 2, 2018, 1:06pm

Hello Dusty,

Extra information:

sudo ldconfig -p | egrep -i libcuda
libcudart.so.9.2 (libc6,x86-64) => /usr/local/cuda-9.2/lib64/libcudart.so.9.2
libcudart.so.9.0 (libc6,x86-64) => /usr/local/cuda-9.0/targets/x86_64-linux/lib/libcudart.so.9.0
libcudart.so (libc6,x86-64) => /usr/local/cuda-9.0/targets/x86_64-linux/lib/libcudart.so
libcudart.so (libc6,x86-64) => /usr/local/cuda-9.2/lib64/libcudart.so

but no libcuda.so which I would expect to see.

Here are the currently installed libcuda.so

find /usr -type f -name “libcuda.*”

/usr/local/cuda-9.2/doc/man/man7/libcuda.7
/usr/local/cuda-9.2/doc/man/man7/libcuda.so.7
/usr/local/cuda-9.2/lib64/stubs/libcuda.so
/usr/local/cuda-9.0/targets/aarch64-linux/lib/stubs/libcuda.so
/usr/local/cuda-9.0/targets/x86_64-linux/lib/stubs/libcuda.so
/usr/local/cuda-9.0/doc/man/man7/libcuda.7
/usr/local/cuda-9.0/doc/man/man7/libcuda.so.7
/usr/share/man/man7/libcuda.7
/usr/share/man/man7/libcuda.so.7

Regards

Martin

dusty_nv · July 2, 2018, 2:28pm

You could try adding a symbolic link from libcuda to a known location. On x86, typically the NVIDIA driver will install that (not JetPack).

Are you able to build this on your Jetson, or just having trouble with laptop? It seems to be a configuration issue with the environment.

NVIDIA Webinar &mdash; Deep Reinforcement Learning in Robotics with NVIDIA Jetson

NVIDIA Webinar — Deep Reinforcement Learning in Robotics with NVIDIA Jetson