Deepstream inference on TX1 using faster rcnn resnet 18 trained using TLT

zararyounis · November 12, 2019, 12:52pm

Hello,
We are trying to infer faster rcnn resnet 18 on Jetson TX1. We followed the steps on https://github.com/NVIDIA-AI-IOT/deepstream_4.x_apps and we received an error that the plugin _Proposal was not found. Later we discovered that this particular plugin does not exist in the TensorRT 5.1.6(Jetson TX1 Jetpack 4.2.2) but exists in Tensorrt OSS https://github.com/NVIDIA/TensorRT and in order to use it we need libnvinfer_plugin generated using the steps on Tensorrt OSS Github Page(linked above) which must then be replaced with libnvinfer_plugin as stated by the following instructions:

FasterRCNN requires two TensorRT plugins to run. They are the cropAndResizePlugin and the proposalPlugin. Currently, these plugins are not included in the TensorRT 5.1GA (5.1.5.0) installation package, but they can be obtained from the TensorRT Open Source Software (OSS) in GitHub and checkout the branch release/5.1. Please follow the installation guide here, compile the open sourced plugins, and replace the libnvinfer_plugin.* in the installation directory with the one built from TensorRT OSS.
To integrate FasterRCNN model into DeepStream, additional DeepStream plugin is required. It is available here: https://github.com/NVIDIA-AI-IOT/deepstream_4.x_apps.
Replace /Your_deepstream_SDK_v4.0_xxxxx_path with your actual DeepStream SDK 4.0 path in deepstream_4.x_apps/nvdsinfer_customparser_frcnn_uff/Makefile and deepstream_4.x_apps/Makefile. Compile the plugin and sample app

Source: https://docs.nvidia.com/metropolis/TLT/tlt-getting-started-uide/index.html#deepstream_deployment

We tried cross compiling following the steps on Tenssort OSS Github Page we also included the toolchain file for aarch64, using the command

cmake .. -DTRT_LIB_DIR=$TRT_RELEASE/lib -DTRT_BIN_DIR=`pwd`/out -DCMAKE_TOOLCHAIN_FILE=../cmake/toolchains/cmake_aarch64.toolchain -DCUDA_VERSION=10.0

but we received the following error

Building for TensorRT version: 5.1.5.0, library version: 5.1.5
CHECK for 
-- Targeting TRT Platform: aarch64
-- CUDA version set to 10.0
-- cuDNN version set to 7.5
-- Protobuf version set to 3.0.0
CMake Error at /usr/share/cmake-3.15/Modules/FindPackageHandleStandardArgs.cmake:137 (message):
  Could NOT find CUDA (missing: CUDA_CUDART_LIBRARY) (found suitable version
  "10.0", minimum required is "10.0")
Call Stack (most recent call first):
  /usr/share/cmake-3.15/Modules/FindPackageHandleStandardArgs.cmake:378 (_FPHSA_FAILURE_MESSAGE)
  /usr/share/cmake-3.15/Modules/FindCUDA.cmake:1092 (find_package_handle_standard_args)
  CMakeLists.txt:154 (find_package)

-- Configuring incomplete, errors occurred!
See also "/media/dl2/AVT/tx1crosscompile/TensorRT/build/CMakeFiles/CMakeOutput.log".
See also "/media/dl2/AVT/tx1crosscompile/TensorRT/build/CMakeFiles/CMakeError.log".

CMakeError.log is as follows:

Performing C++ SOURCE FILE Test CMAKE_HAVE_LIBC_PTHREAD failed with the following output:
Change Dir: /media/dl2/AVT/tx1crosscompile/TensorRT/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/make cmTC_8433d/fast && /usr/bin/make -f CMakeFiles/cmTC_8433d.dir/build.make CMakeFiles/cmTC_8433d.dir/build
make[1]: Entering directory '/media/dl2/AVT/tx1crosscompile/TensorRT/build/CMakeFiles/CMakeTmp'
Building CXX object CMakeFiles/cmTC_8433d.dir/src.cxx.o
/usr/bin/c++    -Wno-deprecated-declarations  -DBUILD_SYSTEM=cmake_oss -DCMAKE_HAVE_LIBC_PTHREAD   -std=c++11 -o CMakeFiles/cmTC_8433d.dir/src.cxx.o -c /media/dl2/AVT/tx1crosscompile/TensorRT/build/CMakeFiles/CMakeTmp/src.cxx
Linking CXX executable cmTC_8433d
/usr/bin/cmake -E cmake_link_script CMakeFiles/cmTC_8433d.dir/link.txt --verbose=1
/usr/bin/c++  -Wno-deprecated-declarations  -DBUILD_SYSTEM=cmake_oss -DCMAKE_HAVE_LIBC_PTHREAD    CMakeFiles/cmTC_8433d.dir/src.cxx.o  -o cmTC_8433d 
CMakeFiles/cmTC_8433d.dir/src.cxx.o: In function `main':
src.cxx:(.text+0x3e): undefined reference to `pthread_create'
src.cxx:(.text+0x4a): undefined reference to `pthread_detach'
src.cxx:(.text+0x5b): undefined reference to `pthread_join'
src.cxx:(.text+0x6f): undefined reference to `pthread_atfork'
collect2: error: ld returned 1 exit status
CMakeFiles/cmTC_8433d.dir/build.make:86: recipe for target 'cmTC_8433d' failed
make[1]: *** [cmTC_8433d] Error 1
make[1]: Leaving directory '/media/dl2/AVT/tx1crosscompile/TensorRT/build/CMakeFiles/CMakeTmp'
Makefile:121: recipe for target 'cmTC_8433d/fast' failed
make: *** [cmTC_8433d/fast] Error 2

Source file was:
#include <pthread.h>

void* test_func(void* data)
{
  return data;
}

int main(void)
{
  pthread_t thread;
  pthread_create(&thread, NULL, test_func, NULL);
  pthread_detach(thread);
  pthread_join(thread, NULL);
  pthread_atfork(NULL, NULL, NULL);
  pthread_exit(NULL);

  return 0;
}

Determining if the function pthread_create exists in the pthreads failed with the following output:
Change Dir: /media/dl2/AVT/tx1crosscompile/TensorRT/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/make cmTC_47df0/fast && /usr/bin/make -f CMakeFiles/cmTC_47df0.dir/build.make CMakeFiles/cmTC_47df0.dir/build
make[1]: Entering directory '/media/dl2/AVT/tx1crosscompile/TensorRT/build/CMakeFiles/CMakeTmp'
Building CXX object CMakeFiles/cmTC_47df0.dir/CheckFunctionExists.cxx.o
/usr/bin/c++    -Wno-deprecated-declarations  -DBUILD_SYSTEM=cmake_oss -DCHECK_FUNCTION_EXISTS=pthread_create   -std=c++11 -o CMakeFiles/cmTC_47df0.dir/CheckFunctionExists.cxx.o -c /media/dl2/AVT/tx1crosscompile/TensorRT/build/CMakeFiles/CheckLibraryExists/CheckFunctionExists.cxx
Linking CXX executable cmTC_47df0
/usr/bin/cmake -E cmake_link_script CMakeFiles/cmTC_47df0.dir/link.txt --verbose=1
/usr/bin/c++  -Wno-deprecated-declarations  -DBUILD_SYSTEM=cmake_oss -DCHECK_FUNCTION_EXISTS=pthread_create    CMakeFiles/cmTC_47df0.dir/CheckFunctionExists.cxx.o  -o cmTC_47df0 -lpthreads 
/usr/bin/ld: cannot find -lpthreads
collect2: error: ld returned 1 exit status
CMakeFiles/cmTC_47df0.dir/build.make:86: recipe for target 'cmTC_47df0' failed
make[1]: *** [cmTC_47df0] Error 1
make[1]: Leaving directory '/media/dl2/AVT/tx1crosscompile/TensorRT/build/CMakeFiles/CMakeTmp'
Makefile:121: recipe for target 'cmTC_47df0/fast' failed
make: *** [cmTC_47df0/fast] Error 2

CMakeOutput.log file is as follows:

The target system is: Linux -  - aarch64
The host system is: Linux - 5.0.0-32-generic - x86_64
Compiling the CXX compiler identification source file "CMakeCXXCompilerId.cpp" succeeded.
Compiler: /usr/bin/c++ 
Build flags: 
Id flags:  

The output was:
0

Compilation of the CXX compiler identification source "CMakeCXXCompilerId.cpp" produced "a.out"

The CXX compiler identification is GNU, found in "/media/dl2/AVT/tx1crosscompile/TensorRT/build/CMakeFiles/3.15.5/CompilerIdCXX/a.out"

Determining if the include file pthread.h exists passed with the following output:
Change Dir: /media/dl2/AVT/tx1crosscompile/TensorRT/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/make cmTC_0113c/fast && /usr/bin/make -f CMakeFiles/cmTC_0113c.dir/build.make CMakeFiles/cmTC_0113c.dir/build
make[1]: Entering directory '/media/dl2/AVT/tx1crosscompile/TensorRT/build/CMakeFiles/CMakeTmp'
Building CXX object CMakeFiles/cmTC_0113c.dir/CheckIncludeFile.cxx.o
/usr/bin/c++    -Wno-deprecated-declarations  -DBUILD_SYSTEM=cmake_oss    -std=c++11 -o CMakeFiles/cmTC_0113c.dir/CheckIncludeFile.cxx.o -c /media/dl2/AVT/tx1crosscompile/TensorRT/build/CMakeFiles/CMakeTmp/CheckIncludeFile.cxx
Linking CXX executable cmTC_0113c
/usr/bin/cmake -E cmake_link_script CMakeFiles/cmTC_0113c.dir/link.txt --verbose=1
/usr/bin/c++  -Wno-deprecated-declarations  -DBUILD_SYSTEM=cmake_oss     CMakeFiles/cmTC_0113c.dir/CheckIncludeFile.cxx.o  -o cmTC_0113c 
make[1]: Leaving directory '/media/dl2/AVT/tx1crosscompile/TensorRT/build/CMakeFiles/CMakeTmp'

Determining if the function pthread_create exists in the pthread passed with the following output:
Change Dir: /media/dl2/AVT/tx1crosscompile/TensorRT/build/CMakeFiles/CMakeTmp

Run Build Command(s):/usr/bin/make cmTC_84f65/fast && /usr/bin/make -f CMakeFiles/cmTC_84f65.dir/build.make CMakeFiles/cmTC_84f65.dir/build
make[1]: Entering directory '/media/dl2/AVT/tx1crosscompile/TensorRT/build/CMakeFiles/CMakeTmp'
Building CXX object CMakeFiles/cmTC_84f65.dir/CheckFunctionExists.cxx.o
/usr/bin/c++    -Wno-deprecated-declarations  -DBUILD_SYSTEM=cmake_oss -DCHECK_FUNCTION_EXISTS=pthread_create   -std=c++11 -o CMakeFiles/cmTC_84f65.dir/CheckFunctionExists.cxx.o -c /media/dl2/AVT/tx1crosscompile/TensorRT/build/CMakeFiles/CheckLibraryExists/CheckFunctionExists.cxx
Linking CXX executable cmTC_84f65
/usr/bin/cmake -E cmake_link_script CMakeFiles/cmTC_84f65.dir/link.txt --verbose=1
/usr/bin/c++  -Wno-deprecated-declarations  -DBUILD_SYSTEM=cmake_oss -DCHECK_FUNCTION_EXISTS=pthread_create    CMakeFiles/cmTC_84f65.dir/CheckFunctionExists.cxx.o  -o cmTC_84f65 -lpthread 
make[1]: Leaving directory '/media/dl2/AVT/tx1crosscompile/TensorRT/build/CMakeFiles/CMakeTmp'

Also we do not have much space on Jetson TX1 since all packages mentioned in Jetpack 4.2.2 were installed. Therefore we would like to cross compile.

Morganh · November 13, 2019, 10:06am

Hi zararyounis,
I’m addressing into this issue and will update to you once there is any findings.

zararyounis · November 13, 2019, 1:51pm

Thanks,

Eagerly awaiting your response.

Morganh · November 15, 2019, 7:53am

Hi zararyounis,
After checking, for TRT5.1 OSS branch,there is limitation to cross compile aarch64 version components.
If possible, could you please generate the nvinfer_plugin.so* from Jetson boards.
Any one is fine. You can generate it from another TX1 or nano or xavier as long as it is installed via the same Jetpack.

Step:

$ git clone -b release/5.1 https://github.com/nvidia/TensorRT  && cd  TensorRT
$ git submodule update --init --recursive && export TRT_SOURCE=`pwd`
$ cd $TRT_SOURCE  
$ mkdir -p build && cd build
$ wget https://github.com/Kitware/CMake/releases/download/v3.13.5/cmake-3.13.5.tar.gz
$ tar xvf cmake-3.13.5.tar.gz
$ cd cmake-3.13.5/ && ./configure && make && sudo make install
$ cd ..
$ /usr/local/bin/cmake .. -DTRT_BIN_DIR=`pwd`/out
$ make nvinfer_plugin -j$(nproc)

The nvinfer_plugin.so* will be available in the pwd/out folder.

From above steps, totally it will cost about 850M size. And 440M is from cmake-3.13.5.

zararyounis · December 2, 2019, 8:14am

Thank you, Moganh.