nvivafilter on tx1 l4t 28.2.1 : SIGSEGV

Hello experts,

I try to use nvivafilter in nvcamerasrc based gstreamer pipeline but merely inserting nvivafilter in my otherwise perfectly working pipeline makes my pipeline crash with a SIGSEGV.

Details : I work on a TX1 devkit with a custom camera and l4t 28.2.1.

The following pipeline works perfectly and endlessly :

nvidia@cam5-0002:~/nvsample_cudaprocess$ gst-launch-1.0 -v nvcamerasrc sensor-id=1 ! \
'video/x-raw(memory:NVMM), width=(int)2448, height=(int)2048' ! \
nvjpegenc ! multifilesink max-files=5 location=/run/user/1001/%05d

If I merely add a nvivafilter in my pipeline my pipeline crashes, with or without specifying a .so file, and when I specify the nvsample_cudaprocess example, even when I fill the three pointers with NULL in the ‘init’ entry point.

Exemple :

nvidia@cam5-0002:~/nvsample_cudaprocess$ gst-launch-1.0 -v nvcamerasrc sensor-id=1 ! \
'video/x-raw(memory:NVMM), width=(int)2448, height=(int)2048' ! \
nvivafilter customer-lib-name="`pwd`/libnvsample_cudaprocess.so" ! \
nvjpegenc ! multifilesink max-files=5 location=/run/user/1001/%05d
Setting pipeline to PAUSED ...
plugin init

Available Sensor modes :
2464 x 2048 FR=30.000000 CF=0x1209208a10 SensorModeType=4 CSIPixelBitDepth=12 DynPixelBitDepth=12
1936 x 1080 FR=30.000000 CF=0x1209208a10 SensorModeType=4 CSIPixelBitDepth=12 DynPixelBitDepth=12
2464 x 2065 FR=30.000000 CF=0x1209208a10 SensorModeType=4 CSIPixelBitDepth=12 DynPixelBitDepth=12
1936 x 1105 FR=30.000000 CF=0x1209208a10 SensorModeType=4 CSIPixelBitDepth=12 DynPixelBitDepth=12
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock

NvCameraSrc: Trying To Set Default Camera Resolution. Selected sensorModeIndex = 0 WxH = 2464x2048 FrameRate = 30.000000 ...

/GstPipeline:pipeline0/GstNvCameraSrc:nvcamerasrc0.GstPad:src: caps = "video/x-raw\(memory:NVMM\)\,\ width\=\(int\)2448\,\ height\=\(int\)2048\,\ format\=\(string\)NV12\,\ framerate\=\(fraction\)30/1"
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = "video/x-raw\(memory:NVMM\)\,\ width\=\(int\)2448\,\ height\=\(int\)2048\,\ format\=\(string\)NV12\,\ framerate\=\(fraction\)30/1"
/GstPipeline:pipeline0/Gstnvivafilter:nvivafilter0.GstPad:src: caps = "video/x-raw\(memory:NVMM\)\,\ width\=\(int\)2448\,\ height\=\(int\)2048\,\ format\=\(string\)NV12\,\ framerate\=\(fraction\)30/1"
/GstPipeline:pipeline0/GstNvJpegEnc:nvjpegenc0.GstPad:sink: caps = "video/x-raw\(memory:NVMM\)\,\ width\=\(int\)2448\,\ height\=\(int\)2048\,\ format\=\(string\)NV12\,\ framerate\=\(fraction\)30/1"
/GstPipeline:pipeline0/Gstnvivafilter:nvivafilter0.GstPad:sink: caps = "video/x-raw\(memory:NVMM\)\,\ width\=\(int\)2448\,\ height\=\(int\)2048\,\ format\=\(string\)NV12\,\ framerate\=\(fraction\)30/1"
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = "video/x-raw\(memory:NVMM\)\,\ width\=\(int\)2448\,\ height\=\(int\)2048\,\ format\=\(string\)NV12\,\ framerate\=\(fraction\)30/1"
Caught SIGSEGV
#0  0x0000007f85ce4030 in __GI___poll (fds=0x669390, nfds=2,
#1  0x0000007f85de53ac in ?? () from /lib/aarch64-linux-gnu/libglib-2.0.so.0
#2  0x00000000004b9300 in ?? ()
Spinning.  Please run 'gdb gst-launch-1.0 1855' to continue debugging, Ctrl-C to quit, or Ctrl-\ to dump core

The pipeline crashes immediately; even the first file is not created.

Standalone cuda/argus programs work on the same platform. Cuda is cuda 9

Any hint ?

Hi,

Deepstream 1.5 only supports JetPack3.2, which is rel-28.2.
Could you reflash your device with JetPack3.2 and give it a try again?

Thanks.

Hi AastaLLL,

I don’t really need to use nvivafilter if it is a deprecated plugin as I have not yet a valuable .so for it.

As I start with a new project, should I use Deepstream 2.0 instead ? I work with TX1 boards.

Which plugin is the new equivalent in Deepstream 2.0 of nvivafilter ?

Thanks

Hi,

Please use JetPack3.2 to get it works.
DeepStream for Jetson only supports rel-28.2 due to some internal dependency.

By the way, DeepStream 2.0 is for Tesla, it cannot be used on aarch64 environment.
Thanks.

Is the dependency related to the kernel or to the files in the rootfs (or something else). I’d like to know as I compile my own (l4t 28.2.1 based) kernel ?

Hi,

The dependency comes from DeepStream library.
Please use rel28.2 to make it work.

Thanks.

Hi AastaLLL

I have installed a fresh jetpack 3.2.1 (using JetPack-L4T-3.2.1-linux-x64_b23.run found at https://developer.nvidia.com/embedded/jetpack-3_2_1, on a freshly installed ubuntu 16.4 laptop and let it reflash completely my TX1 devkit.

The devkit is equipped with a ov5693 camera that came with the devkit IIRC.

The kernel version is “Linux version 4.4.38-tegra (buildbrain@mobile-u64-638) (gcc version 4.8.5 (GCC) ) #1 SMP PREEMPT Thu Mar 1 20:44:58 PST 2018”

As in original message, the following pipeline works :

GST_DEBUG=1 gst-launch-1.0 -v nvcamerasrc ! nvjpegenc ! multifilesink max-files=5 location=/run/user/1001/%05d

Th following pipeline (the same one, but with nvivafilter added) crashes at startup

nvidia@tegra-ubuntu:~$ GST_DEBUG=1 gst-launch-1.0 -v nvcamerasrc ! nvivafilter ! nvjpegenc ! multifilesink max-files=5 location=/run/user/1001/%05d
Setting pipeline to PAUSED ...

Available Sensor modes :
2592 x 1944 FR=30.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
2592 x 1458 FR=30.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
1280 x 720 FR=120.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock

NvCameraSrc: Trying To Set Default Camera Resolution. Selected sensorModeIndex = 1 WxH = 2592x1458 FrameRate = 30[ 4609.990552] misc tegra_camera_ctrl: ISO BW req 2147483647 > 2550000 (max) capping to max
.000000 ...

[ 4610.004664] misc tegra_camera_ctrl: vi_v4l2_update_isobw: requested iso bw is larger than max
/GstPipeline:pipeline0/GstNvCameraSrc:nvcamerasrc0.GstPad:src: caps = "video/x-raw\(memory:NVMM\)\,\ width\=\(int\)640\,\ height\=\(int\)480\,\ framerate\=\(fraction\)30/1\,\ format\=\(string\)NV12"
/GstPipeline:pipeline0/Gstnvivafilter:nvivafilter0.GstPad:src: caps = "video/x-raw\(memory:NVMM\)\,\ width\=\(int\)640\,\ height\=\(int\)480\,\ framerate\=\(fraction\)30/1\,\ format\=\(string\)NV12"
/GstPipeline:pipeline0/GstNvJpegEnc:nvjpegenc0.GstPad:sink: caps = "video/x-raw\(memory:NVMM\)\,\ width\=\(int\)640\,\ height\=\(int\)480\,\ framerate\=\(fraction\)30/1\,\ format\=\(string\)NV12"
/GstPipeline:pipeline0/Gstnvivafilter:nvivafilter0.GstPad:sink: caps = "video/x-raw\(memory:NVMM\)\,\ width\=\(int\)640\,\ height\=\(int\)480\,\ framerate\=\(fraction\)30/1\,\ format\=\(string\)NV12"
Caught SIGSEGV
#0  0x0000007f79b93fe0 in __GI___poll (fds=0x5f01d0, nfds=2,
#1  0x0000007f79c953c4 in ?? () from /lib/aarch64-linux-gnu/libglib-2.0.so.0
#2  0x000000000049aaf0 in ?? ()
Spinning.  Please run 'gdb gst-launch-1.0 21038' to continue debugging, Ctrl-C to quit, or Ctrl-\ to dump core.

Can you provide me either a fix or an example of a pipeline using nvivafilter that works for you with the same setup ?

Hi AastaLLL,

have you been able to reproduce (and hence confirm) the bug described in comment #7 ?

do you plan to provide a fix ?

TIA

I also see this.
As workaround, you can try to convert to I420 before nvjpegenc:

gst-launch-1.0 -ev nvcamerasrc ! 'video/x-raw(memory:NVMM), format=NV12, width=640, height=480, framerate=30/1' ! nvivafilter ! 'video/x-raw(memory:NVMM), format=NV12' ! nvvidconv ! 'video/x-raw(memory:NVMM), format=I420' ! nvjpegenc ! multifilesink max-files=5 location=/run/user/1001/%05d

or use jpegenc:

gst-launch-1.0 -ev nvcamerasrc ! 'video/x-raw(memory:NVMM), format=NV12, width=640, height=480, framerate=30/1' ! nvivafilter ! 'video/x-raw(memory:NVMM), format=NV12' ! nvvidconv ! 'video/x-raw, format=NV12' ! jpegenc ! multifilesink max-files=5 location=/run/user/1001/%05d

Hi Honey_Patouceul

Thank you for the workaround.

PS: I like your pseudo

Hi AastaLLL,

will the bug described in comment #7 be fixed in a jetpack 3.2.2 or jetpack 3.3.1 release ?

Hi,

We recently found this issue is caused by the incompatible TensorRT cache.
Could you try to clean-up the cache and then launch the app again?

rm /home/nvidia/Model/IVA_secondary_carcolor_V1/*.cache
rm /home/nvidia/Model/IVASecondary_Make_V1/*.cache
rm /home/nvidia/Model/ivaSecondary_VehicleTypes_V1/*.cache
rm /home/nvidia/Model/ResNet_18/*.cache

Thanks.

Thanks AastaLLL,

Unfortunately, the clean-up of the cache does not fix the problem :

nvidia@tegra-ubuntu:~/macq/nvsample_cudaprocess$ find /home/nvidia/Model/ -name \*.cache | xargs rm
nvidia@tegra-ubuntu:~/macq/nvsample_cudaprocess$ GST_DEBUG=1 gst-launch-1.0 -v nvcamerasrc ! nvivafilter ! nvjpegenc !         multifilesink max-files=5 location=/run/user/1001/%05d
Setting pipeline to PAUSED ...

Available Sensor modes :
2592 x 1944 FR=30.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
2592 x 1458 FR=30.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
1280 x 720 FR=120.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...

NvCameraSrc: Trying To Set Default Camera Resolution. Selected sensorModeIndex = 1 WxH = 2592x1458 FrameRate = 30.000000 ...

New clock: GstSystemClock
/GstPipeline:pipeline0/GstNvCameraSrc:nvcamerasrc0.GstPad:src: caps = "video/x-raw\(memory:NVMM\)\,\ width\=\(int\)640\,\ height\=\(int\)480\,\ framerate\=\(fraction\)30/1\,\ format\=\(string\)NV12"
/GstPipeline:pipeline0/Gstnvivafilter:nvivafilter0.GstPad:src: caps = "video/x-raw\(memory:NVMM\)\,\ width\=\(int\)640\,\ height\=\(int\)480\,\ framerate\=\(fraction\)30/1\,\ format\=\(string\)NV12"
/GstPipeline:pipeline0/GstNvJpegEnc:nvjpegenc0.GstPad:sink: caps = "video/x-raw\(memory:NVMM\)\,\ width\=\(int\)640\,\ height\=\(int\)480\,\ framerate\=\(fraction\)30/1\,\ format\=\(string\)NV12"
/GstPipeline:pipeline0/Gstnvivafilter:nvivafilter0.GstPad:sink: caps = "video/x-raw\(memory:NVMM\)\,\ width\=\(int\)640\,\ height\=\(int\)480\,\ framerate\=\(fraction\)30/1\,\ format\=\(string\)NV12"
Caught SIGSEGV
#0  0x0000007f8710b030 in __GI___poll (fds=0x5ef700, nfds=2,
#1  0x0000007f8720c3ac in ?? () from /lib/aarch64-linux-gnu/libglib-2.0.so.0
#2  0x00000000005352d0 in ?? ()
Spinning.  Please run 'gdb gst-launch-1.0 5614' to continue debugging, Ctrl-C to quit, or Ctrl-\ to dump core.

Hi,

Could you follow the pipeline listed in this document.
It should work.

Thanks.

Hi,

yes that pipeline works :

gst-launch-1.0 nvcamerasrc fpsRange="30 30" ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! nvtee ! nvivafilter cuda-process=true customer-lib-name="libnvsample_cudaprocess.so" !  'video/x-raw(memory:NVMM), format=(string)NV12' ! nvoverlaysink -e

but :

  1. what does ‘nvtee’ do ? It does not seem to be a nvidia version of ‘tee’, and the info we get with
gst-inspect-1.0 nvtee

does not explain how to use it

  1. if I replace ‘nvoverlaysink’ by ‘nvjpegenc ! fakesink’, the pipeline crashes at startup with same SIGSEGV error in ‘__GI___poll’ as in my original post

For nvtee, please refer to source code of nvgstcatpure-1.0
https://developer.nvidia.com/embedded/dlc/sources-r2821

For jpeg encoding please try below pipeline:

gst-launch-1.0 nvcamerasrc fpsRange="30 30" num-buffers=30 ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! nvtee ! nvivafilter cuda-process=true customer-lib-name="libnvsample_cudaprocess.so" !  'video/x-raw(memory:NVMM), format=(string)NV12' ! <b>nvvidconv ! video/x-raw,format=I420 ! nvjpegenc ! multifilesink location=snap-%02d.jpg</b> -e