[Jetson Nano] deepstream-app crash after IR cut filter on IP camera

I was finally able to get a working config for the deepstream reference application for a HikVision IP camera’s RTSP feed after disabling tracker and tiled-display. The application was launched when the camera in question was in night mode. When turning on a light in the area to test the reference inference model, the application crashed.

I have been able to consistently reproduce the issue if the application is launched, and the camera IR cut filter is triggered either on or off. Looking at the logs with GST debug turned to 4, it appears the connection times out when the cut filter is triggered, and when GST attempts to reconnect, the nvv4l2decoder is unable to queue buffers to re-access the feed.

Any help would be appreciated.

0:00:31.456982884 12457   0x7f00038e80 WARN                 rtspsrc gstrtspsrc.c:3151:on_timeout:<source> source 6f47e1c0, stream 6f47e1c0 in session 1 timed out
0:00:31.457112574 12457   0x7f00038f20 INFO                    task gsttask.c:318:gst_task_func:<udpsrc4:src> Task resume from paused
0:00:31.457145179 12457   0x7ed0004e80 INFO                 basesrc gstbasesrc.c:2839:gst_base_src_loop:<udpsrc5> pausing after gst_base_src_get_range() = flushing
0:00:31.457158461 12457   0x7f00038f20 INFO                    task gsttask.c:316:gst_task_func:<udpsrc4:src> Task going to paused
0:00:31.457232109 12457   0x7ed0004e80 INFO                    task gsttask.c:316:gst_task_func:<udpsrc5:src> Task going to paused
0:00:31.457323309 12457   0x7ed0004e80 INFO                    task gsttask.c:318:gst_task_func:<udpsrc5:src> Task resume from paused
0:00:31.457370289 12457   0x7ed0004e80 INFO                 basesrc gstbasesrc.c:2839:gst_base_src_loop:<udpsrc5> pausing after gst_base_src_get_range() = eos
0:00:31.457552169 12457   0x7ed0004e80 INFO                    task gsttask.c:316:gst_task_func:<udpsrc5:src> Task going to paused
0:00:31.603001164 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:31.716354554 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:31.889212315 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:32.055872679 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:32.222467521 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:32.374757677 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:32.551055003 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:32.688068940 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:32.855427238 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:33.018744338 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:33.187637466 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:33.329875596 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:33.503299723 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:33.666631199 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:33.865287006 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:33.982283611 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:34.154470310 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:34.315250731 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:34.464092777 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:34.637692376 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:34.811319111 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:34.948599822 12457   0x7ed80024f0 INFO            videodecoder gstvideodecoder.c:1330:gst_video_decoder_sink_event_default:<nvv4l2decoder0> upstream tags: taglist, video-codec=(string)"H.264\ \(Main\ Profile\)", minimum-bitrate=(uint)2615807, maximum-bitrate=(uint)16215063, bitrate=(uint)6124302;
0:00:34.948778993 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:35.107929736 12457   0x7ed80024f0 INFO            videodecoder gstvideodecoder.c:1330:gst_video_decoder_sink_event_default:<nvv4l2decoder0> upstream tags: taglist, video-codec=(string)"H.264\ \(Main\ Profile\)", minimum-bitrate=(uint)1271761, maximum-bitrate=(uint)16215063, bitrate=(uint)6099029;
0:00:35.108089115 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
**PERF: 6.22 (6.09)	
0:00:35.270968494 12457   0x7ed80024f0 INFO            videodecoder gstvideodecoder.c:1330:gst_video_decoder_sink_event_default:<nvv4l2decoder0> upstream tags: taglist, video-codec=(string)"H.264\ \(Main\ Profile\)", minimum-bitrate=(uint)171593, maximum-bitrate=(uint)16215063, bitrate=(uint)6068316;
0:00:35.271125790 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:35.434907066 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:35.600740270 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:35.772269920 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:35.932362196 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:36.101949555 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:36.266383712 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:36.392632059 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:36.527615315 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:36.640502856 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:36.738013133 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:36.839535178 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:36.915718405 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:37.022812883 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:37.086260319 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:37.168617504 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:37.225569933 12457   0x7ed80024f0 INFO            videodecoder gstvideodecoder.c:1330:gst_video_decoder_sink_event_default:<nvv4l2decoder0> upstream tags: taglist, video-codec=(string)"H.264\ \(Main\ Profile\)", minimum-bitrate=(uint)171593, maximum-bitrate=(uint)16215063, bitrate=(uint)5932976;
0:00:37.225738427 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:37.350218916 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:37.471322341 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1474:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:sink> v4l2 provided buffer that is too big for the memory it was writing into.  v4l2 claims 64 bytes used but memory is only 0B.  This is probably a driver bug.
0:00:37.583122458 12457   0x7ed80024f0 INFO               GST_EVENT gstevent.c:814:gst_event_new_caps: creating caps event video/x-h264, stream-format=(string)avc, alignment=(string)au, codec_data=(buffer)014d0029ffe10031674d00299a6503c0113f2e02dc04040500000303e800005dc0e86000b5a80002d6a0bbcb8d0c0016b500005ad4177970a001000468ee3c80, level=(string)4.1, profile=(string)main
0:00:37.584050085 12457   0x7ed80024f0 INFO               h264parse gsth264parse.c:1793:gst_h264_parse_update_src_caps:<h264parse0> resolution changed 1920x1080
0:00:37.584094618 12457   0x7ed80024f0 INFO               h264parse gsth264parse.c:1817:gst_h264_parse_update_src_caps:<h264parse0> pixel aspect ratio has been changed 1/1
0:00:37.584130504 12457   0x7ed80024f0 INFO               h264parse gsth264parse.c:1853:gst_h264_parse_update_src_caps:<h264parse0> PAR 1/1
0:00:37.584180870 12457   0x7ed80024f0 INFO               baseparse gstbaseparse.c:4004:gst_base_parse_set_latency:<h264parse0> min/max latency 0:00:00.083333333, 0:00:00.083333333
0:00:37.584792031 12457   0x7ed80024f0 INFO               GST_EVENT gstevent.c:814:gst_event_new_caps: creating caps event video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)4.1, profile=(string)main, pixel-aspect-ratio=(fraction)1/1, width=(int)1920, height=(int)1080, framerate=(fraction)12/1, interlace-mode=(string)progressive, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true
0:00:37.585292877 12457   0x7ed80024f0 INFO           basetransform gstbasetransform.c:1308:gst_base_transform_setcaps:<capsfilter0> reuse caps
0:00:37.585367983 12457   0x7ed80024f0 INFO               GST_EVENT gstevent.c:814:gst_event_new_caps: creating caps event video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)4.1, profile=(string)main, pixel-aspect-ratio=(fraction)1/1, width=(int)1920, height=(int)1080, framerate=(fraction)12/1, interlace-mode=(string)progressive, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true
0:00:37.586780154 12457   0x7ebc7da8a0 INFO                    task gsttask.c:316:gst_task_func:<nvv4l2decoder0:src> Task going to paused
0:00:37.586852656 12457   0x7ebc7da8a0 INFO                    task gsttask.c:318:gst_task_func:<nvv4l2decoder0:src> Task resume from paused
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
0:00:37.592673168 12457   0x7ed80024f0 INFO                    v4l2 gstv4l2object.c:3146:gst_v4l2_object_setup_pool:<nvv4l2decoder0:sink> accessing buffers via mode 3
0:00:37.592909893 12457   0x7ed80024f0 INFO          v4l2bufferpool gstv4l2bufferpool.c:814:gst_v4l2_buffer_pool_set_config:<nvv4l2decoder0:pool:sink> increasing minimum buffers to 2
0:00:37.592975363 12457   0x7ed80024f0 INFO          v4l2bufferpool gstv4l2bufferpool.c:821:gst_v4l2_buffer_pool_set_config:<nvv4l2decoder0:pool:sink> increasing minimum buffers to 10
0:00:37.593015677 12457   0x7ed80024f0 INFO          v4l2bufferpool gstv4l2bufferpool.c:827:gst_v4l2_buffer_pool_set_config:<nvv4l2decoder0:pool:sink> reducing maximum buffers to 32
0:00:37.593052344 12457   0x7ed80024f0 INFO          v4l2bufferpool gstv4l2bufferpool.c:838:gst_v4l2_buffer_pool_set_config:<nvv4l2decoder0:pool:sink> can't allocate, setting maximum to minimum
0:00:37.700156196 12457   0x7ed80024f0 WARN                    v4l2 gstv4l2object.c:4408:gst_v4l2_object_probe_caps:<nvv4l2decoder0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:37.700221511 12457   0x7ed80024f0 WARN                    v4l2 gstv4l2object.c:2370:gst_v4l2_object_add_interlace_mode:0x7ebc0ede20 Failed to determine interlace mode
0:00:37.700284116 12457   0x7ed80024f0 INFO                    v4l2 gstv4l2object.c:4464:gst_v4l2_object_probe_caps:<nvv4l2decoder0:src> probed caps: video/x-raw, format=(string)NV12, width=(int)[ 48, 4096 ], height=(int)[ 48, 4096 ], framerate=(fraction)[ 0/1, 2147483647/1 ]
0:00:37.700644073 12457   0x7ed80024f0 INFO               GST_EVENT gstevent.c:814:gst_event_new_caps: creating caps event video/x-raw(memory:NVMM), format=(string)NV12, width=(int)1920, height=(int)1080, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, chroma-site=(string)mpeg2, colorimetry=(string)bt709, framerate=(fraction)12/1
0:00:37.701680402 12457     0x3593d680 INFO               GST_EVENT gstevent.c:814:gst_event_new_caps: creating caps event video/x-raw(memory:NVMM), format=(string)NV12, width=(int)1920, height=(int)1080, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, framerate=(fraction)12/1, batch-size=(int)1, num-surfaces-per-frame=(int)1
0:00:37.701936346 12457   0x7ed80024f0 INFO                    v4l2 gstv4l2object.c:3146:gst_v4l2_object_setup_pool:<nvv4l2decoder0:src> accessing buffers via mode 2
0:00:37.702089110 12457   0x7ed80024f0 INFO          v4l2bufferpool gstv4l2bufferpool.c:814:gst_v4l2_buffer_pool_set_config:<nvv4l2decoder0:pool:src> increasing minimum buffers to 2
0:00:37.702119736 12457   0x7ed80024f0 INFO          v4l2bufferpool gstv4l2bufferpool.c:821:gst_v4l2_buffer_pool_set_config:<nvv4l2decoder0:pool:src> increasing minimum buffers to 8
0:00:37.702143226 12457   0x7ed80024f0 INFO          v4l2bufferpool gstv4l2bufferpool.c:827:gst_v4l2_buffer_pool_set_config:<nvv4l2decoder0:pool:src> reducing maximum buffers to 32
0:00:37.702192915 12457   0x7ed80024f0 INFO          v4l2bufferpool gstv4l2bufferpool.c:838:gst_v4l2_buffer_pool_set_config:<nvv4l2decoder0:pool:src> can't allocate, setting maximum to minimum
0:00:37.702216249 12457   0x7ed80024f0 INFO          v4l2bufferpool gstv4l2bufferpool.c:843:gst_v4l2_buffer_pool_set_config:<nvv4l2decoder0:pool:src> adding needed video meta
0:00:37.702279896 12457   0x7ed80024f0 INFO          v4l2bufferpool gstv4l2bufferpool.c:827:gst_v4l2_buffer_pool_set_config:<nvv4l2decoder0:pool:src> reducing maximum buffers to 32
0:00:37.702301303 12457   0x7ed80024f0 INFO          v4l2bufferpool gstv4l2bufferpool.c:838:gst_v4l2_buffer_pool_set_config:<nvv4l2decoder0:pool:src> can't allocate, setting maximum to minimum
0:00:37.702364221 12457     0x3593d6d0 INFO               GST_EVENT gstevent.c:814:gst_event_new_caps: creating caps event video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, framerate=(fraction)12/1, batch-size=(int)1, num-surfaces-per-frame=(int)1, format=(string)RGBA
0:00:37.702408337 12457   0x7ed80024f0 WARN            v4l2videodec gstv4l2videodec.c:1433:gst_v4l2_video_dec_decide_allocation:<nvv4l2decoder0> Duration invalid, not setting latency
0:00:37.702579122 12457     0x3593ced0 INFO           basetransform gstbasetransform.c:1308:gst_base_transform_setcaps:<nvosd0> reuse caps
0:00:37.702673968 12457     0x3593ced0 INFO               GST_EVENT gstevent.c:814:gst_event_new_caps: creating caps event video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, framerate=(fraction)12/1, batch-size=(int)1, num-surfaces-per-frame=(int)1, format=(string)RGBA
0:00:37.706828188 12457   0x7ed80024f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1054:gst_v4l2_buffer_pool_start:<nvv4l2decoder0:pool:src> Uncertain or not enough buffers, enabling copy threshold
0:00:37.706988869 12457   0x7ed80024f0 ERROR          v4l2allocator gstv4l2allocator.c:1358:gst_v4l2_allocator_qbuf:<nvv4l2decoder0:pool:src:allocator> failed queueing buffer 1: Device or resource busy
0:00:37.707015797 12457   0x7ed80024f0 ERROR         v4l2bufferpool gstv4l2bufferpool.c:1379:gst_v4l2_buffer_pool_qbuf:<nvv4l2decoder0:pool:src> could not queue a buffer 1
0:00:37.707054808 12457   0x7ed80024f0 ERROR          v4l2allocator gstv4l2allocator.c:1358:gst_v4l2_allocator_qbuf:<nvv4l2decoder0:pool:src:allocator> failed queueing buffer 2: Device or resource busy
0:00:37.707072778 12457   0x7ed80024f0 ERROR         v4l2bufferpool gstv4l2bufferpool.c:1379:gst_v4l2_buffer_pool_qbuf:<nvv4l2decoder0:pool:src> could not queue a buffer 2
0:00:37.707158509 12457   0x7ed80024f0 ERROR          v4l2allocator gstv4l2allocator.c:1358:gst_v4l2_allocator_qbuf:<nvv4l2decoder0:pool:src:allocator> failed queueing buffer 8: Device or resource busy
0:00:37.707177832 12457   0x7ed80024f0 ERROR         v4l2bufferpool gstv4l2bufferpool.c:1379:gst_v4l2_buffer_pool_qbuf:<nvv4l2decoder0:pool:src> could not queue a buffer 8
0:00:37.707424870 12457   0x7ed80024f0 INFO                    task gsttask.c:457:gst_task_set_lock: setting stream lock 0x7eb4007ba0 on task 0x7eb4012830
0:00:37.707453568 12457   0x7ed80024f0 INFO                GST_PADS gstpad.c:6154:gst_pad_start_task:<nvv4l2decoder0:src> created task 0x7eb4012830
0:00:37.951543173 12457   0x7ebc7da8a0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1518:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:src> Driver should never set v4l2_buffer.field to ANY
0:00:37.951859118 12457   0x7ebc7da8a0 INFO                    task gsttask.c:316:gst_task_func:<nvv4l2decoder0:src> Task going to paused

Hi hunterjm,

Which platform you’re using? Nano or T4?

Thanks

I am using the Jetson Nano. Sorry I forgot to include that initially!

Hi,
Do you have other IP cameras to try? Probably it is specific to the HikVision camera?

I have a VivoTek IP camera I can try using this evening. Is my initial analysis correct in that it does not appear to be gracefully handling a stream disconnect/reconnect?

Following up here, the same crash does not occur when the Vivotek IP camera switches day/night. I also do not see the same stream timeout error. I went back to the basics, and created a gstreamer pipeline command, with which I get the following output as the IR cut filter is flipped on the HikVision:

$ gst-launch-1.0 rtspsrc location=rtsp://192.168.x.x:554 ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvideoconvert ! "video/x-raw(memory:NVMM),format=RGBA" ! nvegltransform ! nveglglessink sync=False
Setting pipeline to PAUSED ...

Using winsys: x11 
Opening in BLOCKING MODE 
Pipeline is live and does not need PREROLL ...
Got context from element 'eglglessink0': gst.egl.EGLDisplay=context, display=(GstEGLDisplay)NULL;
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://192.168.x.x:554
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (request) SETUP stream 1
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 

(gst-launch-1.0:17985): GStreamer-CRITICAL **: 21:33:18.574: gst_mini_object_unref: assertion 'mini_object != NULL' failed
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
Caught SIGSEGV
#0  0x0000007f80342048 in __GI___poll (fds=0x5597c83b50, nfds=547612848752, timeout=<optimized out>) at ../sysdeps/unix/sysv/linux/poll.c:41
#1  0x0000007f8044ee40 in  () at /usr/lib/aarch64-linux-gnu/libglib-2.0.so.0
#2  0x00000055979c03a0 in  ()

Any pointers where to look further would be appreciated!

Hi,
It looks specific to HikVision IP cameras. Can the pipeline be launched with avdec_h264 and omxh264dec? We don’t have the camera. Need your help to collect more information.

Dane, it definitely looks specific to the nvv4l2decoder. The following gst-launch command works without issue, even when flipping the IR cut filter on and off multiple times. I am happy to provide as much information as you need. Let me know what else would be helpful.

$ gst-launch-1.0 rtspsrc location=rtsp://192.168.x.x:554 ! rtph264depay ! h264parse ! omxh264dec ! nveglglessink

After continuing to test and play with camera settings and different camera models, I ran into a configuration with the same model camera that did not have the issue when switching between day/night. After comparing the settings on the IP camera, I determined the main difference was the exposure setting. The camera that was having issues had exposure set at 1/6 while the one that worked correctly was set to 1/25.

After switching the exposure to 1/25, I was able to trigger day/night by turning lights on/off without deepstream crashing. Since omxh264dec did not have the same issue even when exposure was set to 1/6, I do still believe the issue lies within nvv4l2decoder, and it might warrant further investigation since longer exposure times in night mode are common to increase the brightness of the image.

Regardless, since I have determined the source of the issue, I will mark this post as the accepted answer.

1 Like