VIDIOC_DQBUF blocks

Hi all,

Trying to make use of V4l2. I’m opening the device with O_NONBLOCK, as instructed in the sample source files:

/**
     * Creates a new V4L2 Video Decoder object named \a name.
     *
     * This method internally calls \c v4l2_open on the decoder dev node
     * \c "/dev/nvhost-nvdec" and checks for \c V4L2_CAP_VIDEO_M2M_MPLANE
     * capability on the device. This method allows the caller to specify
     * additional flags with which the device should be opened.
     *
     * The device is opened in blocking mode, which can be modified by passing
     * the @a O_NONBLOCK flag to this method.
     *
     * @returns Reference to the newly created decoder object else \a NULL in
     *          case of failure during initialization.
     */

My code:

_fd = v4l2_open(_settings._devicePath.c_str(), O_NONBLOCK | O_RDWR);

However, when calling

v4l2_ioctl(..., VIDIOC_DQBUF, ...)

it blocks indefinitely when there are no buffers to dequeue.

I’m using latest R28.2.1 of JetPack, and also fully upgraded the devel board with the JetPack provided wizard

Update:

Executing vidioc_dqevent ioctl sometimes results in EAGAIN, regardless of the blocking mode set on the device fd

Hi andrei2hqw6,
We suggest you refer to tegra_multimedia_api\samples\01_video_encode and use NvVideoEncoder class.

Hi DaneLLL.

The provided sample code works as advertised, but once we dig bellow the surface, we run into some out of ordinary things. We can not make use of the sample code directly, due to more technical reasons/requirements we have. We do inspire from it and we did manage to create work arounds for the points below. However, that is very far from ideal, and we would like that fixed. If that’s not possible, we would like to at least confirm the work arounds we have in place. Or perhaps we are really doing something very wrong. Here are the points:

  1. The call is made here: /samples/common/classes/NvV4l2ElementPlane.cpp:126. The provided sample app did not create the decoder device FD in non-blocking mode, yet it has EAGAIN treatment at 155. To make matters complicated that branch is sometimes executed after sending EOS, sometimes is not. Sometimes we get a buffer with bytesused == 0, sometimes it blocks completely.

  2. There is a severe race in the internal library loaded by libv4l2. There are cases where after sending EOS on OUTPUT plane, the NvV4l2ElementPlane.cpp:126 never returns. Attached with a debugger and it is permanently blocked on waiting a condition variable. Also, the stack trace has thousands upon thousands frames. I’m forced to do pthread_cancel after a complicated logic to wait a period of time. It becomes even more complicated because the call to pthread_cancel must be made from a different thread too, not to block our calling thread. At the point where the pthread_cancel is made, all bets are off: memory leaks could occur, as the killed thread may or may not have a way to cleanup.

  3. 2 or 3 out of 10 times, VIDIOC_DQBUF results in EAGAIN directly, without a buffer with flags set on V4L2_BUF_FLAG_LAST. That’s when the things described at point 2 are happening.

Update: Here are some logs:

Failed to query video capabilities: Inappropriate ioctl for device
Failed to query video capabilities: Inappropriate ioctl for device
NvMMLiteOpen : Block : BlockType = 261 
TVMR: NvMMLiteTVMRDecBlockOpen: 7647: NvMMLiteBlockOpen 
NvMMLiteBlockCreate : Block : BlockType = 261 
TVMR: cbBeginSequence: 1179: BeginSequence  1280x720, bVPR = 0
TVMR: LowCorner Frequency = 0 
TVMR: cbBeginSequence: 1529: DecodeBuffers = 6, pnvsi->eCodec = 4, codec = 0 
TVMR: cbBeginSequence: 1600: Display Resolution : (1280x720) 
TVMR: cbBeginSequence: 1601: Display Aspect Ratio : (1280x720) 
TVMR: cbBeginSequence: 1669: ColorFormat : 5 
TVMR: cbBeginSequence:1674 ColorSpace = NvColorSpace_YCbCr709_ER
TVMR: cbBeginSequence: 1809: SurfaceLayout = 3
TVMR: cbBeginSequence: 1902: NumOfSurfaces = 13, InteraceStream = 0, InterlaceEnabled = 0, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1, BitDepthForSurface = 8 LumaBitDepth = 8, ChromaBitDepth = 8, ChromaFormat = 5
TVMR: cbBeginSequence: 1904: BeginSequence  ColorPrimaries = 1, TransferCharacteristics = 1, MatrixCoefficients = 1
******* Dequeue thread V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE VIDIOC_DQBUF execution completed with EAGAIN. Flags are 0x0; pid: 3947
******* Dequeue thread V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE VIDIOC_DQBUF execution completed with EAGAIN. Flags are 0x0; pid: 3947
TVMR: FrameRate = 30 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: NvMMLiteTVMRDecDoWork: 6531: NVMMLITE_TVMR: EOS detected
TVMR: TVMRBufferProcessing: 5486: Processing of EOS 
TVMR: TVMRBufferProcessing: 5563: Processing of EOS Done
******* Dequeue thread V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE VIDIOC_DQBUF execution completed with EAGAIN. Flags are 0x0; pid: 3947
******* thread v4l2_out_buf_dq will be terminated by force
TVMR: TVMRFrameStatusReporting: 6132: Closing TVMR Frame Status Thread -------------
TVMR: TVMRVPRFloorSizeSettingThread: 5942: Closing TVMRVPRFloorSizeSettingThread -------------
TVMR: TVMRFrameDelivery: 5982: Closing TVMR Frame Delivery Thread -------------
TVMR: NvMMLiteTVMRDecBlockClose: 7815: Done 
 decoder I stats: 1000/12532674
 decoder O stats: 1000/1572864000
rescaler I stats: 0/0
rescaler O stats: 0/0
 encoder I stats: 0/0
 encoder O stats: 0/0

Please notice that we get EAGAIN for the startup too, before queuing any buffers on any of the queues

This is same identical app ran again, but this time the NVIDIA native libs behind libv4l2 handled the situation properly. I did not modify anything, just re-run the app. Also, I get a final real decoded frame with V4L2_BUF_FLAG_LAST set on it.

Failed to query video capabilities: Inappropriate ioctl for device
Failed to query video capabilities: Inappropriate ioctl for device
NvMMLiteOpen : Block : BlockType = 261 
TVMR: NvMMLiteTVMRDecBlockOpen: 7647: NvMMLiteBlockOpen 
NvMMLiteBlockCreate : Block : BlockType = 261 
TVMR: cbBeginSequence: 1179: BeginSequence  1280x720, bVPR = 0
TVMR: LowCorner Frequency = 0 
TVMR: cbBeginSequence: 1529: DecodeBuffers = 6, pnvsi->eCodec = 4, codec = 0 
TVMR: cbBeginSequence: 1600: Display Resolution : (1280x720) 
TVMR: cbBeginSequence: 1601: Display Aspect Ratio : (1280x720) 
TVMR: cbBeginSequence: 1669: ColorFormat : 5 
TVMR: cbBeginSequence:1674 ColorSpace = NvColorSpace_YCbCr709_ER
TVMR: cbBeginSequence: 1809: SurfaceLayout = 3
TVMR: cbBeginSequence: 1902: NumOfSurfaces = 13, InteraceStream = 0, InterlaceEnabled = 0, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1, BitDepthForSurface = 8 LumaBitDepth = 8, ChromaBitDepth = 8, ChromaFormat = 5
TVMR: cbBeginSequence: 1904: BeginSequence  ColorPrimaries = 1, TransferCharacteristics = 1, MatrixCoefficients = 1
******* Dequeue thread V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE VIDIOC_DQBUF execution completed with EAGAIN. Flags are 0x0; pid: 4097
******* Dequeue thread V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE VIDIOC_DQBUF execution completed with EAGAIN. Flags are 0x0; pid: 4097
TVMR: FrameRate = 30 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: NvMMLiteTVMRDecDoWork: 6531: NVMMLITE_TVMR: EOS detected
TVMR: TVMRBufferProcessing: 5486: Processing of EOS 
TVMR: TVMRBufferProcessing: 5563: Processing of EOS Done
TVMR: TVMRFrameStatusReporting: 6132: Closing TVMR Frame Status Thread -------------
TVMR: TVMRVPRFloorSizeSettingThread: 5942: Closing TVMRVPRFloorSizeSettingThread -------------
TVMR: TVMRFrameDelivery: 5982: Closing TVMR Frame Delivery Thread -------------
TVMR: NvMMLiteTVMRDecBlockClose: 7815: Done 
 decoder I stats: 1000/12532674
 decoder O stats: 1000/1572864000
rescaler I stats: 0/0
rescaler O stats: 0/0
 encoder I stats: 0/0
 encoder O stats: 0/0

This is a typical deadlock that happens, and we are forced to do pthread_cancel. It has thousands of frames in the stack. Did not show all, but is quite clear that something went wrong.

The output from app:

Failed to query video capabilities: Inappropriate ioctl for device
Failed to query video capabilities: Inappropriate ioctl for device
NvMMLiteOpen : Block : BlockType = 261 
TVMR: NvMMLiteTVMRDecBlockOpen: 7647: NvMMLiteBlockOpen 
NvMMLiteBlockCreate : Block : BlockType = 261 
TVMR: cbBeginSequence: 1179: BeginSequence  1280x720, bVPR = 0
TVMR: LowCorner Frequency = 0 
TVMR: cbBeginSequence: 1529: DecodeBuffers = 6, pnvsi->eCodec = 4, codec = 0 
TVMR: cbBeginSequence: 1600: Display Resolution : (1280x720) 
TVMR: cbBeginSequence: 1601: Display Aspect Ratio : (1280x720) 
TVMR: cbBeginSequence: 1669: ColorFormat : 5 
TVMR: cbBeginSequence:1674 ColorSpace = NvColorSpace_YCbCr709_ER
TVMR: cbBeginSequence: 1809: SurfaceLayout = 3
TVMR: cbBeginSequence: 1902: NumOfSurfaces = 13, InteraceStream = 0, InterlaceEnabled = 0, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1, BitDepthForSurface = 8 LumaBitDepth = 8, ChromaBitDepth = 8, ChromaFormat = 5
TVMR: cbBeginSequence: 1904: BeginSequence  ColorPrimaries = 1, TransferCharacteristics = 1, MatrixCoefficients = 1
******* Dequeue thread V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE VIDIOC_DQBUF execution completed with EAGAIN. Flags are 0x0; pid: 4664
******* Dequeue thread V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE VIDIOC_DQBUF execution completed with EAGAIN. Flags are 0x0; pid: 4664
TVMR: FrameRate = 30 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
Dec: 649; Res: 0; Enc: 0
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: FrameRate = 30.000030 
TVMR: NvMMLiteTVMRDecDoWork: 6531: NVMMLITE_TVMR: EOS detected
TVMR: TVMRBufferProcessing: 5486: Processing of EOS 
TVMR: TVMRBufferProcessing: 5563: Processing of EOS Done
******* Dequeue thread V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE VIDIOC_DQBUF execution completed with EAGAIN. Flags are 0x0; pid: 4664

The GDB attached:

shiretu@tegra-ubuntu:~$ gdb -p 4664
GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.5) 7.11.1
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "aarch64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word".
Attaching to process 4664
[New LWP 4665]
[New LWP 4666]
[New LWP 4667]
[New LWP 4668]
[New LWP 4669]
[New LWP 4671]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/aarch64-linux-gnu/libthread_db.so.1".
0x0000007f9c4a6d94 in nanosleep () at ../sysdeps/unix/syscall-template.S:86
86	../sysdeps/unix/syscall-template.S: No such file or directory.
(gdb) info threads 
  Id   Target Id         Frame 
* 1    Thread 0x7f9c867000 (LWP 4664) "ubnt_avcodec_te" 0x0000007f9c4a6d94 in nanosleep () at ../sysdeps/unix/syscall-template.S:86
  2    Thread 0x7f974fa1e0 (LWP 4665) "ubnt_avcodec_te" 0x0000007f9c7f330c in __pthread_cond_wait (cond=0xc3cfd0, mutex=0xc3cfa0) at pthread_cond_wait.c:186
  3    Thread 0x7f96cfa1e0 (LWP 4666) "ubnt_avcodec_te" 0x0000007f9c7f330c in __pthread_cond_wait (cond=0xc3d760, mutex=0xc3d730) at pthread_cond_wait.c:186
  4    Thread 0x7f964fa1e0 (LWP 4667) "ubnt_avcodec_te" 0x0000007f9c7f330c in __pthread_cond_wait (cond=0xc3d7d0, mutex=0xc3d7a0) at pthread_cond_wait.c:186
  5    Thread 0x7f95cfa1e0 (LWP 4668) "ubnt_avcodec_te" 0x0000007f9c7f330c in __pthread_cond_wait (cond=0xc3dae0, mutex=0xc3dab0) at pthread_cond_wait.c:186
  6    Thread 0x7f950fa1e0 (LWP 4669) "ubnt_avcodec_te" 0x0000007f9c7f330c in __pthread_cond_wait (cond=0xc461b0, mutex=0xc46180) at pthread_cond_wait.c:186
  7    Thread 0x7f8e5ff1e0 (LWP 4671) "v4l2_out_buf_dq" 0x0000007f9c7f330c in __pthread_cond_wait (cond=0xc46140, mutex=0xc46110) at pthread_cond_wait.c:186
(gdb) bt
#0  0x0000007f9c4a6d94 in nanosleep () at ../sysdeps/unix/syscall-template.S:86
#1  0x0000007f9c4cc258 in usleep (useconds=<optimized out>) at ../sysdeps/posix/usleep.c:32
#2  0x00000000005b4718 in ubnt::avcodec::internal::v4l2::V4l2Device::WaitForEOS (this=0xc168a0) at /home/shiretu/work/evostream/sources/ubnt_avcodec/src/v4l2/v4l2device.cpp:263
#3  0x00000000005c2238 in ubnt::avcodec::internal::v4l2::V4l2Decoder::Flush (this=0xc14f80) at /home/shiretu/work/evostream/sources/ubnt_avcodec/src/v4l2/v4l2decoder.cpp:95
#4  0x00000000005dbdfc in TestLibavComponents::Run (this=0x7fc1322fa0) at /home/shiretu/work/evostream/sources/ubnt_avcodec_tests/src/testlibavcomponents.cpp:145
#5  0x00000000005d7c60 in Test::Execute<TestLibavComponents> (argc=2, ppArgv=0x7fc13231a8) at /home/shiretu/work/evostream/sources/ubnt_avcodec_tests/include/test.h:32
#6  0x00000000005d7bd4 in main (argc=2, ppArgv=0x7fc13231a8) at /home/shiretu/work/evostream/sources/ubnt_avcodec_tests/src/main.cpp:16
(gdb) thread 7
[Switching to thread 7 (Thread 0x7f8e5ff1e0 (LWP 4671))]
#0  0x0000007f9c7f330c in __pthread_cond_wait (cond=0xc46140, mutex=0xc46110) at pthread_cond_wait.c:186
186	pthread_cond_wait.c: No such file or directory.
(gdb) bt
#0  0x0000007f9c7f330c in __pthread_cond_wait (cond=0xc46140, mutex=0xc46110) at pthread_cond_wait.c:186
#1  0x0000007f9c299080 in ?? () from /usr/lib/aarch64-linux-gnu/tegra/libnvos.so
#2  0x0000007f9be4d514 in TegraV4L2_Poll_CPlane () from /usr/lib/aarch64-linux-gnu/tegra/libtegrav4l2.so
#3  0x0000007f9bea0f80 in plugin_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l/plugins/libv4l2_nvvideocodec.so
#4  0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#5  0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#6  0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#7  0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#8  0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#9  0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#10 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#11 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#12 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#13 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#14 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#15 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#16 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#17 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#18 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#19 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#20 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#21 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#22 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#23 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#24 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#25 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#26 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#27 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#28 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#29 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#30 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#31 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#32 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#33 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#34 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#35 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#36 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#37 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#38 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#39 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#40 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#41 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#42 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#43 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#44 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#45 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#46 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#47 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#48 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#49 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#50 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#51 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#52 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#53 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#54 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#55 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#56 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#57 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#58 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#59 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#60 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#61 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#62 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#63 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#64 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#65 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#66 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#67 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#68 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#69 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#70 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#71 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#72 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#73 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
#74 0x0000007f9c7cc6d0 in v4l2_ioctl () from /usr/lib/aarch64-linux-gnu/libv4l2.so.0
---Type <return> to continue, or q <return> to quit---

Hi andrei2hqw6,
In general, we suggest users use NvVideoEncoder or NvVideoDecoder. For your case, it requires further support. Please contact your region salesperson so that we can have further cooperation.

Hi DaneLLL,

I am using NvVideoDecoder and I have seen similar inconsistent behavior and unreliability as andrei2hqw6 mentions. The NvVideoDecoder implementation is quite a thin wrapper over the V4L2 implementation and I don’t see how using NvVideoDecoder is supposed to solve any of the problems he mentions.

Thanks,

Chris Richardson

1 Like

Hi Chris,
Can the issue be reproduced with 00_video_decode? If yes, please help provide steps so that we can do further check.

Hi DaneLLL,

Yes, the issue(s) can be replicated with a bit modified sample code. It is not happening with unmodified sources because of pure luck, and an almost-always-winning race between main thread and the capture loop thread. 99.9999% of cases, the main thread wins the race and everything seems fine, when in fact is not fine at all.

patch: https://gist.github.com/shiretu/221193b76f5f1f26a08975042e3000c2
running output: https://gist.github.com/shiretu/5f0572b9777df11c4ce3bb3328823d07
debugger attached: https://gist.github.com/shiretu/a26da0e596ad5d936ff9b44ef3f28d5c

We will check and update.

The sleep is not put in correct place. Please refer to below patch

@@ -1159,6 +1159,8 @@ main(int argc, char *argv[])
                 current_file = current_file % ctx.file_count;
                 continue;
             }
+            // delay EOS
+            sleep(30);
         }
         // It is necessary to queue an empty buffer to signal EOS to the decod
er
         // i.e. set v4l2_buf.m.planes[0].bytesused = 0 and queue the buffer

Hi DaneLLL,

I’m very very confused.

What do you mean the sleep is not put in a correct place? What makes your version correct?

Your sleep is after the EOS. Once you send EOS, decoder is flushing and you should not keep calling dqBuffer when all buffers are flushed out.

That’s not what V4L2 specs say:
https://linuxtv.org/downloads/v4l-dvb-apis/uapi/v4l/vidioc-qbuf.html#return-value

EINVAL
The buffer type is not supported, or the index is out of bounds, or no buffers have been allocated yet, or the userptr or length are invalid.

The sample implementation is avoiding a bug with very complicated buffer machinery, when in turn it should simply error out.

This is very important. What you call “sample code” is no longer sample code. Is more like use this bag of work-arounds and treat the sample code as some sort of low quality SDK. Which is very far from being OK

1 Like

Hi shiretu,
We think the current implementation is good for most cases, although it may not precisely meet v4l2 specs. We would suggest you not to call dqBuffer after you send EOS, but if you do need the precise SW implementation, please contact your region salesperson. Let’s go over business review to have further cooperation.

Hi DaneLLL,

On the short term, I will account the frames in and the frames out. However, please consider patching the v4l2 plugins to at least don’t enter pthread_cond_wait when EOS was encountered, or reliably signal the V4L2_BUF_FLAG_LAST. Either of those will literally eliminate the buffers counting complexity from app layers. Besides, it is crystal clear from the logs of the plugin itself that is fully aware of when EOS was delivered on input. It is probably boiling down to an if statement just before the pthread call.

Best regards,
Andrei

Hi DaneLLL,

I would beg to differ that the current behavior is good enough for most use cases. I spent a lot of time debugging hangs, no EOS output and inconsistent shutdown behavior related to NvDecoder, naively assuming its behavior was correct and race free and that I must have been doing something wrong. In addition to that, the fact that it doesn’t actually support O_NONBLOCK is frustrating. I’m still not sure I have a solution that is 100% reliable to shutdown and restart many times due to these races and inconsistent behavior.

My customers expect the systems I am working on to run 24/7 for a year or more without rebooting and I need all of this to be rock solid. Please consider making enhancements to NvDecoder to work a bit more properly and in-line with how V4L2 devices are supposed to work.

Thanks,

Chris Richardson

1 Like