I am getting corrupted frames from 18MP AR1820 USB3 camera most of the time at all resolutions. Less frequent at low res. This is Bayer camera, I am using V4L2_PIX_FMT_YUYV to get raw buffers. The problem happens in USB stack. Not reproducible on Windows or Linux x86 host (Ubuntu 14.04 LTS). Here is TX1 kernel log for bad case.
Error message: uvcvideo: Dropping payload (out of sync).
You’ll probably want to provide “lsusb -t” output. If you run lsusb with no arguments you’ll also see an ID, and can print verbose USB info (where ####:#### are numbers of ID from plain lsusb) for the specific device via:
The USB transfer mode is bulk. This would be appropriate for stop motion cameras, but not full motion (full motion would use isochronous). Bulk transfer is essentially stop-and-go, and depends on the camera saving or buffering data while waiting for the ok to continue sending. Is the camera sending individual snapshots, or is it full motion?
If the USB Video Class expects a constant bandwidth available, bulk transfer stops could result in dropping data or buffer overflows/underflows.
I don’t know enough about V4L2, but how large is an individual frame? It doesn’t seem likely that USB would have to halt a bulk transfer unless a large amount of data is transferred, or unless something else is competing for USB bandwidth. What I’m curious about, and cannot answer, is whether V4L2 itself has any behavior difference between a bulk transfer device and one with isochronous transfer. Obviously much of V4L2 was designed around full motion devices and thus isochronous behavior. I do not know what would happen within V4L2 if a bulk transfer pauses…speculating, perhaps V4L2 expects isochronous behavior.
One thing to simplify testing is be sure USB autosuspend is disabled:
I doubt USB itself would be an issue with bulk versus isochronous modes with any class beneath the protocols. The consumer of the USB data would be more likely to be sensitive to this. Being part of a class within USB means there are probably some standardized functions to be expected, but the application consuming the device does not necessarily have to be aware of USB transfer mode…certainly the commands to access the device would be known to the consuming program.
I can’t say for sure, but at 18MP, it may be possible a single frame of uncompressed color to require being interrupted and restarted in a similar way as a hard drive being read when other USB devices also want to use the USB pipe.
I’m unsure of how to do it, but it seems there is a need to determine if bulk transfer stop/start is losing data, or if it is the end application expecting the data not liking to be interrupted. Or, it could of course be something completely different, but the behavior of the issue decreasing at low resolution tends to increase the odds of this.
I would like to suggest you below steps to check if this issue can be improved,
Increase TRBS_PER_SEGMENT to 256
diff --git a/drivers/usb/host/xhci.h b/drivers/usb/host/xhci.h
index 8820871b..0322430 100644
--- a/drivers/usb/host/xhci.h
+++ b/drivers/usb/host/xhci.h
@@ -1238,7 +1238,7 @@ union xhci_trb {
* since the command ring is 64-byte aligned.
* It must also be greater than 16.
*/
-#define TRBS_PER_SEGMENT 64
+#define TRBS_PER_SEGMENT 256
/* Allow two commands + a link TRB, along with any reserved command TRBs */
#define MAX_RSVD_CMD_TRBS (TRBS_PER_SEGMENT - 3)
#define TRB_SEGMENT_SIZE (TRBS_PER_SEGMENT*16)
Increase UVC_URBS and UVC_MAX_PACKETS
diff --git a/drivers/media/usb/uvc/uvcvideo.h b/drivers/media/usb/uvc/uvcvideo.h
index af505fd..e5f36ef 100644
--- a/drivers/media/usb/uvc/uvcvideo.h
+++ b/drivers/media/usb/uvc/uvcvideo.h
@@ -112,9 +112,9 @@
#define DRIVER_VERSION "1.1.1"
/* Number of isochronous URBs. */
-#define UVC_URBS 5
+#define UVC_URBS 76
/* Maximum number of packets per URB. */
-#define UVC_MAX_PACKETS 32
+#define UVC_MAX_PACKETS 64
/* Maximum number of video buffers. */
#define UVC_MAX_VIDEO_BUFFERS 32
/* Maximum status buffer size in bytes of interrupt URB. */
I tried this patch to fix my issue with corrupt video from multiple high resolution usb3 cameras but it did not help unfortunately. Do you have any other suggestions?
The patch did not fix the issue also it looks like the patch you mentioned above is for cameras running in isochronous mode is that true? I am getting closer but our cameras are running in bulk mode I can successfully get two cameras to work on a single controller at a rate of 148MB/s each. My understanding is the usb stack limits bulk transfers to 80% of usb3 bandwidth I am still only using 2/3 of that number. I am using a pci card for an additional usb controller I start getting the dropping payload stuff whenever I attempt to access a third camera. I am using the firmware patch as well.
I got it working with kernel from 24.1 release. Notes:
“Increase TRBS_PER_SEGMENT to 256” patch was already in 24.1 kernel
had to add “Increase UVC_URBS and UVC_MAX_PACKETS” patch
setting “set cpus run at max rate” was critical step for getting uncorrupted frames at the beginning after skipping first 3 frames at start.
Like mentioned in previous post I also got it working only with 2 cameras streaming simultaneously (16MP @ 3.3fps, 2 bytes per pixel). Adding 3rd camera brings the frame corruption back. Looks like cameras are fighting for available bandwidth. Cameras are streaming in bulk transfer mode. I think I need to slow-down the readout rate/clock.
One thing I wonder about…since this is bulk mode it is expected that the camera uploading an image to the computer would halt if told to do so, and then continue when possible (just like a hard drive). Is the camera also pausing snapping of a new frame when the current frame is not yet uploaded? Either pausing the snap of a new shot or having sufficient buffer would be required to stop overrun at the camera’s buffer…in the case of multiple cameras this would be even more critical that a new frame not be snapped until the buffer in the camera itself is sufficient (if there is no pause in the camera itself then you are back to needing isochronous mode).