Argus - Syncing multiple capture sessions

I am attempting to capture images from multiple cameras (up to 6). I was able to do so by creating multiple capture sessions (one for each camera). My issue is that the sessions are not synchronized. The cameras do not capture their frames at the same time. There is a delay that ranges from a few milliseconds to about 100 milliseconds. It is not consistent. Does anyone have any ideas on how to sync the cameras so there is not much of a delay?

Thank you

hello mmorsy,

since it’s software capture request to send the controls to low-level drivers, you may also had hardware pins to connect each sensor to sync the sensors.
there should be a master camera device to handle the frame sync pin and the others being slaves for synchronization.

may I know all of your camera sensors were same,
also, may I know which sensor you’re working with.
thanks

All 6 cameras are Leopard Imaging IMX274.

The reason I’m creating multiple capture sessions is because when I put all cameras on a single session, even though they seem to be perfectly synced, I can’t use more than 3 cameras for some reason. If I try to capture 4 cameras or more, it will only capture one and report it for all of them. That’s why I created multiple capture sessions so that each session only holds one camera. Now I can capture all 6 cameras, but it results in the syncing issue I mentioned.

hello mmorsy,

as I mentioned in comment #2,
you may need to contact Leopard team to have hardware solution for camera synchronization.
thanks

I contacted them and I’m waiting for a response. Do you have any idea why a capture session can’t handle more than 3 cameras? If I fix that, my issue might be resolved.

hello mmorsy,

may I know the sensor resolution and also frame rate of your use-case.
had you tried to lower the bandwidth in a single capture session, is it still cannot handle more than 3-cameras?
thanks

I’ve tried it with multiple resolutions (3840x2160, 1920x1080). I also tried frame rates between 1 to 30 fps. I have the same issue every time I try to capture with more than 3 cameras.

hello mmorsy,

we had verify 6-camera use-case with our reference camera board, the running resolution is 2592x1944@30-fps. it works as expect with Argus camera, multi-session.
could you please share the kernel message when the failure reproduced.
thanks

I was always able to capture 6 cameras using multi-session. The issue is that they’re not synchronized. That’s why I am trying to capture them using a single session (then they might be synchronized). Is that possible? Right now I can only capture 3 cameras.

Also, I’ve been getting these error messages when my program terminates.

(Argus) Objects still active during exit: [CameraProvider (0x1098e20): refs: 1, cref: 0]
NvEglHandle: Unfreed handle upon egl deinit: type=2 refcnt=2
NvEglHandle: Unfreed handle upon egl deinit: type=1 refcnt=1
NvEglHandle: Unfreed handle upon egl deinit: type=1 refcnt=1
NvEglHandle: Unfreed handle upon egl deinit: type=2 refcnt=2
NvEglHandle: Unfreed handle upon egl deinit: type=2 refcnt=2
NvEglHandle: Unfreed handle upon egl deinit: type=2 refcnt=2
NvEglHandle: Unfreed handle upon egl deinit: type=7 refcnt=1
NvEglHandle: Unfreed handle upon egl deinit: type=1 refcnt=1
NvEglHandle: Unfreed handle upon egl deinit: type=6 refcnt=1
NvEglHandle: Unfreed handle upon egl deinit: type=1 refcnt=1
NvEglHandle: Unfreed handle upon egl deinit: type=6 refcnt=1
NvEglHandle: Unfreed handle upon egl deinit: type=1 refcnt=1
NvEglHandle: Unfreed handle upon egl deinit: type=2 refcnt=2
NvEglHandle: Unfreed handle upon egl deinit: type=1 refcnt=1
NvEglHandle: Unfreed handle upon egl deinit: type=2 refcnt=2

Do you know what this means or how to fix it?

Thank you

hello mmorsy,

  1. Right now I can only capture 3 cameras.
    you had also confirm these 3-cameras were synchronized, right?

  2. NvEglHandle messages is trying to deinit global internal state.
    could you please have another try to launch six cameras in single session without EGL, please have a try to save the output result into files.
    thanks

  1. I am using IArgusCaptureMetadata with each frame to get their timestamps and I’m finding the time difference between each camera, which turns out to be 0.000. I am assuming these timestamps are specific to each camera’s frame and not just repetitions of the first camera’s frame. So yes, they appear synchronized.

  2. I tried to launch 6 cameras and I get these error messages:

SCF: Error BadValue: device not found (in src/common/DeviceRegistry.h, function assign(), line 142)
SCF: Error BadValue: (propagating from src/api/CameraDriver.cpp, function createSession(), line 503)
(Argus) Error BadValue: (propagating from src/api/CaptureSessionImpl.cpp, function initialize(), line 116)
(Argus) Error BadValue: (propagating from src/api/CameraProviderImpl.cpp, function createCaptureSession(), line 234)

The same happens when I try 5. But when I try 4, the first camera launches but not the other 3.

When I use a single session, I don’t get the NvEglHandle messages. I only get it with multiple sessions. So how would I fix it?

Thank you

hello mmorsy,

let’s have clarification as below,
[case-1] multiple sensors per multi sessions

launch each camera sensor with different capture request, they’re not synchronized.

[case-2] multiple sensors per single sessions

using a single capture request to launch several camera frames at once.

argus_camera and MMAPI examples were based-on case-1, you’re also able to capture images from multiple cameras without issues.
you got failures with case-2, and it’s now only able to launch 3-cameras.

FYI,
here’s source limitation of scf session, you’re not able to launch all 6-cameras in a single session,
thanks

Hi mmorsy,

Our 6 x IMX274 camera kit doesn’t support synchronization function since IMX274 doesn’t support slave mode.
We have another camera kit LI-TX1-CB-6CAM + IMX185 camera. This camera kit supports synchronization function. We set the IMX185 to salve mode in driver and use the FPGA (on LI-TX1-CB-6CAM) to send trigger singles to 6 cameras.

IMX185-MIPI: https://leopardimaging.com/product/li-imx185-mipi-m12/
LI-TX1-CB-6CAM: https://leopardimaging.com/product/li-tx1-cb-6cam/

I am aware of the IMX185, but I need better resolution. Will there be an update to the IMX274 driver so it supports synchronization?

Thank you

hello mmorsy,

could you please have a try to replace couple of binaries, libscf.so and libargus.so
please back-up and replace them with the attachment, thanks
Topic1039183_Oct02.7z (1.91 MB)

Where can I find libscf.so and libargus.so? In what directory?

hello mmorsy,

all binaries were located under

/usr/lib/aarch64-linux-gnu/tegra/

please back-up before overwrite them. you may restart camera services or simply perform a warm-reboot to make the change takes effect.
thanks

Hi JerryChang
I now have the same problem with you, all my imx274 work in master mode(csi/mipi mode),could you tell me how to sync the sensors?
Should I change all the sensors to salve mode? and pull all their XVS/XHS to the one signal from fpga?
Thanks very much for your reply.

hello fm2012,

couple of questions need your feedback.

  1. may I know how may camera senors you would like to synchronize for your use-case.
  2. could you please share what’s your device, as Leopard team mentioned in comment #13, their board don’t support synchronization function.

Hi JerryChang
1)I use 4 camera sensor(IMX274).
2)Actually I use my own platform,I plan to use FPGA to control or capture 4 IMX274 sensor,and stitch them.