If my understanding is correct, these values are populated by vinotify which comes from the RTCPU. I don’t understand where these would be synchronized. Is there a way of ensuring they have the same time? Or a way of determining the offset so that we can compensate in software?
you’re right, we’re using the timestamp of sensor start of frame to make it as v4l2 buffer timestamp.
since there are stream setup and buffer initialization stages, time offset between v4l2 buffer and system time is expect.
you can enable the dynamic debug prints of vi_notify.c to confirm the SOF timestamp.
also, may I know what’s your use-case about timestamp synchronization.
thanks
Hi JerryChang,
Thanks for the reply. While I understand that you are saying there is a delay between SOF and when we get the buffer, buy why is it 4 seconds IN THE FUTURE?
Note in my original post:
kernel time: 116.856140 sec
SOF time: 121.247362176 sec
We have a system with an IMU and are trying to synchronize physical movement with camera vision.
I just tried jetpack 3.3 on my Jetson dev kit and the ov5693 and experience the same problem. While I don’t have my debug code in place to see the timestamps per frame, I do see that stopping the streaming displays messages on the console.
struct timeval
timestamp
For capture streams this is time when the first data byte was captured, as returned by the clock_gettime() function for the relevant clock id; see V4L2_BUF_FLAG_TIMESTAMP_* in Buffer Flags. For output streams the driver stores the time at which the last data byte was actually sent out in the timestamp field. This permits applications to monitor the drift between the video and system clock. For output streams that use V4L2_BUF_FLAG_TIMESTAMP_COPY the application has to fill in the timestamp which will be copied by the driver to the capture stream.
there is bug about RTCPU and kernel system out-of-sync, we’re not root cause yet.
please download [Tegra X2 (Parker Series SoC) Technical Reference Manual], and please refer to [Figure 202: NOTIFY System Diagram] and [Real-Time CPU Transport] chapter for more details.
may I have details about your use-case?
I would suggest you to overwrite the timestamp of vi-notify for a temporary solution.
thanks
Hey, I am working together with phabsch.
We need achieve high accuracy synchronization between cameras with other sensors like IMU and Lidar to do some SLAM applications.
my suggestion to unblock your process would be overwrite the timestamp of each capture frame for r28.2.1/ TX2.
for example,
please have modification in below VI driver function, and update the timestamp with linux kernel function.
Hi JerryChang,
I do already have that work around in place, but I am seeing problems with it. It seems that the vi_notify_wait portion of code after the nvhost_syncpt_wait_timeout_ext() is prone to delays. I haven’t figured out yet whether this is a kernel thread scheduling issue or something else.
I have 4 cameras synced together running at 60 fps. I can see the image is acquired at the same time on all 4 cameras. I used a chasing LED strobe with 100us between LED increment. When I use the timestamp from the RTCPU the values are really close across buffers (+/- 1us). But if I restamp the value in this function I have seen up to 84ms (yes milliseconds) delta between synchronized frames (the content is still synced).
I am thinking that since this code runs in a separate thread for each channel it is being delayed due to thread context switching.
We could probably work with this and just restamp each of the frames with the lowest of all 4 frames, but I am concerned that we are seeing 84ms between them. This means that we sometimes drop multiple frames (because the next acquisition in our code doesn’t start until all 4 frames have captured).
JerryChang, restamp frames in software is not good idea: it will add delay and jitter under load (code in #14 runs as thread, not in interrupt)
I found, that CLOCK_MONOTONIC_RAW return time, close to v4l2 timestamps, but after some hours it got offset too (looks like it work on close, but different frequency. offset grows with uptime)