Flickr in concurrent operation of two cameras

dumbogeorge · May 24, 2017, 1:01pm

Hi Folks,

I am using gstreamer command line to record concurrently from two SONY IMX274 cameras. I am recording at 1080p@30fps - as a h265 stream (5 Mbps).

I see flickr and frame drops.

However when I record ‘stand alone’ from each camera - I do not see flickr.

Any ideas why could it be so ?

Is there a way to attach videos ?

Thanks,

DaneLLL · May 25, 2017, 3:08am

Hi,
Please share your pipeline.
Please also check if you see flickers in camera preview
gst-launch-1.0 nvcamerasrc sensor-id=0 ! ‘video/x-raw(memory:NVMM),width=1920,height=1080’ ! nvoverlaysink overlay-x=400 overlay-y=600 overlay-w=640 overlay-h=360 overlay=1 & gst-launch-1.0 nvcamerasrc sensor-id=1 ! ‘video/x-raw(memory:NVMM),width=1920,height=1080’ ! nvoverlaysink overlay-w=640 overlay-h=360 overlay=2

dumbogeorge · May 25, 2017, 7:41am

Thanks DaneLLL. My command line is -

gst-launch-1.0 -vv nvcamerasrc sensor-id=0 fpsRange="30.0 30.0" ! 'video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)I420, framerate=(fraction)30/1' !   omxh265enc iframeinterval=24 bitrate=5000000 ! h265parse  ! matroskamux  ! filesink location=/home/ubuntu/junk11.mkv -e nvcamerasrc sensor-id=1 fpsRange="30.0 30.0" ! 'video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)I420, framerate=(fraction)30/1' !   omxh265enc iframeinterval=24 bitrate=5000000 ! h265parse  ! matroskamux  ! filesink location=/home/ubuntu/junk12.mkv -e

dumbogeorge · May 25, 2017, 8:02am

Hi DaneLLL
I ran your preview command line. Two observations -

When - I run your cmd line after fresh power up, I do not see flicker
When I run your cmd line (after killing first run with ctrl-c and using kill command) for second or third time - I see flicker…

After first run the flicker never goes away.

Please note that flicker DOES go away after jacking up the frequencies of modules…

ubuntu@tegra-ubuntu:~$ sudo  /home/ubuntu/jetson_clocks.sh 
ubuntu@tegra-ubuntu:~$ sudo  /home/ubuntu/jetson_clocks.sh --show
CPU frequency Governor: userspace
CPU MinFreq=102000 MaxFreq=1734000 CurrentFreq=1734000
CPU HOTPLUG: 0
Online CPUs: 0-3
cpu0: 1734000
cpu1: 1734000
cpu2: 1734000
cpu3: 1734000
CPU Cluster Switching: Disabled
GPU MinFreq=0 MaxFreq=998400000 CurrentFreq=998400000 FreqOverride=1
EMC MinFreq=0 MaxFreq=1600000000 CurrentFreq=1600000000 FreqOverride=1
Fan: speed=255

Now no more flicker with your preview command line. But it flickers with my recording command line.

THanks

DaneLLL · May 25, 2017, 8:23am

Hi,
Does it happen with omxh264enc?

dumbogeorge · May 25, 2017, 10:58am

Yes. It happens with omxh264enc and/or with lower resolutions as well.

gst-launch-1.0 -vv nvcamerasrc sensor-id=0 fpsRange="30.0 30.0" ! 'video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)I420, framerate=(fraction)30/1' !   omxh264enc iframeinterval=24 bitrate=5000000 ! h264parse  ! matroskamux  ! filesink location=/home/ubuntu/junk11.mkv -e nvcamerasrc sensor-id=1 fpsRange="30.0 30.0" ! 'video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)I420, framerate=(fraction)30/1' !   omxh264enc iframeinterval=24 bitrate=5000000 ! h264parse  ! matroskamux  ! filesink location=/home/ubuntu/junk12.mkv -e

DaneLLL · May 26, 2017, 1:19am

Hi,
The issue is in the camera sources. We have verified dual camera encoding with our internal dual camera board and it works fine. Also there are successful multiple encoding cases:
[url]https://devtalk.nvidia.com/default/topic/979908/jetson-tx1/gstreamer-transcoding-performance-issue/post/5033461/#5033461[/url]

Please share your device tree for dual cameras. Do you have separate ISP files to the two cameras?

dumbogeorge · May 26, 2017, 6:01am

Hi DaneLLL,
I believe we have only one ISP files as far as I can comprehend. I am checking with Leopard. Will update.

I also feel that it could be camera settings. As the flicker seem to go down when lighting is more uniform. If I try to record from a room , and capture broad daylight scene outside the window (high lighting contrasts) then it flickers more. This is the type of scene that is typically used for HDR testing.

Thanks,

dumbogeorge · May 28, 2017, 1:25pm

Hi DaneLLL,

I could not understand what ‘device tree’ refers to ? Can you please explain ?

“Please share your device tree for dual cameras”

Further more are there other cameras I can buy which are likely to give me less dual camera encoding issues ?

Thanks

linuxdev · May 28, 2017, 5:05pm

Device tree is a tree of setup instructions for various devices which have options better left out of the kernel (the kernel remains abstract, the specific settings stay in the device tree)…this is set prior to the kernel using those devices, e.g., loaded by the boot loader (the drivers depend on this being set up before loading the driver). Examples are whether a GPIO is to be I/O or some other purpose, or perhaps settings in a register which has options for a device. The file this is loaded from is a compiled binary format…this is the “DTB” file format for a device tree. The source code format is human readable and is the “DTS” file. The two can be compiled/de-compiled to change format.

In the kernel you can run “make dtbs” to put together a series of DTS files to produce a single DTB file. The DTB usually does not change, so most people never build a DTB. The location where U-Boot looks for a DTB file is in the FDT key/value pair entry of “/boot/extlinux/extlinux.conf”. Here is an example from R24.2.1:

FDT /boot/tegra210-jetson-tx1-p2597-2180-a01-devkit.dtb

If you wanted to examine a DTB, then edit it, and rebuild a DTB from the extracted DTS, here’s the recipe (this is often the place to verify kernel source changes made it in; in some cases this is also simpler to edit compared to building a kernel):

# Produce a DTS source file as "extracted.dts":
dtc -I dtb -O dts -o /tmp/extracted.dts /boot/the_firmware_in_extlinux.dtb
# Examine and perhaps edit "extracted.dts".
# Put edited DTS back into a new DTB as "modified.dtb":
dtc -I dts -O dtb -o /tmp/modified.dtb /tmp/extracted.dts

The only downside in simply reverse compiling a DTB to DTS format is that comments in the kernel source will be left out, and also some names may not match what is used in the kernel source.

Note that you can install package “dtc” and that this is not dependent upon architecture…most dtc from most systems work just find with a prepackaged dtc (such as from “apt-get search dtc” and “sudo apt-get install ”. The kernel always has the source for this with it, and there is a “make dtc” target which produces a version of dtc guaranteed compatible with that kernel.

kayccc · June 8, 2017, 5:05am

Hi dumbogeorge,

Is there any update after checked with Leopard?

Have you clarified and confirmed the issue was in the camera sources ?

Thanks

dumbogeorge · June 8, 2017, 6:18am

Hi Kayccc,

Sorry about not giving an update earlier. I checked with Leopard folks, they have tried to repeat in on their cameras. They does seem to occur on their board/Cameras, and they are still investigating it.

I found an interim solution. With trial an error I observed that if we do not try to start two cameras ‘concurrently’ then the flicker is not there.

Instead of

gst-launch-1.0 -vv nvcamerasrc sensor-id=0 fpsRange="30.0 30.0" ! 'video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)I420, framerate=(fraction)30/1' !   omxh265enc iframeinterval=24 bitrate=5000000 ! h265parse  ! matroskamux  ! filesink location=/home/ubuntu/junk11.mkv -e nvcamerasrc sensor-id=1 fpsRange="30.0 30.0" ! 'video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)I420, framerate=(fraction)30/1' !   omxh265enc iframeinterval=24 bitrate=5000000 ! h265parse  ! matroskamux  ! filesink location=/home/ubuntu/junk12.mkv -e

If I do

gst-launch-1.0 -vv nvcamerasrc sensor-id=0 fpsRange="30.0 30.0" ! 'video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)I420, framerate=(fraction)30/1' !   omxh265enc iframeinterval=24 bitrate=5000000 ! h265parse  ! matroskamux  ! filesink location=/home/ubuntu/junk11.mkv -e & 

gst-launch-1.0 -vv nvcamerasrc sensor-id=1 fpsRange="30.0 30.0" ! 'video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)I420, framerate=(fraction)30/1' !   omxh265enc iframeinterval=24 bitrate=5000000 ! h265parse  ! matroskamux  ! filesink location=/home/ubuntu/junk12.mkv -e &

Then I do not see flicker !