Unsupported 240 FPS capture via nvcamerasrc using Leopard Imaging IMX 274

Hi,

   I am using a triple IMX 274 suite, which claims to support 1280x540 @ 240 FPS on Jetson TX2, 28.2 L4T.

   But I have confirmed the maximized fps can only reach 80 fps. I have communicated with support enginneer from Leopard Imaging.

   They suggest me to report the bug here and ask for NVIDIA support.

   Following is the command line which I used and video file properties.

   gst-launch-1.0 -e nvcamerasrc fpsRange=240,240 intent=3 ! 'video/x-raw(memory:NVMM), width=1280, height=540, framerate=240/1, format=NV12' ! omxh264enc ! mp4mux ! filesink location=test.mp4

   Video
     Dimensions : 1280x540
     Codec      : H.264 ( Contrained Baseline Profile )
     Framerate  : 79 frames per second
     Bitrate    : 4146 kbps

  I also tried to remove the encoder, with Ridgerun provided gstshark, it also show 

0:00:03.766512319 17186 0x530130 TRACE GST_TRACER :0:: framerate, pad=(string)nvcamerasrc0_src, fps=(uint)79;

reference found

gst-launch-1.0 -e nvcamerasrc sensor-id=0 fpsRange="240.0 240.0" ! 'video/x-raw(memory:NVMM), width=(int)1280, height=(int)540, format=(string)I420, framerate=(fraction)240/1' ! nvvidconv ! 'video/x-raw(memory:NVMM), format=(string)I420' ! omxh264enc bitrate=8000000 ! 'video/x-h264, stream-format=(string)byte-stream' ! filesink location=IMX274-240fps_0.avi -e

tx2_240fps.tar.gz (50.7 KB)

Well, I am working on L4T R28.2. I have tried the image provided by Andrey1984. It will cause SIGSEGV, maybe it’s for 24.2.

The properties of nvcamerasrc shows it already changed to framerate; [0/1, 2147483647/1].

BTW, I can do 120FPS capture with OV camera of the evaluation board. After updating to IMX274 driver, it seems the fps is now limited to 79 FPS.

I double confirmed with Leopard Imaging, they said via v4l2, 240fps is working from driver perspetive. Here is evidance:

v4l2-ctl --set-fmt-video=width=1280,height=540 --stream-mmap --stream-count=500 -d /dev/video0
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 240.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 240.00 fps
<<<<<<<<<<<<<<<<<<<

hello pin.jiang,

could you please have a try to boost system performance by execute jetson_clock

sudo ./jetson_clocks.sh

in addition,
could you launch the IMX274@240-fps sensor mode and dump VI and ISP clock rate via debugfs as below, thanks

sudo -i
cat /sys/kernel/debug/bpmp/debug/clk/vi/rate
cat /sys/kernel/debug/bpmp/debug/clk/isp/rate

I just do what you asked.

when the system is idle ( not running the camera ) :

 cat /sys/kernel/debug/bpmp/debug/clk/vi/rate
 409600000

 cat /sys/kernel/debug/bpmp/debug/clk/isp/rate
 768000000

when the system is busy ( IMX274 running @ 240FPS ) :

 cat /sys/kernel/debug/bpmp/debug/clk/vi/rate
 230400000

 cat /sys/kernel/debug/bpmp/debug/clk/isp/rate
 396800000

hello pin.jiang,

it’s weird that clock rate become half during the sensor is running,
please execute below commands to boost the system performance for testing.
thanks

sudo ./jetson_clocks.sh
echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked
cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate
cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate
echo max_rate > /sys/kernel/debug/bpmp/debug/clk/vi/rate
echo max_rate > /sys/kernel/debug/bpmp/debug/clk/isp/rate

I execute the commands as following:

nvidia@tegra-ubuntu:~$ sudo ./jetson_clocks.sh
[sudo] password for nvidia: 
nvidia@tegra-ubuntu:~$ sudo -i
root@tegra-ubuntu:~# cat /sys/kernel/debug/bpmp/debug/clk/vi/rate
409600000
root@tegra-ubuntu:~# cat /sys/kernel/debug/bpmp/debug/clk/isp/rate
768000000
root@tegra-ubuntu:~# echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
root@tegra-ubuntu:~# echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked
root@tegra-ubuntu:~# cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate
1011200000
root@tegra-ubuntu:~# cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate
1088000000
root@tegra-ubuntu:~# echo 1011200000 > /sys/kernel/debug/bpmp/debug/clk/vi/rate
root@tegra-ubuntu:~# echo 1088000000 > /sys/kernel/debug/bpmp/debug/clk/isp/rate 


root@tegra-ubuntu:~# gst-launch-1.0 -e nvcamerasrc fpsRange=240,240 intent=3 ! 'video/x-raw(memory:NVMM), width=1280, height=540, framerate=240/1, format=NV12' ! omxh264enc ! mp4mux ! filesink location=test.mp4

NvCameraSrc: Trying To Set Default Camera Resolution. Selected sensorModeIndex = 3 WxH = 1280x540 FrameRate = 240.000000 ...

Setting pipeline to PLAYING ...
New clock: GstSystemClock
Framerate set to : 240 at NvxVideoEncoderSetParameterNvMMLiteOpen : Block : BlockType = 4

And I open another window

root@tegra-ubuntu:~# cat /sys/kernel/debug/bpmp/debug/clk/vi/rate
1011200000
root@tegra-ubuntu:~# cat /sys/kernel/debug/bpmp/debug/clk/isp/rate
1088000000

But I found the saved MP4 is still 79 FPS only. I will upload the mp4 shortly later.

Hi pin,
Do you see 240 fps with fpsdisplaysink?
[url]https://devtalk.nvidia.com/default/topic/1027772/jetson-tx2/two-csi-cameras-on-tx2/post/5227935/#5227935[/url]

I use the script:

GST_DEBUG=fpsdisplay:5 gst-launch-1.0 nvcamerasrc queue-size=3 sensor-id=0 fpsRange=“240 240” ! ‘video/x-raw(memory:NVMM), width=1280, height=540, framerate=240/1, format=NV12’ ! fpsdisplaysink video-sink=false sync=false async=false

… DEBUG fpsdisplay fpsdisplaysink.c:376:display_current_fps: Updated max-fps to 81.593611
… DEBUG fpsdisplay fpsdisplaysink.c:376:display_current_fps: Updated min-fps to 81.593611
… DEBUG fpsdisplay fpsdisplaysink.c:376:display_current_fps: Updated min-fps to 79.186946
… DEBUG fpsdisplay fpsdisplaysink.c:376:display_current_fps: Updated min-fps to 79.182683

So saved the mp4 file, gst_shark and fpsdisplaysink all indicate the fps is around 80. But I do achieved 120 FPS with the on-board OV camera.

Hi pin, the log shows your sensor can only achieve ~80fps. Are you able to contact Leopard to get further support?

Yes. But Simon Zhu from Leopard Imaging asked me to run below commands on Jetson Tx2:

v4l2-ctl --set-fmt-video=width=1280,height=540 --stream-mmap --stream-count=1000 -d /dev/video0

v4l2-ctl --set-fmt-video=width=1280,height=540,pixelformat=RG10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=10 --set-ctrl=gain=2000 --stream-to=imx274.raw -d /dev/video0

<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 240.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 240.00 fps
<<<<<<<<<<<<<<<<<<<

They claimed that driver shall be no problem from v4l2, they asked me to turn to NVIDIA.

hello pin.jiang,

we’re investigate this internally, will get back to you once we come out some conclusions.

Any update on this topic?