Problems minimizing latency and maximizing quality for rtsp and mpeg-ts.

Autonomous Machines Jetson & Embedded Systems Jetson TX2
1

We have a live streaming requirement to stream both rtsp (axis camera) and udp mpeg ts from another e ncoder. The application requires low latency and smooth scrolling of video, since users will be using ptz cameras.

For RTSP we are using the following pipeline:

rtspsrc location=rtsp://10.3.1.214/axis-media/media.amp latency=200 ! rtph264depay 
   ! h264parse ! queue  ! omxh264dec ! nvvidconv ! nvoverlaysink display-id=2 sync=true

With sync=false, we get jerky video. WIth sync=true and default latency, we get about 1 sec
delay. With both sync=true and latency=200 we get a good compromise. Setting the latency
too low results in jerky video.

For the mpeg-ts stream, the pipeline is similar:

udpsrc uri=udp://127.0.0.1:1132 ! tsparse set-timestamps=true ! tsdemux 
 ! h264parse ! queue  ! omxh264dec  ! nvvidconv ! nvoverlaysink display-id=2 sync=true

The video is smooth when using PTZ, but the delay is over a second, making positioning
the camera interactively difficult.

If we set sync=false, the latency is low but the video is very jerky when panning.

Unfortunately, there does not seem to be a latency parameter in any of the elments
in the mpeg ts decode pipeline that directly controls the latency of the video.

Is there anything we can adjust to minimize this latency.

Thanks in advance,

Cary

1 Like
2

Hi Cary,
You can try if executing jetson_clocks helps:

sudo ./jetson_clocks.sh

Timestamps is totally handles in gstreamer frameworks and NVIDIA-developed omxh264dec does not give any private handling. Ideally HW decoder omxh264dec should bring you better performance then SW decoder avdec_h264.

3

Thanks DaneLLL,

We are using the omxh264dec.

The issue is that the video jerks when we pan the AXIS I/P camera from side to side when sync=false. When sync=true the picture is smooth but we get 2 second delay. How can we get low delay without jerkiness.

Thanks a lot,

Unni

4

Hi Unni,
Can you reproduce the issue with default camera? You can set up rtspserver with nvcamerasrc via test-launch.
[url]https://devtalk.nvidia.com/default/topic/1018689/jetson-tx2/vlc-playing-gstreamer-flow/post/5187270/#5187270[/url]

5

It is easy to reproduce. All you have to do is decode a UDP transport stream. Do you have the capability?

6

Please share full steps of reproducing it with default camera. Besides, what is the delay with avdec_h264?

7

Delay is over 2 seconds when sync=true. The latency is acceptable when sync=false, except the picture jerks when the camera is moved from side to side.

8

We do not observe long latency in streaming with nvcamerasrc:
Server

$ ./test-launch "nvcamerasrc ! video/x-raw(memory:NVMM),width=640,height=480 ! tee name=t t. ! queue ! nvoverlaysink overlay-x=50 overlay-y=50 overlay-w=640 overlay-h=480 t. ! queue ! omxh264enc ! video/x-h264,profile=baseline ! rtph264pay name=pay0 pt=96"

Client

$ export RTSP_PATH=rtsp://127.0.0.1:8554/test
$ gst-launch-1.0 rtspsrc location="$RTSP_PATH" latency=500 ! rtph264depay ! h264parse ! omxh264dec ! nvoverlaysink overlay-x=800 overlay-y=50 overlay-w=640 overlay-h=480 overlay=2

By setting latency=500, we see 500ms delay as expected.
External Media

9

how to change nvcamerasrc to my usb camera?

10

@DaneLL

the omxh264 is giving the following error messages:
(gst-launch-1.0:9588): GStreamer-CRITICAL **: 06:25:03.264: gst_caps_fixate: assertion ‘GST_IS_CAPS (caps)’ failed

(gst-launch-1.0:9588): GStreamer-CRITICAL **: 06:25:03.264: gst_caps_get_structure: assertion ‘GST_IS_CAPS (caps)’ failed

(gst-launch-1.0:9588): GStreamer-CRITICAL **: 06:25:03.264: gst_structure_get_string: assertion ‘structure != NULL’ failed

(gst-launch-1.0:9588): GStreamer-CRITICAL **: 06:25:03.264: gst_mini_object_unref: assertion ‘mini_object != NULL’ failed

I tried the nvv4l2decoder no critical messages, but slow.

Is there a better decoder to use?

James

11

Hi,
For using nvv4l2decoder, please set enable-max-performance

$ gst-launch-1.0 rtspsrc location="$RTSP_PATH" latency=500 ! rtph264depay ! h264parse ! nvv4l2decoder <b>enable-max-performance=1</b> ! nvoverlaysink overlay-x=800 overlay-y=50 overlay-w=640 overlay-h=480 overlay=2

Executing ‘sudo nvpmodel -m 0’ and ‘sudo jetson_clocks’ can run system in max performance and should also bring performance improvement.

12

Hi, what’s the different by using cv2.VideoCapture(‘rtsp://’)?
I try to compare fps, in v4l2 gstreamer is faster but in rtsp I can see the different.

13

What does “enable-max-performance” do? I haven’t found much about this in the documentation? Why isn’t it just max performance by default? I am using the AGX Xavier

14

Hi,
We run dynamic frequency in NVDEC engine to reduce power consumption. If your source is 640x480 you may not need to run at max clock always. Yo can check the clock by executing sudo tegrastats.
More information about tegrastats is in
https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/AppendixTegraStats.html#wwconnect_header