I apply the rt patch successfully, but the cycle test result is not normal,
nvidia@jetson-xxxxxxxxxxxxx:~$ uname -a
Linux jetson-xxxxxxxxxxxxx 4.9.108-rt76-rt #1 SMP PREEMPT RT Mon Dec 24 04:42:00 UTC 2018 aarch64 aarch64 aarch64 GNU/Linux
nvidia@jetson-xxxxxxxxxxxxx:~$ zcat /proc/config.gz | grep CONFIG_HZ
# CONFIG_HZ_PERIODIC is not set
# CONFIG_HZ_100 is not set
# CONFIG_HZ_250 is not set
# CONFIG_HZ_300 is not set
CONFIG_HZ_1000=y
CONFIG_HZ=1000
nvidia@jetson-xxxxxxxxxxxxx:~$ zcat /proc/config.gz | grep PREEMPT
CONFIG_PREEMPT_RCU=y
CONFIG_PREEMPT=y
CONFIG_PREEMPT_RT_BASE=y
CONFIG_HAVE_PREEMPT_LAZY=y
CONFIG_PREEMPT_LAZY=y
# CONFIG_PREEMPT_NONE is not set
# CONFIG_PREEMPT_VOLUNTARY is not set
# CONFIG_PREEMPT__LL is not set
# CONFIG_PREEMPT_RTB is not set
CONFIG_PREEMPT_RT_FULL=y
CONFIG_PREEMPT_COUNT=y
# CONFIG_DEBUG_PREEMPT is not set
# CONFIG_PREEMPT_TRACER is not set
the install of the rt os is ok, the result of the cycle test:
I haven’t patched, but just speculating. Did you first set “sudo nvpmodel -m 0”? Did you follow that up with “sudo ~nvidia/jetson_clocks.sh”? If not, try those steps, and then run the test again.
What was the failure message on nvpmodel and jetson_clocks.sh? Both will depend on kernel features which are custom to the Xavier. It is possible that some feature required for these broke or was removed from the RT patch. I could see removing some energy saving feature, but mostly that would be a bad sign if those functions no longer work.
nvidia@jetson-xxxxxxxxxxxxx:~$ uname -a
Linux jetson-xxxxxxxxxxxxx 4.9.108-rt76-rt #1 SMP PREEMPT RT Mon Dec 24 04:42:00 UTC 2018 aarch64 aarch64 aarch64 GNU/Linux
nvidia@jetson-xxxxxxxxxxxxx:~$ zcat /proc/config.gz | grep CONFIG_HZ
# CONFIG_HZ_PERIODIC is not set
# CONFIG_HZ_100 is not set
# CONFIG_HZ_250 is not set
# CONFIG_HZ_300 is not set
CONFIG_HZ_1000=y
CONFIG_HZ=1000
nvidia@jetson-xxxxxxxxxxxxx:~$ zcat /proc/config.gz | grep PREEMPT
CONFIG_PREEMPT_RCU=y
CONFIG_PREEMPT=y
CONFIG_PREEMPT_RT_BASE=y
CONFIG_HAVE_PREEMPT_LAZY=y
CONFIG_PREEMPT_LAZY=y
# CONFIG_PREEMPT_NONE is not set
# CONFIG_PREEMPT_VOLUNTARY is not set
# CONFIG_PREEMPT__LL is not set
# CONFIG_PREEMPT_RTB is not set
CONFIG_PREEMPT_RT_FULL=y
CONFIG_PREEMPT_COUNT=y
# CONFIG_DEBUG_PREEMPT is not set
# CONFIG_PREEMPT_TRACER is not set
So we are looking for a RT patch that works as Jetson TX2 do.
Looks like performance was indeed at max. One final thing to check is that the direct hardware access libraries are in place. If this shows all “ok”, then there is some other issue, but it is fast to check:
sha1sum -c /etc/nv_tegra_release
Neither the Cortex-A series nor Linux were designed for hard realtime, so even if things work as expected you won’t get better than soft realtime. I won’t be able to offer better advice, but you might mention some specifics on what subset of realtime needs improvement for your specific case (there may be workarounds, but a benchmark test doesn’t really say what specific needs you have).
/usr/lib/aarch64-linux-gnu/tegra/libnvosd.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvgov_camera.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvmmlite_image.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvomx.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvgov_force.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnveventlib.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvmedia.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvmmlite_utils.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libglx.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvexif.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvrm_gpu.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvtx_helper.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvargus_socketserver.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvscf.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvfnetstorehdfx.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvmm_parser.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvrm.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvmm_contentpipe.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvdla_utils.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvgov_gpucompute.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvgov_tbc.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvos.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvtnr.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvgov_graphics.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvimp.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvfnet.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvphs.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvavp.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvcapture.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvmmlite_video.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvfnetstoredefog.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvgov_il.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvodm_imager.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvjpeg.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvargus_socketclient.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvtvmr.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvdla_compiler.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvtracebuf.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvidia-egl-wayland.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvdc.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvgov_boot.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvgov_generic.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libtegrav4l2.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvmm.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvapputil.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvcameratools.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvcam_imageencoder.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnveglstream_camconsumer.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvmm_utils.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvomxilclient.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvwinsys.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvargus.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvgov_spincircle.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvphsd.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnveglstreamproducer.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvdla_runtime.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvll.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvrm_graphics.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvcolorutil.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvddk_2d_v2.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvtestresults.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvcamv4l2.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvcamerautils.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvcamlog.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvparser.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvddk_vic.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvdla_core.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvdla_os.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvgov_ui.so: OK
/usr/lib/aarch64-linux-gnu/tegra/libnvmmlite.so: OK
/usr/lib/aarch64-linux-gnu/libv4l/plugins/libv4l2_nvvidconv.so: OK
/usr/lib/aarch64-linux-gnu/libv4l/plugins/libv4l2_nvvideocodec.so: OK
/usr/lib/xorg/modules/drivers/nvidia_drv.so: OK
/usr/lib/xorg/modules/extensions/libglx.so: OK
In fact, the soft realtime is enough for us now. It will be OK if the max latency benchmark test performs as well as Jetson TX2.
The original working environment is a normal desktop (intel core i5-7500 cpu@3.4GHZx4) with soft realtime OS(PREEMPT RT ubuntu16.04) to control a 12 degree of freedom robot using ethercat and tensorflow. The max latency benchmark test on it remains stable below 100, and works well on it.
To make it smaller, we transfer the environment to Jetson TX2 with a special jetpack including soft realtime OS (PREEMPT RT ubuntu16.04) successfully, whose max latency benchmark test is also remains stable below 100.
It is said that Xavier is much powerful than TX2, so we tried to transfer it with soft realtime OS.
However, the max latency benchmark test shows there may be something wrong with my installation about PREEMPT rt patch. Because the test is also higher that 800 if I failed to install on the original desktop environment.
So we are still looking for a way to install PREEMPT RT patch that the max latency benchmark test on it remains stable below 100. It will be good if the OS is also ubuntu16.04 just as TX2 did, since the gcc and other libs wouldn’t be change and no compatibility problem will happen.
I know this requirement is rare and may not have a solution yet, but thanks for supporting so long any way.
Beyond what was already mentioned there isn’t much I can add. I have experimented with Xavier under different loads and it does remain responsive (not benchmarked, just my feel for the Xavier) under heavier loads than does the TX2. Someone else may have additional comments on making the RT patches work.
That is correct. The question from the original poster is about adding more patches. I don’t know how much of that would be present at this current kernel, but for what is present, then the “CONFIG_PREEMPT=y” is definitive.
We also plan to use XAVIER with RT.
Is it that with latest jetpack the RT patch is already applied ?
If I understand this thread correctly, although the patch is already in, the performance is not as expected. Is that right ?