I’m trying to record an MJPEG stream to a h265 file on my Nano with GStreamer. I’ve been running my setup for a while now, and everything’s working great, except that I couldn’t run my application with Docker.
Saw that Jetpack 4.2.1 contained experimental support for nvidia-docker, so I set out on an adventure to Dockerize my final microservice (my goal is to run all of my services with k3s, currently my recording-service is prohibiting this)
What I did
- wiped my Nano and installed Jetpack 4.2.1 on it
- built my Docker image by starting from the nvcr.io/nvidia/l4t-base:r32.2 image
- ran the image
Inside of the container, my GStreamer outputs the following error:
===== NVMEDIA: NVENC =====
Opening channel /dev/nvhost-msenc failed
NVMEDIA: NvMMLiteVideoEncBlockOpen: 500: - Could not get NVENC Channel handle for inst 0
NvMMLiteBlockCreate : Block : BlockType = 8
Caught SIGSEGV
exec gdb failed: No such file or directory
Spinning. Please run 'gdb gst-launch-1.0 54' to continue debugging, Ctrl-C to quit, or Ctrl-\ to dump core.
Inside of the container, it seems that there is no ‘nvhost-msenc’, or ‘nvhost-nvjpg’, which GStreamer is trying to use. On my bare-metal Nano, ‘nvhost-msenc’ and ‘nvhost-nvjpg’, are available.
How to reproduce
Inside of the container:
user@jetson:~/applications$ docker run --runtime=nvidia nvcr.io/nvidia/l4t-base:r32.2 ls -al /dev | grep nv
crw-rw---- 1 root video 506, 1 Aug 17 13:56 nvhost-as-gpu
crw-rw---- 1 root video 242, 0 Aug 17 13:56 nvhost-ctrl
crw-rw---- 1 root video 506, 2 Aug 17 13:56 nvhost-ctrl-gpu
crw-rw---- 1 root root 506, 3 Aug 17 13:56 nvhost-dbg-gpu
crw-rw---- 1 root video 506, 0 Aug 17 13:56 nvhost-gpu
crw-rw---- 1 root video 242, 9 Aug 17 13:56 nvhost-nvdec
crw-rw---- 1 root root 506, 4 Aug 17 13:56 nvhost-prof-gpu
crw-rw---- 1 root video 242, 13 Aug 17 13:56 nvhost-vic
crw-rw-rw- 1 root root 195, 255 Aug 17 16:17 nvidiactl
crw-rw---- 1 root video 10, 61 Aug 17 13:56 nvmap
Outside of the container:
user@jetson:~/applications$ ls -al /dev | grep nv
crw-rw---- 1 root video 506, 1 Aug 17 15:56 nvhost-as-gpu
crw-rw---- 1 root video 242, 0 Aug 17 15:56 nvhost-ctrl
crw-rw---- 1 root video 506, 2 Aug 17 15:56 nvhost-ctrl-gpu
crw-rw---- 1 root video 242, 26 Aug 17 15:56 nvhost-ctrl-isp
crw-rw---- 1 root video 242, 30 Aug 17 15:56 nvhost-ctrl-isp.1
crw-rw---- 1 root video 242, 10 Aug 17 15:56 nvhost-ctrl-nvdec
crw-rw---- 1 root video 242, 34 Aug 17 15:56 nvhost-ctrl-vi
crw-rw---- 1 root video 506, 6 Aug 17 15:56 nvhost-ctxsw-gpu
crw-rw---- 1 root root 506, 3 Aug 17 15:56 nvhost-dbg-gpu
crw-rw---- 1 root video 506, 0 Aug 17 15:56 nvhost-gpu
crw-rw---- 1 root video 242, 25 Aug 17 15:56 nvhost-isp
crw-rw---- 1 root video 242, 29 Aug 17 15:56 nvhost-isp.1
crw-rw---- 1 root video 242, 17 Aug 17 15:56 nvhost-msenc
crw-rw---- 1 root video 242, 9 Aug 17 15:56 nvhost-nvdec
crw-rw---- 1 root video 242, 21 Aug 17 15:56 nvhost-nvjpg
crw-rw---- 1 root root 506, 4 Aug 17 15:56 nvhost-prof-gpu
crw-rw---- 1 root video 506, 7 Aug 17 15:56 nvhost-sched-gpu
crw-rw---- 1 root video 242, 1 Aug 17 15:56 nvhost-tsec
crw-rw---- 1 root video 242, 5 Aug 17 15:56 nvhost-tsecb
crw-rw---- 1 root video 506, 5 Aug 17 15:56 nvhost-tsg-gpu
crw-rw---- 1 root video 242, 33 Aug 17 15:56 nvhost-vi
crw-rw---- 1 root video 242, 13 Aug 17 15:56 nvhost-vic
crw-rw-rw- 1 root root 195, 255 Aug 17 18:17 nvidiactl
crw-rw---- 1 root video 10, 61 Aug 17 15:56 nvmap
I tried mounting /dev/nvhost-msenc from the host to the container, but no luck there either.
The command I’m using for GStreamer (I’m templating the {url} and {output}):
gst-launch-1.0 -e -m souphttpsrc location={url} do-timestamp=true ! multipartdemux ! image/jpeg,framerate=25/2 ! nvjpegdec ! video/x-raw,framerate=25/2 ! omxh265enc preset-level=3 ! mp4mux ! filesink location={output}
Any ideas? Am I doing something wrong, is there a reason nvhost-msenc isn’t available, or is this just part of the adventure towards full nvidia-docker support on the Nano?