qcasey1
September 11, 2019, 4:04pm
1
Hi all, loving this jetson nano board. The last step in my project is transcoding a raw stream mapping of MJPEG images (at ~25fps) to H264 using the hardware capabilities of this board.
I looked at the GStreamer PDF and I don’t see MJPEG video mentioned anywhere, so I went ahead and used the single image transcode as a jumping off point. This is what I’m using:
gst-launch-1.0 filesrc location=video.mkv ! image/jpeg, width=3840, height=2160, framerate=25/1 ! jpegparse ! jpegdec ! 'video/x-raw, format=(string)I420' ! omxh264enc ! matroskamux ! filesink location=video-compressed.mkv -e
The above doesn’t produce an error, but the FPS of the output file is around 1, or incredibly low. It should be 25-30. Also possibly unsolvable, but the transcode takes a while at about 50% real time speed. IIRC I was able to do a H264 to vp8 at 450% real time.
Am I doing something wrong with one of my pipelines?
Thanks in advance for any insights
EDIT: I think it was VP8 not vp9
Hi qcasey1, it looks like you are using the jpegdec element, which isn’t hardware accelerated.
Instead, try using the nvjpegdec element instead, which is hw-accelerated and included in the GStreamer PDF.
qcasey1
September 11, 2019, 5:45pm
3
Thanks for the response, but when I use nvjpegdec I get a whole bunch of
NvRmPrivFlush: NvRmChannelSubmit failed (err = 196623, SyncPointIdx = 12, SyncPointValue = 0)
before gstreamer eventually exits. Also would this effect the output video’s frames per second?
qcasey1
September 11, 2019, 5:46pm
4
Also it hangs my file transfers, is this a memory issue?
EDIT: my most recent dmesg shows this as a memory issue I think
[68497.308008] sd 0:0:0:0: [sda] tag#0 data out submit err -11 uas-tag 1 inflight: s-out a-cmd s-cmd work
[68497.308011] sd 0:0:0:0: [sda] tag#0 CDB: opcode=0x2a 2a 00 0e 8a aa 30 00 00 28 00
[68531.933836] jpegparse0:sink invoked oom-killer: gfp_mask=0x26080c0(GFP_KERNEL|__GFP_ZERO|__GFP_NOTRACK), nodemask=0, order=0, oom_score_adj=0
[68531.946611] jpegparse0:sink cpuset=/ mems_allowed=0
[68531.946624] CPU: 2 PID: 24381 Comm: jpegparse0:sink Not tainted 4.9.140-tegra #1
[68531.946627] Hardware name: NVIDIA Jetson Nano Developer Kit (DT)
[68531.946629] Call trace:
[68531.946638] [<ffffff800808bdb8>] dump_backtrace+0x0/0x198
[68531.946643] [<ffffff800808c37c>] show_stack+0x24/0x30
[68531.946649] [<ffffff800845d7e0>] dump_stack+0x98/0xc0
[68531.946654] [<ffffff8008259190>] dump_header+0x6c/0x1b8
[68531.946659] [<ffffff80081c9b88>] oom_kill_process+0x268/0x498
[68531.946663] [<ffffff80081ca264>] out_of_memory+0x1e4/0x308
[68531.946668] [<ffffff80081cfd8c>] __alloc_pages_nodemask+0x844/0xd38
[68531.946673] [<ffffff800820009c>] __pte_alloc+0x3c/0x128
[68531.946678] [<ffffff80082030b4>] alloc_set_pte+0x48c/0x550
[68531.946682] [<ffffff8008204d04>] handle_mm_fault+0x72c/0xa68
[68531.946687] [<ffffff80080a36b0>] do_page_fault+0x308/0x518
[68531.946691] [<ffffff80080a392c>] do_translation_fault+0x6c/0x80
[68531.946694] [<ffffff8008080954>] do_mem_abort+0x54/0xb0
[68531.946698] [<ffffff80080833c8>] el0_da+0x20/0x24
[68531.946700] Mem-Info:
[68531.946709] active_anon:18 inactive_anon:0 isolated_anon:0
active_file:422 inactive_file:1203 isolated_file:0
unevictable:3037 dirty:0 writeback:0 unstable:0
slab_reclaimable:7231 slab_unreclaimable:12573
mapped:157 shmem:0 pagetables:2154 bounce:0
free:116139 free_pcp:162 free_cma:104001
[68531.946716] Node 0 active_anon:72kB inactive_anon:0kB active_file:1688kB inactive_file:4812kB unevictable:12148kB isolated(anon):0kB isolated(file):0kB mapped:628kB dirty:0kB writeback:0kB shmem:0kB shmem_thp: 0kB shmem_pmdmapped: 0kB anon_thp: 0kB writeback_tmp:0kB unstable:0kB pages_scanned:1489 all_unreclaimable? no
[68531.946724] DMA free:442340kB min:22636kB low:28292kB high:33948kB active_anon:72kB inactive_anon:0kB active_file:1492kB inactive_file:4740kB unevictable:0kB writepending:0kB present:2076672kB managed:2042692kB mlocked:0kB slab_reclaimable:876kB slab_unreclaimable:1156kB kernel_stack:416kB pagetables:972kB bounce:0kB free_pcp:648kB local_pcp:0kB free_cma:416004kB
[68531.946726] lowmem_reserve[]: 0 1961 1961 1961
[68531.946743] Normal free:22216kB min:22416kB low:28020kB high:33624kB active_anon:0kB inactive_anon:0kB active_file:72kB inactive_file:72kB unevictable:12148kB writepending:0kB present:2082816kB managed:2008620kB mlocked:0kB slab_reclaimable:28048kB slab_unreclaimable:49136kB kernel_stack:3552kB pagetables:7644kB bounce:0kB free_pcp:0kB local_pcp:0kB free_cma:0kB
[68531.946745] lowmem_reserve[]: 0 0 0 0
[68531.946755] DMA: 250*4kB (MEC) 30*8kB (MEC) 16*16kB (MEC) 384*32kB (UMEC) 1*64kB (E) 2*128kB (EC) 2*256kB (EC) 2*512kB (EC) 3*1024kB (UEC) 7*2048kB (UEC) 100*4096kB (C) = 442648kB
[68531.946798] Normal: 774*4kB (ME) 682*8kB (UME) 204*16kB (UMH) 291*32kB (UMH) 3*64kB (U) 5*128kB (U) 1*256kB (U) 0*512kB 0*1024kB 0*2048kB 0*4096kB = 22216kB
[68531.946835] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
[68531.946837] 4754 total pagecache pages
[68531.946840] 15 pages in swap cache
[68531.946842] Swap cache stats: add 911696, delete 912013, find 669808/950576
[68531.946844] Free swap = 1802344kB
[68531.946846] Total swap = 2025648kB
[68531.946849] 1039872 pages RAM
[68531.946851] 0 pages HighMem/MovableOnly
[68531.946852] 27044 pages reserved
[68531.946854] 118784 pages cma reserved
[68531.946857] [ pid ] uid tgid total_vm rss nr_ptes nr_pmds swapents oom_score_adj name
[68531.946876] [ 1993] 0 1993 8635 0 14 4 222 0 systemd-journal
[68531.946882] [ 2658] 0 2658 3884 0 8 3 387 -1000 systemd-udevd
[68531.946887] [ 3512] 62583 3512 20988 2 11 3 164 0 systemd-timesyn
[68531.946891] [ 3513] 0 3513 1601 0 7 3 128 0 rpcbind
[68531.946895] [ 3514] 102 3514 2639 0 9 4 183 0 systemd-resolve
[68531.946899] [ 3544] 0 3544 1920 2 7 4 793 0 haveged
[68531.946903] [ 3711] 0 3711 78217 0 22 5 428 0 ModemManager
[68531.946907] [ 3714] 109 3714 54992 0 12 4 878 0 rsyslogd
[68531.946911] [ 3796] 0 3796 25003 2 17 3 1993 0 networkd-dispat
[68531.946915] [ 3801] 114 3801 1622 0 7 3 253 0 avahi-daemon
[68531.946918] [ 3855] 0 3855 2603 2 9 3 184 0 systemd-logind
[68531.946922] [ 3924] 0 3924 1660 0 7 4 94 0 cron
[68531.946926] [ 3934] 103 3934 1799 2 8 3 249 -900 dbus-daemon
[68531.946930] [ 4062] 114 4062 1490 0 7 3 85 0 avahi-daemon
[68531.946935] [ 4166] 0 4166 100370 0 31 4 843 0 NetworkManager
[68531.946939] [ 4180] 0 4180 2354 0 9 3 148 0 wpa_supplicant
[68531.946942] [ 4326] 0 4326 57866 0 15 3 227 0 polkitd
[68531.946947] [ 4698] 111 4698 61942 0 23 4 960 0 whoopsie
[68531.946951] [ 4799] 0 4799 324867 0 34 5 129 0 nvs-service
[68531.946955] [ 4822] 0 4822 1654 0 6 3 86 0 nvmemwarning.sh
[68531.946959] [ 4877] 0 4877 1318 0 6 4 33 0 agetty
[68531.946963] [ 4900] 0 4900 1318 0 6 3 35 0 agetty
[68531.946966] [ 4916] 105 4916 2367 0 8 4 125 0 kerneloops
[68531.946970] [ 4923] 0 4923 1374 0 8 3 36 0 agetty
[68531.946974] [ 4924] 0 4924 1654 0 8 4 76 0 nvgetty.sh
[68531.946977] [ 4932] 105 4932 2367 0 9 4 117 0 kerneloops
[68531.946981] [ 4974] 0 4974 2603 0 8 3 199 -1000 sshd
[68531.946985] [ 5042] 0 5042 1318 0 6 3 35 0 getty
[68531.946990] [ 5168] 0 5168 787 0 5 3 38 0 nvphsd
[68531.946994] [ 5218] 0 5218 19489 0 8 3 104 0 nvphsd
[68531.946998] [ 5438] 0 5438 2094 0 9 3 318 0 dhclient
[68531.947002] [ 5868] 1000 5868 3154 2 11 3 336 0 systemd
[68531.947006] [ 5896] 1000 5896 24439 0 15 4 632 0 (sd-pam)
[68531.947010] [ 7596] 0 7596 68456 0 24 4 483 0 packagekitd
[68531.947013] [ 1100] 0 1100 24742 0 33 3 1215 0 nvargus-daemon
[68531.947017] [ 8106] 1000 8106 1810 0 6 4 194 0 tmux: server
[68531.947021] [ 8109] 1000 8109 1990 2 7 3 420 0 bash
[68531.947026] [22561] 0 22561 3106 2 9 3 249 0 sshd
[68531.947029] [22608] 1000 22608 3106 0 9 3 259 0 sshd
[68531.947033] [22609] 1000 22609 2015 2 8 3 455 0 bash
[68531.947040] [24375] 1000 24375 944854 755756 1611 8 34611 0 gst-launch-1.0
[68531.947043] [24386] 0 24386 3033 0 9 4 240 0 sshd
[68531.947047] [24388] 0 24388 478 0 5 3 22 0 sh
[68531.947051] [24389] 0 24389 453 0 5 4 29 0 run-parts
[68531.947055] [24403] 0 24403 478 0 5 3 21 0 91-release-upgr
[68531.947059] [24404] 0 24404 478 0 4 3 22 0 91-release-upgr
[68531.947062] [24405] 0 24405 2876 113 9 3 942 0 lsb_release
[68531.947066] [24406] 0 24406 458 0 5 3 19 0 cut
[68531.947070] [24407] 0 24407 1074 0 4 3 31 0 sshd
[68531.947074] [24408] 0 24408 1654 0 6 3 85 0 nvmemwarning.sh
[68531.947077] [24409] 0 24409 1654 4 6 3 74 0 nvmemwarning.sh
[68531.947080] [24410] 0 24410 1654 0 6 3 78 0 nvmemwarning.sh
[68531.947084] Out of memory: Kill process 24375 (gst-launch-1.0) score 521 or sacrifice child
[68531.955551] Killed process 24375 (gst-launch-1.0) total-vm:3779416kB, anon-rss:0kB, file-rss:3023024kB, shmem-rss:0kB
[68531.998268] oom_reaper: reaped process 24375 (gst-launch-1.0), now anon-rss:0kB, file-rss:3024688kB, shmem-rss:0kB
It would appear to be running out of memory, perhaps due to the file size or video resolution. You can run tegrastats in the background to keep an eye on the memory usage, and mount a swap file if needed.
You could try inserting queue element(s) into the pipeline and limiting the max-size-buffers, and also as a test reducing the video resolution.
qcasey1
September 11, 2019, 6:02pm
6
Gotcha. Swap is on, I had 6 but bumped it up to 8G. When running this I watch tegrastats and htop, but gstreamer never swaps to disk and maxes out the 4G of RAM.
Any way around this? Do you have an example of limiting the max-size-buffers? I only see it on the videotestsrc in the doc
DaneLLL
September 12, 2019, 3:24am
7
Hi,L4T Jetson/r32.2.1 patch - eLinux.org
qcasey1
September 12, 2019, 3:33pm
8
Thanks, that looks promising. How would I go about applying that?
Hi @DaneLLL ,
Did you mean :
Installing a another version of jpeglib as in this link ?
Or may this be tried as well ?
Something else ?
Thanks for clarifying this.
DaneLLL
September 16, 2019, 3:12am
10
Hi,
Installing a another version of jpeglib as in this link?
With the lib, we are able to run
$ gst-launch-1.0 filesrc location= rat_mjpeg_1080p_24fps_Q15.mkv ! <b>matroskademux</b> ! jpegparse ! nvjpegdec ! 'video/x-raw' ! nvvidconv ! 'video/x-raw(memory:NVMM),format=I420' ! omxh264enc ! matroskamux ! filesink location=video-compressed.mkv -e
Or may this be tried as well ?
Applying this patch should bring better performance. It requires extra effort of re-building libgstnvjpeg.so but is worth a try.
qcasey1
September 17, 2019, 4:14pm
11
Thanks @DaneLLL , is there a guide somewhere on how to apply this patch / rebuild libgstnvjpeg.so? Or are there plans to make that mainstream?
DaneLLL
September 18, 2019, 1:58am
12
Hi,
/usr/lib/aarch64-linux-gnu/tegra/libnvjpeg.so
Also I have corrected #10 . It has to have matroskademux in the pipeline.