Video playback: Full-screen mode vs. Windowed mode

Hi everyone, we followed L4T_Jetson_TK1_Multimedia_User_Guide_V1.2.pdf and found some problems when building our customized media player on Jetson TK1. Here’s the comparison of full-screen mode and windowed mode when playing the same 2160p 30fps h264 encoded video:

Full-screen Mode*:
CPU usage: 5%
comment: very smooth

Windowed Mode**:
CPU usage: 70%
comment: showing “WARNING: from element /GstPipeline:pipeline0/GstXvImageSink:xvimagesink0: A lot of buffers are being dropped.”

We want to make our media player under windowed mode. Does anyone have any idea what’s the difference between HDMI Overlay Sink and Xvimagesink, or what’s the right way to construct pipeline? Thanks.

*pipeline: gst-launch-0.10 filesrc location=<filename.mp4> ! qtdemux name=demux
demux.video_00 ! queue ! nv_omx_h264dec ! nv_omx_hdmi_videosink –v -e

**pipeline: gst-launch-0.10 filesrc location=<filename.mp4> ! qtdemux name=demux
demux.video_00 ! queue ! nv_omx_h264dec ! ‘video/x-nv-yuv’ ! nvvidconv
! xvimagesink –v -e

Try this:

gst-launch-1.0 -v -e uridecodebin uri=file:///path/to/the/video.mp4 ! nveglglessink

Hi kulve,
I tried the pipeline you mentioned but it shows

ubuntu@tegra-ubuntu:~/Videos$ gst-launch-1.0 -v -e uridecodebin uri=file:///home/ubuntu/Videos/2160p.mp4 ! nveglglessink
Setting pipeline to PAUSED ...
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0: source = "\(GstFileSrc\)\ source"
Pipeline is PREROLLING ...
Got context from element 'eglglessink0': gst.egl.EGLDisplay=context, display=(GstEGLDisplay)NULL;
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.GstPad:src: caps = video/quicktime, variant=(string)iso
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstTypeFindElement:typefind.GstPad:src: caps = NULL
Missing element: H.264 decoder
WARNING: from element /GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0: No decoder available for type 'video/x-h264, stream-format=(string)avc, alignment=(string)au, level=(string)5.1, profile=(string)high, codec_data=(buffer)01640033ffe1001a67640033acd300f0010fb0110000030001000003003c0f18313801000468cebcb0, width=(int)3840, height=(int)2160, framerate=(fraction)30/1, pixel-aspect-ratio=(fraction)1/1'.
Additional debug info:
gsturidecodebin.c(930): unknown_type_cb (): /GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-buffers = 5
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-time = 0
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-bytes = 2097152
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0.GstPad:sink_0: caps = audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)1210, rate=(int)44100, channels=(int)2
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-buffers = 5
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-time = 0
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-bytes = 2097152
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0.GstPad:src_0: caps = audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)1210, rate=(int)44100, channels=(int)2
Missing element: MPEG-4 AAC decoder
WARNING: from element /GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0: No decoder available for type 'audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)1210, rate=(int)44100, channels=(int)2'.
Additional debug info:
gsturidecodebin.c(930): unknown_type_cb (): /GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-buffers = 5
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-time = 0
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-bytes = 2097152
ERROR: from element /GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0: Your GStreamer installation is missing a plug-in.
Additional debug info:
gstdecodebin2.c(3928): gst_decode_bin_expose (): /GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0:
no suitable plugins found
ERROR: pipeline doesn't want to preroll.
Setting pipeline to NULL ...
Caught SIGSEGV
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-buffers = 5
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-time = 0
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstMultiQueue:multiqueue0: max-size-bytes = 2097152
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstAacParse:aacparse0.GstPad:src: caps = audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)1210, rate=(int)44100, channels=(int)2
/GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstDecodeBin:decodebin0/GstAacParse:aacparse0.GstPad:sink: caps = audio/mpeg, mpegversion=(int)4, framed=(boolean)true, stream-format=(string)raw, level=(string)2, base-profile=(string)lc, profile=(string)lc, codec_data=(buffer)1210, rate=(int)44100, channels=(int)2
#0  __libc_do_syscall ()
#1  0xb6d82ec2 in pthread_join (threadid=<optimized out>, thread_return=0x0)
#2  0xb6dfc6a6 in ?? () from /lib/arm-linux-gnueabihf/libglib-2.0.so.0
Spinning.  Please run 'gdb gst-launch-1.0 2727' to continue debugging, Ctrl-C to quit, or Ctrl-\ to dump core.

and ‘gdb gst-launch-1.0 2727’ shows that

Loaded symbols for /usr/lib/arm-linux-gnueabihf/gstreamer-1.0/libgstaudioparsers.so
__libc_do_syscall () at ../ports/sysdeps/unix/sysv/linux/arm/libc-do-syscall.S:43
43	../ports/sysdeps/unix/sysv/linux/arm/libc-do-syscall.S: No such file or directory.

Did I miss anything? Thanks for your help!

Oops… it seems that I didn’t install gstreamer-1.0 :p. I fixed it by
sudo apt-get install gstreamer1.0*

Now I can play H264 2160p 30fps video smoothly in windowed mode with 20% CPU usage by using the pipeline kulve mentioned.

Thank you, kulve!

Hi Nick,
If you only want to play videos at particular location with fixed size, you can do so with nv_omx_hdmi_videosink too by specifying the correct coordinates to the properties:
overlay-x (x coordinate)
overlay-y (y coordinate)
overlay-h (window height)
overlay-w (window width)

If you want windowed playback, nveglglessink is the way to go.
Regards,
Rahool

hi rpaliwal_nvidia,

i want to play videos at particular location with fixed size, and i can do so with nv_omx_hdmi_videosink

code:
gst-lauch-0.10
filesrc location=“filename.mov”
! qtdemux
! queue
! nv_omx_h264dec
! nv_omx_hdmi_videosink overlay-x=280 overlay-y=200 overlay-w=640 overlay-h=480
sync=false

that code is ok with one video window
if i want play many video windows, maybe 4 video windows or 8 video windows
how i want to do it?

thanks
Mark Pei

I think you should be able to run multiple gst-launch-0.10 commands with different overlay coordinates to nv_omx_hdmi_videosink.

EDIT: please don’t discuss about one issue in multiple threads.

hi kulve,

i also try it with different overlay coordinates to nv_omx_hdmi_videosink, but it not run

https://devtalk.nvidia.com/default/topic/821241/tk1-gstreamer-h264/#4494837