Video and Audio Inputs Test cmd

BalajiNP · October 19, 2017, 9:43am

Hi,

In Jetson Tx2 board, in Camera module connector, we connect the HDMI to CSI2 bridge board, so we will get the video and audio input from this.

From the below gstreamer cmd, we are able to get the video

gst-launch-1.0 v4l2src ! ‘video/x-raw, width=3840, height=2160, framerate=30/1, format=UYVY’ ! nvvidconv ! ‘video/x-raw(memory:NVMM), width=3840, height=2160, framerate=30/1, format=I420’ ! nvoverlaysink sync=false

How to get the audio input from Gstreamer and how to get th both (video and audio) inputs at the same time with sync one.

Please provide the cmd for this.

-Thanks.

DaneLLL · October 20, 2017, 2:52am

Hi BalajiNP,
We have verified the case of onboard camera + USB Audio input

$ arecord -l
(...skip)
card 2: M [Plantronics Blackwire C220 M], device 0: USB Audio [USB Audio]
  Subdevices: 0/1
  Subdevice #0: subdevice #0

$ gst-launch-1.0 nvcamerasrc num-buffers=300 ! omxh264enc ! queue ! mux. alsasrc num-buffers=1000 device="hw:2,0" ! voaacenc ! queue ! qtmux name=mux ! filesink location=a.mp4

Please check card number and device number of your HDMI audio input and configure device=“hw:2,0” accordingly.

BalajiNP · October 20, 2017, 4:52am

Hi DaneLLL,

Thanks for the reply.

We used the below cmd to test the video input:

gst-launch-1.0 v4l2src ! ‘video/x-raw, width=1920, height=1080, framerate=30/1, format=UYVY’ ! nvvidconv ! ‘video/x-raw(memory:NVMM), width=1920, height=1080, framerate=30/1, format=I420’ ! nvoverlaysink sync=false

We used the v4l2src, not nvcamerasrc. Do you have any cmd based on v4l2src?
Or please provide the cmd to test the video based on nvcamerasrc. since need to confirm the video input first.

-Thanks.

DaneLLL · October 20, 2017, 4:58am

Hi BalajiNP, it should work if you replace nvcamerasrc with v4l2src ! ‘video/x-raw, width=1920, height=1080, framerate=30/1, format=UYVY’ ! nvvidconv ! ‘video/x-raw(memory:NVMM), width=1920, height=1080, framerate=30/1, format=I420’

BalajiNP · October 20, 2017, 5:49pm

Hi DaneLLL,

We tried this below cmd:

gst-launch-1.0 v4l2src device=/dev/video0 ! ‘video/x-raw, width=1920, height=1080, framerate=60/1, format=UYVY’ ! nvvidconv ! ‘video/x-raw(memory:NVMM), format=I420’ ! nvoverlaysink alsasrc ! alsasink device=hw:0,7

Then it is working. Need the cmd to record the video and audio stream in file. Trying this below cmd:

gst-launch-1.0 v4l2src ! ‘video/x-raw, width=1920, height=1080, framerate=30/1, format=UYVY’ ! nvvidconv ! ‘video/x-raw(memory:NVMM), format=I420’ ! queue ! omxh264enc bitrate=8000000 ! h264parse ! matroskamux name=mux ! filesink location=test_h264.mkv -e alsasrc ! queue ! audioconvert ! lamemp3enc ! mux.

It is not working. Trying to change this.

Please provide the proper cmd for this.

-Thanks.

DaneLLL · October 23, 2017, 2:20am

Looks like you figure it out.
[url]https://devtalk.nvidia.com/default/topic/1025470/jetson-tx2/video-and-audio-recording/post/5215657/#5215657[/url]

Andrey1984 · January 23, 2018, 2:12am

any ideas how to overcome abrupt of execution of the command?

$ gst-launch-1.0 nvcamerasrc num-buffers=300 ! omxh264enc ! queue ! mux. alsasrc num-buffers=1000 device="hw:2,0" ! voaacenc ! queue ! qtmux name=mux ! filesink location=a.mp4

Execution ended after 0:00:10.287445982

Reference: https://devtalk.nvidia.com/default/topic/1025470/jetson-tx2/video-and-audio-recording/post/5229487/#5229487

DaneLLL · January 25, 2018, 8:47am

It is more like an issue in the alsasrc, if audiotestsrc runs well.

Other users may share experience about alsasrc.

Andrey1984 · January 25, 2018, 9:10am

audiotestsrc seems to abrupt with error

gst-launch-1.0 audiotestsrc

ERROR: from element /GstPipeline:pipeline0/GstAudioTestSrc:audiotestsrc0: Internal data flow error.

Additional debug info:
gstbasesrc.c(2948): gst_base_src_loop (): /GstPipeline:pipeline0/GstAudioTestSrc:audiotestsrc0:
streaming task paused, reason not-linked (-1)

found some reference : http://gstreamer-devel.966125.n4.nabble.com/gst-launch-1-0-audiotestsrc-voaacenc-mpegtsmux-filesink-location-aac-ts-Internal-data-flow-error-td4672738.html
the following code executes well

gst-launch-1.0 audiotestsrc ! voaacenc ! aacparse ! mpegtsmux ! filesink location=/tmp/aac.ts

Honey_Patouceul · January 29, 2018, 1:42pm

Hi Andrey,

That’s because audiotestsrc has a source pad for outputting, but there is no sink for consuming its outputs. You may just use fakesink:

gst-launch-1.0 audiotestsrc ! fakesink

Andrey1984 · January 29, 2018, 1:49pm

Thank you for the explanation.
However,that doesn’t resolve the abrupt of execution

$ gst-launch-1.0 nvcamerasrc num-buffers=300 ! omxh264enc ! queue ! mux. alsasrc num-buffers=1000 device="hw:2,0" ! voaacenc ! queue ! qtmux name=mux ! filesink location=a.mp4

gst-launch-1.0 nvcamerasrc num-buffers=300 ! omxh264enc ! queue ! mux. alsasrc num-buffers=1000 device="hw:2,0" ! voaacenc ! queue ! qtmux name=mux ! filesink location=a.mp4
Setting pipeline to PAUSED ...

Available Sensor modes : 
2592 x 1944 FR=30.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
2592 x 1458 FR=30.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
1280 x 720 FR=120.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...

NvCameraSrc: Trying To Set Default Camera Resolution. Selected sensorModeIndex = 1 WxH = 2592x1458 FrameRate = 30.000000 ...

New clock: GstAudioSrcClock
Redistribute latency...
Framerate set to : 30 at NvxVideoEncoderSetParameterNvMMLiteOpen : Block : BlockType = 4 
===== MSENC =====
NvMMLiteBlockCreate : Block : BlockType = 4 
===== MSENC blits (mode: 1) into tiled surfaces =====
Got EOS from element "pipeline0".
Execution ended after 0:00:10.188498923
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

Honey_Patouceul · January 29, 2018, 6:37pm

Not sure what you mean with abrupt of execution, and if I correctly understand your problem.

From the log, it seems it successfully records 10 seconds of video and audio as expected (300 frames from camera at 30 fps and 1000 centiseconds of audio).

If you want to get more than 10 seconds, you may increase num-buffers for nvcamerasrc and alsasrc, or remove them and manually send ctrl-c for stopping, but be aware that it may fill your disk if you don’t stop it soon enough. Saving mp4 file on a different disk than rootfs would be safer.

Andrey1984 · January 30, 2018, 4:20am

Exactly! The idea was to record a long video with sound.Thank you for pointing to the buffers deletion.

Andrey1984 · January 30, 2018, 4:23am

gst-launch-1.0 nvcamerasrc ! omxh264enc ! queue ! mux. alsasrc device="hw:2,0" ! voaacenc ! queue ! qtmux name=mux ! filesink location=a.mp4

With buffers deletion it seems to record the file without abruption on 10 seconds, but the file doesn’t appear to be playable. It wont open in video player.
Totem returns : File contain no playable streams.

Honey_Patouceul · January 30, 2018, 7:10am

Seems working in my case:

nvidia@tegra-ubuntu:~$ gst-launch-1.0 -e alsasrc ! voaacenc ! queue ! qtmux name=mux ! filesink location=a.mp4    nvcamerasrc ! omxh264enc ! queue ! mux.
Setting pipeline to PAUSED ...

Available Sensor modes : 
2592 x 1944 FR=30.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
2592 x 1458 FR=30.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
1280 x 720 FR=120.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
Pipeline is live and does not need PREROLL ...

NvCameraSrc: Trying To Set Default Camera Resolution. Selected sensorModeIndex = 1 WxH = 2592x1458 FrameRate = 30.000000 ...

Setting pipeline to PLAYING ...
New clock: GstAudioSrcClock
Redistribute latency...
Framerate set to : 30 at NvxVideoEncoderSetParameterNvMMLiteOpen : Block : BlockType = 4 
===== MSENC =====
NvMMLiteBlockCreate : Block : BlockType = 4 
===== MSENC blits (mode: 1) into tiled surfaces =====
^Chandling interrupt.
Interrupt: Stopping pipeline ...
EOS on shutdown enabled -- Forcing EOS on the pipeline
Waiting for EOS...
Got EOS from element "pipeline0".
EOS received - stopping pipeline...
Execution ended after 0:00:07.466645325
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

nvidia@tegra-ubuntu:~$ mplayer a.mp4 
MPlayer 1.2.1 (Debian), built with gcc-5.3.1 (C) 2000-2016 MPlayer Team
mplayer: could not connect to socket
mplayer: No such file or directory
Failed to open LIRC support. You will not be able to use your remote control.

Playing a.mp4.
libavformat version 56.40.101 (external)
libavformat file format detected.
[mov,mp4,m4a,3gp,3g2,mj2 @ 0x7f7d4b9608]Protocol name not provided, cannot determine if input is local or a network protocol, buffers and access patterns cannot be configured optimally without knowing the protocol
[lavf] stream 0: audio (aac), -aid 0, -alang eng
[lavf] stream 1: video (h264), -vid 0
VIDEO:  [H264]  640x480  24bpp  3000.000 fps  4267.4 kbps (520.9 kbyte/s)
Clip info:
 major_brand: qt  
 minor_version: 537199360
 compatible_brands: qt  
 creation_time: 2018-01-30 07:07:49
Load subtitles in ./
Failed to open VDPAU backend libvdpau_nvidia.so: cannot open shared object file: No such file or directory
[vdpau] Error when calling vdp_device_create_x11: 1
==========================================================================
Opening video decoder: [ffmpeg] FFmpeg's libavcodec codec family
libavcodec version 56.60.100 (external)
Selected video codec: [ffh264] vfm: ffmpeg (FFmpeg H.264)
==========================================================================
==========================================================================
Opening audio decoder: [ffmpeg] FFmpeg/libavcodec audio decoders
AUDIO: 44100 Hz, 2 ch, floatle, 128.0 kbit/4.53% (ratio: 15999->352800)
Selected audio codec: [ffaac] afm: ffmpeg (FFmpeg AAC (MPEG-2/MPEG-4 Audio))
==========================================================================
AO: [pulse] 44100Hz 2ch floatle (4 bytes per sample)
Starting playback...
Movie-Aspect is undefined - no prescaling applied.
VO: [xv] 640x480 => 640x480 Planar YV12 
A:   5.5 V:   5.8 A-V: -0.317 ct: -0.006   0/  0 46%  1%  1.3% 0 0

Andrey1984 · January 30, 2018, 7:23am

@Honey_Patouceul Thank you. Seems that worked for me.