v4lsrc for yuv sensor

Autonomous Machines Jetson & Embedded Systems Jetson TX1
1

Hi,

Recently, I flashed my Jetson TX1 with the latest Jetpack 2.3, and the following command works perfectly for a new yuv sensor

v4l2-ctl -d /dev/video0 -w --verbose --set-fmt-video=width=1280,height=1080,pixelformat=UYVY --set-ctrl bypass_mode=0 --stream-mmap --stream-count=1 --stream-to=test.yuv

I can use this command to capture a normal picture, but when I use the followed gstreamer command to capture the streaming, a error occured

gst-launch-1.0 -v v4l2src device=/dev/video0 ! ‘video/x-raw,format=(string)UYVY,width=(int)1280,(int)height=1080,framerate=(fraction)30/1’ ! filesink location=test.yuv

ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data flow error.

Due to the yuv sensor has a supporting ISP, so I don’t use the TX1 internal ISP, what could I do to successfully run this command?

2

Hi cloudliu
Could you try format=(string)I420 instead of format=(string)UYVY and could you share your modification for the YUV sensor for helping others user can reference to.

3

Why trying format=(string)I420 instead of format=(string)UYVY? The sensor outputs UYVY image data!

4

Suppose the key point is the gst element v4l2src.

5

Hi ShaneCCC,

the followed gstreamer command to capture the streaming not works,

Test environment is jet 3.0 r27 (TX2).

–
nvidia@tegra-ubuntu:~/works$ gst-launch-1.0 -v v4l2src device=/dev/video0 ! ‘video/x-raw, format=(string)NV21, width=(int)1280,(int)height=1080,framerate=(fraction)30/1’ ! filesink location=test.yuv
WARNING: erroneous pipeline: could not parse caps “video/x-raw,\ format=(string)NV21,\ width=(int)1280,(int)height=1080,framerate=(fraction)30/1”

nvidia@tegra-ubuntu:~/works$ gst-launch-1.0 -v v4l2src device=/dev/video0 ! ‘video/x-raw, format=(string)I420, width=(int)1280,(int)height=1080,framerate=(fraction)30/1’ ! filesink location=test.yuv
WARNING: erroneous pipeline: could not parse caps “video/x-raw,\ format=(string)I420,\ width=(int)1280,(int)height=1080,framerate=(fraction)30/1”

nvidia@tegra-ubuntu:~/works$ gst-launch-1.0 -v v4l2src device=/dev/video0 ! ‘video/x-raw,format=(string)UYVY,width=(int)1280,(int)height=1080,framerate=(fraction)30/1’ ! filesink location=test.yuv
WARNING: erroneous pipeline: could not parse caps “video/x-raw,format=(string)UYVY,width=(int)1280,(int)height=1080,framerate=(fraction)30/1”

nvidia@tegra-ubuntu:~/works$ v4l2-ctl -d /dev/video0 -w --verbose --set-fmt-video=width=1280,height=1080,pixelformat=UYVY --set-ctrl bypass_mode=0 --stream-mmap --stream-count=1 --stream-to=test.yuv
VIDIOC_QUERYCAP: ok
VIDIOC_S_EXT_CTRLS: ok
VIDIOC_G_FMT: ok
VIDIOC_S_FMT: failed: Invalid argument
VIDIOC_REQBUFS: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_STREAMON: ok
libv4l2: error dequeuing buf: Input/output error
VIDIOC_DQBUF: failed: Input/output error
VIDIOC_STREAMOFF: ok

6

@johlim
Could you try

gst-launch-1.0 -v v4l2src device="/dev/video0" ! "video/x-raw,width=1920,height=1080, format=(string)I420" ! nvvidconv ! "video/x-raw(memory:NVMM)" ! nvoverlaysink
7

I did like

gst-launch-1.0 -v v4l2src device="/dev/video0" ! "video/x-raw,width=1920,height=1080, format=(string)I420" ! nvvidconv ! "video/x-raw(memory:NVMM)" !  filesink location=yes.bayer
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = "video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ colorimetry\=\(string\)2:4:7:1\,\ framerate\=\(fraction\)30/1"
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = "video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ colorimetry\=\(string\)2:4:7:1\,\ framerate\=\(fraction\)30/1"
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:src: caps = "video/x-raw\(memory:NVMM\)\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ framerate\=\(fraction\)30/1\,\ format\=\(string\)I420"
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = "video/x-raw\(memory:NVMM\)\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ framerate\=\(fraction\)30/1\,\ format\=\(string\)I420"
/GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps = "video/x-raw\(memory:NVMM\)\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ framerate\=\(fraction\)30/1\,\ format\=\(string\)I420"
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps = "video/x-raw\(memory:NVMM\)\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ framerate\=\(fraction\)30/1\,\ format\=\(string\)I420"
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:sink: caps = "video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ colorimetry\=\(string\)2:4:7:1\,\ framerate\=\(fraction\)30/1"
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = "video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ colorimetry\=\(string\)2:4:7:1\,\ framerate\=\(fraction\)30/1"
^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:26.332250198
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

yes.bayer file’ size is 0 byte

8

@johlim
There’s no problem to get data on reference sensor.
Does v4l2-ctl get the frame data normally?
Any error kernel message?

9

I tested with TX2 developement Kit 's Camera Sensor
and I can get a frame data using v4l2-ctl.
Looking at that file as a hexeditor, ff 3f is repeated. (the folled ov1080.raw)
There is no kernel error message

v4l2-ctl -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat=RG10  --set-ctrl bypass_mode=0 --stream-mmap --stream-count=1 --stream-to=ov1080.raw

gst-launch-1.0 -v v4l2src device=/dev/video0 num-buffers=10 ! 'video/x-bayer,format=rggb,width=1920,height=1080' ! identity silent=false ! filesink location=test.bayer

gst-launch-1.0 -v filesrc blocksize=5038848 num-buffers=4 location=test.bayer ! 'video/x-bayer,format=rggb,width=2592,height=1944,framerate=30/1' ! bayer2rgb ! filesink location=test.rgb

4147200 Jul 21 05:07 ov1080.raw
22118400 Jul 21 05:15 test.bayer
80621568 Jul 21 05:20 test.rgb

10

The the reference sensor board have problem at first two maybe three frame, you can try --stream-count=4 then you can see the difference.

11

Thansk
The contents of file ov1080.raw changed from offset 0x3F4800 (skiped 2 frame)

v4l2-ctl -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat=RG10  --set-ctrl bypass_mode=0 --stream-mmap --stream-count=4 --stream-to=ov1080.raw

ov1080.raw
FF 3F FF 3F FF 3F FF 3F FF 3F FF 3F FF 3F FF 3F
FF 3F FF 3F FF 3F FF 3F FF 3F FF 3F FF 3F FF 3F
D0 03 61 06 E0 03 91 06 50 03 C1 06 21 04 91 06 ← offset 0x3F4800
11 04 31 06 11 04 A1 06 11 04 D1 06 21 04 11 07
C0 03 D1 06 E0 03 F1 06 B0 03 C1 06 51 04 F1 06

the test.bayer file is filled with zero

and

I excutes your script (gst-launch-1.0 -v v4l2src device=“/dev/video0” ! “video/x-raw,width=1920,height=1080, format=(string)I420” ! nvvidconv ! “video/x-raw(memory:NVMM)” ! nvoverlaysink) again, but There is no change on the hdmi screen.

nvidia@tegra-ubuntu:~/works$ gst-launch-1.0 -v v4l2src device=/dev/video0 num-buffers=10 ! 'video/x-bayer,format=rggb,width=1920,height=1080' ! identity silent=false ! filesink location=test.bayer
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
/GstPipeline:pipeline0/GstIdentity:identity0: last-message = event   ******* (identity0:sink) E (type: stream-start (10254), GstEventStreamStart, stream-id=(string)2b7f5679e9f0cee0776e7c9d96ffc3b4682b26f7b8931798b4254fc99bdbde9b, flags=(GstStreamFlags)GST_STREAM_FLAG_NONE, group-id=(uint)0;) 0x6260f0
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = "video/x-bayer\,\ format\=\(string\)rggb\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ colorimetry\=\(string\)2:4:7:1\,\ framerate\=\(fraction\)30/1"
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = "video/x-bayer\,\ format\=\(string\)rggb\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ colorimetry\=\(string\)2:4:7:1\,\ framerate\=\(fraction\)30/1"
/GstPipeline:pipeline0/GstIdentity:identity0: last-message = event   ******* (identity0:sink) E (type: caps (12814), GstEventCaps, caps=(GstCaps)"video/x-bayer\,\ format\=\(string\)rggb\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ colorimetry\=\(string\)2:4:7:1\,\ framerate\=\(fraction\)30/1";) 0x6261d0
/GstPipeline:pipeline0/GstIdentity:identity0.GstPad:src: caps = "video/x-bayer\,\ format\=\(string\)rggb\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ colorimetry\=\(string\)2:4:7:1\,\ framerate\=\(fraction\)30/1"
/GstPipeline:pipeline0/GstFileSink:filesink0.GstPad:sink: caps = "video/x-bayer\,\ format\=\(string\)rggb\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ colorimetry\=\(string\)2:4:7:1\,\ framerate\=\(fraction\)30/1"
/GstPipeline:pipeline0/GstIdentity:identity0.GstPad:sink: caps = "video/x-bayer\,\ format\=\(string\)rggb\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ colorimetry\=\(string\)2:4:7:1\,\ framerate\=\(fraction\)30/1"
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = "video/x-bayer\,\ format\=\(string\)rggb\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ colorimetry\=\(string\)2:4:7:1\,\ framerate\=\(fraction\)30/1"
/GstPipeline:pipeline0/GstIdentity:identity0: last-message = event   ******* (identity0:sink) E (type: segment (17934), GstEventSegment, segment=(GstSegment)"GstSegment, flags=(GstSegmentFlags)GST_SEGMENT_FLAG_NONE, rate=(double)1, applied-rate=(double)1, format=(GstFormat)GST_FORMAT_TIME, base=(guint64)0, offset=(guint64)0, start=(guint64)0, stop=(guint64)18446744073709551615, time=(guint64)0, position=(guint64)0, duration=(guint64)18446744073709551615;";) 0x6262b0
/GstPipeline:pipeline0/GstIdentity:identity0: last-message = chain   ******* (identity0:sink) (4147200 bytes, dts: none, pts: 0:00:03.015792121, duration: 0:00:00.033333333, offset: 0, offset_end:  1, flags: 00002040 discont delta-unit ) 0x7f7400f010
/GstPipeline:pipeline0/GstIdentity:identity0: last-message = chain   ******* (identity0:sink) (4147200 bytes, dts: none, pts: 0:00:04.019790145, duration: 0:00:00.033333333, offset: 1, offset_end:  2, flags: 00002000 delta-unit ) 0x7f7400f120
/GstPipeline:pipeline0/GstIdentity:identity0: last-message = chain   ******* (identity0:sink) (4147200 bytes, dts: none, pts: 0:00:05.023889671, duration: 0:00:00.033333333, offset: 2, offset_end:  3, flags: 00002000 delta-unit ) 0x7f7400f230
/GstPipeline:pipeline0/GstIdentity:identity0: last-message = chain   ******* (identity0:sink) (4147200 bytes, dts: none, pts: 0:00:06.027787403, duration: 0:00:00.033333333, offset: 3, offset_end:  4, flags: 00002000 delta-unit ) 0x7f7400f340
/GstPipeline:pipeline0/GstIdentity:identity0: last-message = chain   ******* (identity0:sink) (4147200 bytes, dts: none, pts: 0:00:07.031855743, duration: 0:00:00.033333333, offset: 4, offset_end:  5, flags: 00002000 delta-unit ) 0x7f7400f010
/GstPipeline:pipeline0/GstIdentity:identity0: last-message = chain   ******* (identity0:sink) (4147200 bytes, dts: none, pts: 0:00:08.035843840, duration: 0:00:00.033333333, offset: 5, offset_end:  6, flags: 00002000 delta-unit ) 0x7f7400f120
/GstPipeline:pipeline0/GstIdentity:identity0: last-message = chain   ******* (identity0:sink) (4147200 bytes, dts: none, pts: 0:00:09.039864018, duration: 0:00:00.033333333, offset: 6, offset_end:  7, flags: 00002000 delta-unit ) 0x7f7400f230
/GstPipeline:pipeline0/GstIdentity:identity0: last-message = chain   ******* (identity0:sink) (4147200 bytes, dts: none, pts: 0:00:10.043839848, duration: 0:00:00.033333333, offset: 7, offset_end:  8, flags: 00002000 delta-unit ) 0x7f7400f340
/GstPipeline:pipeline0/GstIdentity:identity0: last-message = chain   ******* (identity0:sink) (4147200 bytes, dts: none, pts: 0:00:11.047803218, duration: 0:00:00.033333333, offset: 8, offset_end:  9, flags: 00002000 delta-unit ) 0x7f7400f010
/GstPipeline:pipeline0/GstIdentity:identity0: last-message = chain   ******* (identity0:sink) (4147200 bytes, dts: none, pts: 0:00:12.051807069, duration: 0:00:00.033333333, offset: 9, offset_end:  10, flags: 00002000 delta-unit ) 0x7f7400f120
/GstPipeline:pipeline0/GstIdentity:identity0: last-message = event   ******* (identity0:sink) E (type: eos (28174), ) 0x626320
Got EOS from element "pipeline0".
Execution ended after 0:00:12.119787767
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

nvidia@tegra-ubuntu:~/works$ gst-launch-1.0 -v v4l2src device="/dev/video0" ! "video/x-raw,width=1920,height=1080, format=(string)I420" ! nvvidconv ! "video/x-raw(memory:NVMM)" ! nvoverlaysink
Setting pipeline to PAUSED ...
Inside NvxLiteH264DecoderLowLatencyInitNvxLiteH264DecoderLowLatencyInit set DPB and MjstreamingInside NvxLiteH265DecoderLowLatencyInitNvxLiteH265DecoderLowLatencyInit set DPB and MjstreamingPipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstV4l2Src:v4l2src0.GstPad:src: caps = "video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ colorimetry\=\(string\)2:4:7:1\,\ framerate\=\(fraction\)30/1"
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = "video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ colorimetry\=\(string\)2:4:7:1\,\ framerate\=\(fraction\)30/1"
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:src: caps = "video/x-raw\(memory:NVMM\)\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ framerate\=\(fraction\)30/1\,\ format\=\(string\)I420"
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:src: caps = "video/x-raw\(memory:NVMM\)\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ framerate\=\(fraction\)30/1\,\ format\=\(string\)I420"
/GstPipeline:pipeline0/GstNvOverlaySink-nvoverlaysink:nvoverlaysink-nvoverlaysink0.GstPad:sink: caps = "video/x-raw\(memory:NVMM\)\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ framerate\=\(fraction\)30/1\,\ format\=\(string\)I420"
/GstPipeline:pipeline0/GstCapsFilter:capsfilter1.GstPad:sink: caps = "video/x-raw\(memory:NVMM\)\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ framerate\=\(fraction\)30/1\,\ format\=\(string\)I420"
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:sink: caps = "video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ colorimetry\=\(string\)2:4:7:1\,\ framerate\=\(fraction\)30/1"
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = "video/x-raw\,\ format\=\(string\)I420\,\ width\=\(int\)1920\,\ height\=\(int\)1080\,\ pixel-aspect-ratio\=\(fraction\)1/1\,\ interlace-mode\=\(string\)progressive\,\ colorimetry\=\(string\)2:4:7:1\,\ framerate\=\(fraction\)30/1"
12

@johlim
That’s as expect if you try those two gst-launch command with reference sensor board, because it’s bayer sensor these two command are not go to ISP so it couldn’t have problem show on the HDMI or dump to file. But for the YUV sensor it should have no problem with these two command.