Multimedia API scale + encode

Tessier · April 19, 2017, 4:19pm

Hi,

I have modified Multimedia API sample 07 (video_convert) to implement a scaler + H265 encoder setup. All buffers are pitch linear, the converter scales from 1920x1080 to 1280x720, the input is IUV420M. Buffer handling between the capture plane of the converter and the output plane of the encoder is exactly the same as in the sample.

What I see is that only the first ~10 frames are encoded (output is fine), because the encoder’s output plane callback function is stoppped, so it never frees up the buffers. There are no errors, the application just hangs because of the deadlock situation.

Any idea what could cause this?

Thanks.

Detailed debug log:

[DEBUG] (NvV4l2Element.cpp:62) <conv0> :Opened, fd = 7
Failed to query video capabilities: Inappropriate ioctl for device
[DEBUG] (NvV4l2Element.cpp:62) <enc1> :Opened, fd = 8
[DEBUG] (NvV4l2Element.cpp:182) <conv0> :Set controls
[DEBUG] (NvVideoConverter.cpp:200) <conv0> :Setting output plane buffer layout to 0: success
[DEBUG] (NvV4l2ElementPlane.cpp:290) <conv0> :Output Plane:VIDIOC_S_FMT at capture plane successful
[DEBUG] (NvVideoConverter.cpp:143) <conv0> :Setting output plane format: success
[DEBUG] (NvV4l2Element.cpp:182) <conv0> :Set controls
[DEBUG] (NvVideoConverter.cpp:220) <conv0> :Setting capture plane buffer layout to 0: success
[DEBUG] (NvV4l2ElementPlane.cpp:290) <conv0> :Capture Plane:VIDIOC_S_FMT at capture plane successful
[DEBUG] (NvVideoConverter.cpp:114) <conv0> :Setting capture plane format: success
1920 1080 1280 720
[DEBUG] (NvV4l2ElementPlane.cpp:290) <enc1> :Capture Plane:VIDIOC_S_FMT at capture plane successful
NvMMLiteOpen : Block : BlockType = 8
===== MSENC =====
NvMMLiteBlockCreate : Block : BlockType = 8
[DEBUG] (NvV4l2ElementPlane.cpp:290) <enc1> :Output Plane:VIDIOC_S_FMT at capture plane successful
892744264
842091865
[DEBUG] (NvV4l2Element.cpp:182) <enc1> :Set controls
[DEBUG] (NvVideoEncoder.cpp:187) <enc1> :Setting encoder bitrate to 4000000: success
[DEBUG] (NvV4l2Element.cpp:182) <enc1> :Set controls
[DEBUG] (NvVideoEncoder.cpp:221) <enc1> :Setting encoder profile to 0: success
[DEBUG] (NvV4l2ElementPlane.cpp:509) <enc1> :Output Plane:Successfully set stream parameters
[DEBUG] (NvVideoEncoder.cpp:163) <enc1> :Setting framerate to 30/1: success
libv4l2_nvvidconv (0):(761) (INFO) : Allocating (10) OUTPUT PLANE BUFFERS Layout=0
[DEBUG] (NvV4l2ElementPlane.cpp:433) <conv0> :Output Plane:Reqbuf returned 10 buffers
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Output Plane:QueryBuf for 0th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 0, Plane 0, fd = 1026
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 0, Plane 1, fd = 1026
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 0, Plane 2, fd = 1026
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 0, plane 0 to
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 0, plane 1 to
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 0, plane 2 to
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Output Plane:QueryBuf for 1th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 1, Plane 0, fd = 1027
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 1, Plane 1, fd = 1027
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 1, Plane 2, fd = 1027
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 1, plane 0 to
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 1, plane 1 to
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 1, plane 2 to
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Output Plane:QueryBuf for 2th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 2, Plane 0, fd = 1028
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 2, Plane 1, fd = 1028
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 2, Plane 2, fd = 1028
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 2, plane 0 to
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 2, plane 1 to
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 2, plane 2 to
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Output Plane:QueryBuf for 3th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 3, Plane 0, fd = 1029
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 3, Plane 1, fd = 1029
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 3, Plane 2, fd = 1029
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 3, plane 0 to
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 3, plane 1 to
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 3, plane 2 to
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Output Plane:QueryBuf for 4th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 4, Plane 0, fd = 1030
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 4, Plane 1, fd = 1030
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 4, Plane 2, fd = 1030
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 4, plane 0 to
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 4, plane 1 to
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 4, plane 2 to
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Output Plane:QueryBuf for 5th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 5, Plane 0, fd = 1031
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 5, Plane 1, fd = 1031
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 5, Plane 2, fd = 1031
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 5, plane 0 to
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 5, plane 1 to
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 5, plane 2 to
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Output Plane:QueryBuf for 6th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 6, Plane 0, fd = 1032
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 6, Plane 1, fd = 1032
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 6, Plane 2, fd = 1032
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 6, plane 0 to
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 6, plane 1 to
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 6, plane 2 to
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Output Plane:QueryBuf for 7th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 7, Plane 0, fd = 1033
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 7, Plane 1, fd = 1033
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 7, Plane 2, fd = 1033
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 7, plane 0 to
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 7, plane 1 to
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 7, plane 2 to
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Output Plane:QueryBuf for 8th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 8, Plane 0, fd = 1034
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 8, Plane 1, fd = 1034
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 8, Plane 2, fd = 1034
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 8, plane 0 to
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 8, plane 1 to
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 8, plane 2 to
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Output Plane:QueryBuf for 9th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 9, Plane 0, fd = 1035
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 9, Plane 1, fd = 1035
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 9, Plane 2, fd = 1035
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 9, plane 0 to
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 9, plane 1 to
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 9, plane 2 to
libv4l2_nvvidconv (0):(771) (INFO) : Allocating (10) CAPTURE PLANE BUFFERS Layout=0
[DEBUG] (NvV4l2ElementPlane.cpp:433) <conv0> :Capture Plane:Reqbuf returned 10 buffers
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Capture Plane:QueryBuf for 0th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 0, Plane 0, fd = 1036
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 0, Plane 1, fd = 1036
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 0, Plane 2, fd = 1036
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Capture Plane:QueryBuf for 1th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 1, Plane 0, fd = 1037
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 1, Plane 1, fd = 1037
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 1, Plane 2, fd = 1037
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Capture Plane:QueryBuf for 2th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 2, Plane 0, fd = 1038
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 2, Plane 1, fd = 1038
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 2, Plane 2, fd = 1038
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Capture Plane:QueryBuf for 3th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 3, Plane 0, fd = 1039
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 3, Plane 1, fd = 1039
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 3, Plane 2, fd = 1039
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Capture Plane:QueryBuf for 4th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 4, Plane 0, fd = 1040
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 4, Plane 1, fd = 1040
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 4, Plane 2, fd = 1040
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Capture Plane:QueryBuf for 5th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 5, Plane 0, fd = 1041
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 5, Plane 1, fd = 1041
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 5, Plane 2, fd = 1041
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Capture Plane:QueryBuf for 6th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 6, Plane 0, fd = 1042
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 6, Plane 1, fd = 1042
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 6, Plane 2, fd = 1042
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Capture Plane:QueryBuf for 7th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 7, Plane 0, fd = 1043
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 7, Plane 1, fd = 1043
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 7, Plane 2, fd = 1043
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Capture Plane:QueryBuf for 8th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 8, Plane 0, fd = 1044
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 8, Plane 1, fd = 1044
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 8, Plane 2, fd = 1044
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Capture Plane:QueryBuf for 9th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 9, Plane 0, fd = 1045
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 9, Plane 1, fd = 1045
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 9, Plane 2, fd = 1045
[DEBUG] (NvV4l2ElementPlane.cpp:433) <enc1> :Output Plane:Reqbuf returned 6 buffers
[DEBUG] (NvV4l2ElementPlane.cpp:433) <enc1> :Capture Plane:Reqbuf returned 6 buffers
[DEBUG] (NvV4l2ElementPlane.cpp:542) <enc1> :Capture Plane:QueryBuf for 0th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <enc1> :Capture Plane:ExportBuf successful for Buffer 0, Plane 0, fd = 1046
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 0, plane 0 to
[DEBUG] (NvV4l2ElementPlane.cpp:542) <enc1> :Capture Plane:QueryBuf for 1th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <enc1> :Capture Plane:ExportBuf successful for Buffer 1, Plane 0, fd = 1047
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 1, plane 0 to
[DEBUG] (NvV4l2ElementPlane.cpp:542) <enc1> :Capture Plane:QueryBuf for 2th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <enc1> :Capture Plane:ExportBuf successful for Buffer 2, Plane 0, fd = 1048
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 2, plane 0 to
[DEBUG] (NvV4l2ElementPlane.cpp:542) <enc1> :Capture Plane:QueryBuf for 3th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <enc1> :Capture Plane:ExportBuf successful for Buffer 3, Plane 0, fd = 1049
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 3, plane 0 to
[DEBUG] (NvV4l2ElementPlane.cpp:542) <enc1> :Capture Plane:QueryBuf for 4th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <enc1> :Capture Plane:ExportBuf successful for Buffer 4, Plane 0, fd = 1050
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 4, plane 0 to
[DEBUG] (NvV4l2ElementPlane.cpp:542) <enc1> :Capture Plane:QueryBuf for 5th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <enc1> :Capture Plane:ExportBuf successful for Buffer 5, Plane 0, fd = 1051
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 5, plane 0 to
[DEBUG] (NvV4l2ElementPlane.cpp:467) <conv0> :Output Plane:STREAMON successful
[DEBUG] (NvV4l2ElementPlane.cpp:467) <conv0> :Capture Plane:STREAMON successful
[DEBUG] (NvV4l2ElementPlane.cpp:467) <enc1> :Output Plane:STREAMON successful
[DEBUG] (NvV4l2ElementPlane.cpp:467) <enc1> :Capture Plane:STREAMON successful
[DEBUG] (NvV4l2ElementPlane.cpp:781) <conv0> :Capture Plane:Started DQ Thread
[DEBUG] (NvV4l2ElementPlane.cpp:781) <enc1> :Output Plane:Started DQ Thread
[DEBUG] (NvV4l2ElementPlane.cpp:781) <enc1> :Capture Plane:Started DQ Thread
[DEBUG] (NvV4l2ElementPlane.cpp:716) <conv0> :Capture Plane:Starting DQthread
[DEBUG] (NvV4l2ElementPlane.cpp:716) <enc1> :Capture Plane:Starting DQthread
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Capture Plane:Qed buffer 0
[DEBUG] (NvV4l2ElementPlane.cpp:716) <enc1> :Output Plane:Starting DQthread
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Capture Plane:Qed buffer 1
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Capture Plane:Qed buffer 2
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Capture Plane:Qed buffer 3
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Capture Plane:Qed buffer 4
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Capture Plane:Qed buffer 5
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Capture Plane:Qed buffer 6
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Capture Plane:Qed buffer 7
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Capture Plane:Qed buffer 8
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Capture Plane:Qed buffer 9
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Capture Plane:Qed buffer 0
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Capture Plane:Qed buffer 1
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Capture Plane:Qed buffer 2
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Capture Plane:Qed buffer 3
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Capture Plane:Qed buffer 4
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Capture Plane:Qed buffer 5
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 0
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Capture Plane:DQed buffer 0
CONV CAP DQ callback
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Output Plane:Qed buffer 0
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 1
===== NVENC blits (mode: 1) into block linear surfaces =====
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Capture Plane:DQed buffer 1
CONV CAP DQ callback
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Output Plane:Qed buffer 1
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 2
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Capture Plane:DQed buffer 2
CONV CAP DQ callback
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Output Plane:Qed buffer 2
[DEBUG] (NvV4l2ElementPlane.cpp:151) <enc1> :Output Plane:DQed buffer 0
[DEBUG] (NvV4l2ElementPlane.cpp:151) <enc1> :Capture Plane:DQed buffer 0
ENC OUT DQ callback
ENC CAP DQ callback
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Capture Plane:Qed buffer 0
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Capture Plane:Qed buffer 0
[DEBUG] (NvV4l2ElementPlane.cpp:763) <enc1> :Output Plane:Exiting DQthread
[DEBUG] (NvV4l2ElementPlane.cpp:151) <enc1> :Capture Plane:DQed buffer 1
ENC CAP DQ callback
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 3
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Capture Plane:Qed buffer 1
[DEBUG] (NvV4l2ElementPlane.cpp:151) <enc1> :Capture Plane:DQed buffer 2
ENC CAP DQ callback
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Capture Plane:Qed buffer 2
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Capture Plane:DQed buffer 3
CONV CAP DQ callback
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Output Plane:Qed buffer 3
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 4
[DEBUG] (NvV4l2ElementPlane.cpp:151) <enc1> :Capture Plane:DQed buffer 3
ENC CAP DQ callback
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Capture Plane:Qed buffer 3
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Capture Plane:DQed buffer 4
CONV CAP DQ callback
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Output Plane:Qed buffer 4
[DEBUG] (NvV4l2ElementPlane.cpp:151) <enc1> :Capture Plane:DQed buffer 4
ENC CAP DQ callback
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Capture Plane:Qed buffer 4
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 5
[DEBUG] (NvV4l2ElementPlane.cpp:151) <enc1> :Capture Plane:DQed buffer 5
ENC CAP DQ callback
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Capture Plane:Qed buffer 5
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Capture Plane:DQed buffer 5
CONV CAP DQ callback
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Output Plane:Qed buffer 5
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 6
[DEBUG] (NvV4l2ElementPlane.cpp:151) <enc1> :Capture Plane:DQed buffer 0
ENC CAP DQ callback
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Capture Plane:Qed buffer 0
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Capture Plane:DQed buffer 6
CONV CAP DQ callback
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Output Plane:Qed buffer 0
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 7
[DEBUG] (NvV4l2ElementPlane.cpp:151) <enc1> :Capture Plane:DQed buffer 1
ENC CAP DQ callback
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Capture Plane:Qed buffer 1
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Capture Plane:DQed buffer 7
CONV CAP DQ callback
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 8
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 9
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Output Plane:DQed buffer 0
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 0
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Output Plane:DQed buffer 1
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 1
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Output Plane:DQed buffer 2
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 2
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Output Plane:DQed buffer 3
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 3
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Output Plane:DQed buffer 4
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 4
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Output Plane:DQed buffer 5
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 5
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Output Plane:DQed buffer 6
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 6
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Output Plane:DQed buffer 7
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 7
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Output Plane:DQed buffer 8
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 8
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Output Plane:DQed buffer 9
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 9
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Output Plane:DQed buffer 0
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 0

DaneLLL · April 20, 2017, 2:04am

hi Tessier,
Here is a sample demonatrating scale + encode:
[url]NVIDIA Multimedia APIs with UYVY sensor - Jetson TX1 - NVIDIA Developer Forums

Please refer to it and check what could be missing.

Tessier · April 24, 2017, 1:02pm

Hi Dane,

Thanks for the link, but that setup does not contain the handshake I need (for benchmarking), it assumes that camera input is always slower than processing.

In the meantime I figured out that the encoder’s output plane callback is stopped because v4l2_buf->m.planes[0].bytesused is 0. But I couldn’t find out the reason, why. Uncommenting the check in the encoder’s output callback makes it work. But this lead to another questions.

The main loop calls blocking dqBuffer, then reads a frame from file, then calls qBuffer. This is ok and works correctly.
However, if I remove the frame read and put v4l2_buf.m.planes[0].bytesused=1 instead of it (to lower file read overhead) the pipeline hangs because the converter capture thread exits because of byteused==0. I do not really understand why… What does frame read additionally do? Debug log:

[DEBUG] (NvV4l2Element.cpp:62) <conv0> :Opened, fd = 7
Failed to query video capabilities: Inappropriate ioctl for device
[DEBUG] (NvV4l2Element.cpp:62) <enc1> :Opened, fd = 8
[DEBUG] (NvV4l2Element.cpp:182) <conv0> :Set controls
[DEBUG] (NvVideoConverter.cpp:200) <conv0> :Setting output plane buffer layout to 0: success
[DEBUG] (NvV4l2ElementPlane.cpp:290) <conv0> :Output Plane:VIDIOC_S_FMT at capture plane successful
[DEBUG] (NvVideoConverter.cpp:143) <conv0> :Setting output plane format: success
[DEBUG] (NvV4l2Element.cpp:182) <conv0> :Set controls
[DEBUG] (NvVideoConverter.cpp:220) <conv0> :Setting capture plane buffer layout to 1: success
[DEBUG] (NvV4l2ElementPlane.cpp:290) <conv0> :Capture Plane:VIDIOC_S_FMT at capture plane successful
[DEBUG] (NvVideoConverter.cpp:114) <conv0> :Setting capture plane format: success
320 180 1920 1080
[DEBUG] (NvV4l2ElementPlane.cpp:290) <enc1> :Capture Plane:VIDIOC_S_FMT at capture plane successful
NvMMLiteOpen : Block : BlockType = 8
===== MSENC =====
NvMMLiteBlockCreate : Block : BlockType = 8
[DEBUG] (NvV4l2ElementPlane.cpp:290) <enc1> :Output Plane:VIDIOC_S_FMT at capture plane successful
892744264
842091865
[DEBUG] (NvV4l2Element.cpp:182) <enc1> :Set controls
[DEBUG] (NvVideoEncoder.cpp:187) <enc1> :Setting encoder bitrate to 4000000: success
[DEBUG] (NvV4l2Element.cpp:182) <enc1> :Set controls
[DEBUG] (NvVideoEncoder.cpp:221) <enc1> :Setting encoder profile to 0: success
[DEBUG] (NvV4l2ElementPlane.cpp:509) <enc1> :Output Plane:Successfully set stream parameters
[DEBUG] (NvVideoEncoder.cpp:163) <enc1> :Setting framerate to 60/1: success
libv4l2_nvvidconv (0):(761) (INFO) : Allocating (10) OUTPUT PLANE BUFFERS Layout=0
[DEBUG] (NvV4l2ElementPlane.cpp:433) <conv0> :Output Plane:Reqbuf returned 10 buffers
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Output Plane:QueryBuf for 0th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 0, Plane 0, fd = 1026
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 0, plane 0 to
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Output Plane:QueryBuf for 1th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 1, Plane 0, fd = 1027
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 1, plane 0 to
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Output Plane:QueryBuf for 2th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 2, Plane 0, fd = 1028
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 2, plane 0 to
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Output Plane:QueryBuf for 3th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 3, Plane 0, fd = 1029
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 3, plane 0 to
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Output Plane:QueryBuf for 4th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 4, Plane 0, fd = 1030
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 4, plane 0 to
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Output Plane:QueryBuf for 5th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 5, Plane 0, fd = 1031
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 5, plane 0 to
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Output Plane:QueryBuf for 6th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 6, Plane 0, fd = 1032
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 6, plane 0 to
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Output Plane:QueryBuf for 7th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 7, Plane 0, fd = 1033
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 7, plane 0 to
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Output Plane:QueryBuf for 8th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 8, Plane 0, fd = 1034
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 8, plane 0 to
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Output Plane:QueryBuf for 9th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Output Plane:ExportBuf successful for Buffer 9, Plane 0, fd = 1035
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 9, plane 0 to
libv4l2_nvvidconv (0):(771) (INFO) : Allocating (10) CAPTURE PLANE BUFFERS Layout=1
[DEBUG] (NvV4l2ElementPlane.cpp:433) <conv0> :Capture Plane:Reqbuf returned 10 buffers
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Capture Plane:QueryBuf for 0th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 0, Plane 0, fd = 2097153036
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 0, Plane 1, fd = 2097153036
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 0, Plane 2, fd = 2097153036
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Capture Plane:QueryBuf for 1th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 1, Plane 0, fd = 2097153037
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 1, Plane 1, fd = 2097153037
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 1, Plane 2, fd = 2097153037
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Capture Plane:QueryBuf for 2th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 2, Plane 0, fd = 2097153038
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 2, Plane 1, fd = 2097153038
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 2, Plane 2, fd = 2097153038
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Capture Plane:QueryBuf for 3th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 3, Plane 0, fd = 2097153039
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 3, Plane 1, fd = 2097153039
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 3, Plane 2, fd = 2097153039
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Capture Plane:QueryBuf for 4th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 4, Plane 0, fd = 2097153040
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 4, Plane 1, fd = 2097153040
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 4, Plane 2, fd = 2097153040
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Capture Plane:QueryBuf for 5th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 5, Plane 0, fd = 2097153041
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 5, Plane 1, fd = 2097153041
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 5, Plane 2, fd = 2097153041
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Capture Plane:QueryBuf for 6th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 6, Plane 0, fd = 2097153042
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 6, Plane 1, fd = 2097153042
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 6, Plane 2, fd = 2097153042
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Capture Plane:QueryBuf for 7th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 7, Plane 0, fd = 2097153043
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 7, Plane 1, fd = 2097153043
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 7, Plane 2, fd = 2097153043
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Capture Plane:QueryBuf for 8th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 8, Plane 0, fd = 2097153044
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 8, Plane 1, fd = 2097153044
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 8, Plane 2, fd = 2097153044
[DEBUG] (NvV4l2ElementPlane.cpp:542) <conv0> :Capture Plane:QueryBuf for 9th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 9, Plane 0, fd = 2097153045
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 9, Plane 1, fd = 2097153045
[DEBUG] (NvV4l2ElementPlane.cpp:580) <conv0> :Capture Plane:ExportBuf successful for Buffer 9, Plane 2, fd = 2097153045
[DEBUG] (NvV4l2ElementPlane.cpp:433) <enc1> :Output Plane:Reqbuf returned 6 buffers
[DEBUG] (NvV4l2ElementPlane.cpp:433) <enc1> :Capture Plane:Reqbuf returned 6 buffers
[DEBUG] (NvV4l2ElementPlane.cpp:542) <enc1> :Capture Plane:QueryBuf for 0th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <enc1> :Capture Plane:ExportBuf successful for Buffer 0, Plane 0, fd = 1046
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 0, plane 0 to
[DEBUG] (NvV4l2ElementPlane.cpp:542) <enc1> :Capture Plane:QueryBuf for 1th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <enc1> :Capture Plane:ExportBuf successful for Buffer 1, Plane 0, fd = 1047
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 1, plane 0 to
[DEBUG] (NvV4l2ElementPlane.cpp:542) <enc1> :Capture Plane:QueryBuf for 2th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <enc1> :Capture Plane:ExportBuf successful for Buffer 2, Plane 0, fd = 1048
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 2, plane 0 to
[DEBUG] (NvV4l2ElementPlane.cpp:542) <enc1> :Capture Plane:QueryBuf for 3th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <enc1> :Capture Plane:ExportBuf successful for Buffer 3, Plane 0, fd = 1049
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 3, plane 0 to
[DEBUG] (NvV4l2ElementPlane.cpp:542) <enc1> :Capture Plane:QueryBuf for 4th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <enc1> :Capture Plane:ExportBuf successful for Buffer 4, Plane 0, fd = 1050
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 4, plane 0 to
[DEBUG] (NvV4l2ElementPlane.cpp:542) <enc1> :Capture Plane:QueryBuf for 5th buffer successful
[DEBUG] (NvV4l2ElementPlane.cpp:580) <enc1> :Capture Plane:ExportBuf successful for Buffer 5, Plane 0, fd = 1051
[DEBUG] (NvBuffer.cpp:175) <Buffer> Mapped buffer 5, plane 0 to
[DEBUG] (NvV4l2ElementPlane.cpp:467) <conv0> :Output Plane:STREAMON successful
[DEBUG] (NvV4l2ElementPlane.cpp:467) <conv0> :Capture Plane:STREAMON successful
[DEBUG] (NvV4l2ElementPlane.cpp:467) <enc1> :Output Plane:STREAMON successful
[DEBUG] (NvV4l2ElementPlane.cpp:467) <enc1> :Capture Plane:STREAMON successful
[DEBUG] (NvV4l2ElementPlane.cpp:781) <conv0> :Capture Plane:Started DQ Thread
[DEBUG] (NvV4l2ElementPlane.cpp:781) <enc1> :Output Plane:Started DQ Thread
[DEBUG] (NvV4l2ElementPlane.cpp:716) <conv0> :Capture Plane:Starting DQthread
[DEBUG] (NvV4l2ElementPlane.cpp:781) <enc1> :Capture Plane:Started DQ Thread
[DEBUG] (NvV4l2ElementPlane.cpp:716) <enc1> :Capture Plane:Starting DQthread
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Capture Plane:Qed buffer 0
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Capture Plane:Qed buffer 1
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Capture Plane:Qed buffer 2
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Capture Plane:Qed buffer 3
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Capture Plane:Qed buffer 4
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Capture Plane:Qed buffer 5
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Capture Plane:Qed buffer 6
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Capture Plane:Qed buffer 7
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Capture Plane:Qed buffer 8
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Capture Plane:Qed buffer 9
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Capture Plane:Qed buffer 0
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Capture Plane:Qed buffer 1
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Capture Plane:Qed buffer 2
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Capture Plane:Qed buffer 3
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Capture Plane:Qed buffer 4
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Capture Plane:Qed buffer 5
[DEBUG] (NvV4l2ElementPlane.cpp:716) <enc1> :Output Plane:Starting DQthread
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 0
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 1
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 2
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 3
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 4
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 5
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Capture Plane:DQed buffer 0
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 6
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Output Plane:Qed buffer 0
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 7
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 8
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 9
START MAIN---------------------------------------
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Output Plane:DQed buffer 0
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 0
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Output Plane:DQed buffer 1
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 1
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Capture Plane:DQed buffer 1
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Output Plane:Qed buffer 1
===== NVENC blits (mode: 1) into block linear surfaces =====
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Output Plane:DQed buffer 2
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Capture Plane:DQed buffer 2
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Output Plane:Qed buffer 2
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 2
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Output Plane:DQed buffer 3
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Capture Plane:DQed buffer 3
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Output Plane:Qed buffer 3
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 3
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Output Plane:DQed buffer 4
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Capture Plane:DQed buffer 4
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Output Plane:Qed buffer 4
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 4
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Output Plane:DQed buffer 5
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Capture Plane:DQed buffer 5
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Output Plane:Qed buffer 5
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 5
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Output Plane:DQed buffer 6
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Capture Plane:DQed buffer 6
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 6
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Output Plane:DQed buffer 7
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 7
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Output Plane:DQed buffer 8
[DEBUG] (NvV4l2ElementPlane.cpp:151) <enc1> :Capture Plane:DQed buffer 0
[DEBUG] (NvV4l2ElementPlane.cpp:151) <enc1> :Output Plane:DQed buffer 0
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Capture Plane:Qed buffer 0
[DEBUG] (NvV4l2ElementPlane.cpp:151) <enc1> :Capture Plane:DQed buffer 1
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Capture Plane:Qed buffer 1
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 8
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Capture Plane:Qed buffer 0
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Output Plane:Qed buffer 0
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Capture Plane:DQed buffer 7
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Output Plane:DQed buffer 9
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 9
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Output Plane:DQed buffer 0
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 0
[DEBUG] (NvV4l2ElementPlane.cpp:151) <enc1> :Capture Plane:DQed buffer 2
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Capture Plane:Qed buffer 2
[DEBUG] (NvV4l2ElementPlane.cpp:151) <enc1> :Output Plane:DQed buffer 1
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Capture Plane:Qed buffer 1
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Output Plane:Qed buffer 1
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Capture Plane:DQed buffer 8
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Output Plane:DQed buffer 1
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 1
[DEBUG] (NvV4l2ElementPlane.cpp:151) <enc1> :Capture Plane:DQed buffer 3
[DEBUG] (NvV4l2ElementPlane.cpp:151) <enc1> :Output Plane:DQed buffer 2
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Capture Plane:Qed buffer 2
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Capture Plane:Qed buffer 3
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Output Plane:Qed buffer 2
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Output Plane:DQed buffer 2
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Capture Plane:DQed buffer 9
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 2
[DEBUG] (NvV4l2ElementPlane.cpp:151) <enc1> :Capture Plane:DQed buffer 4
[DEBUG] (NvV4l2ElementPlane.cpp:151) <enc1> :Output Plane:DQed buffer 3
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Capture Plane:Qed buffer 3
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Capture Plane:Qed buffer 4
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Output Plane:DQed buffer 3
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 3
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Output Plane:Qed buffer 3
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Capture Plane:DQed buffer 0
[DEBUG] (NvV4l2ElementPlane.cpp:151) <enc1> :Capture Plane:DQed buffer 5
[DEBUG] (NvV4l2ElementPlane.cpp:151) <enc1> :Output Plane:DQed buffer 4
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Capture Plane:Qed buffer 4
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Capture Plane:Qed buffer 5
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Output Plane:DQed buffer 4
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Output Plane:Qed buffer 4
CONV CAP BU=0 ERR
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 4
[DEBUG] (NvV4l2ElementPlane.cpp:763) <conv0> :Capture Plane:Exiting DQthread
[DEBUG] (NvV4l2ElementPlane.cpp:151) <enc1> :Capture Plane:DQed buffer 0
[DEBUG] (NvV4l2ElementPlane.cpp:151) <enc1> :Output Plane:DQed buffer 5
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Capture Plane:Qed buffer 0
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Capture Plane:Qed buffer 5
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Output Plane:DQed buffer 5
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 5
[DEBUG] (NvV4l2ElementPlane.cpp:151) <enc1> :Capture Plane:DQed buffer 1
[DEBUG] (NvV4l2ElementPlane.cpp:151) <enc1> :Output Plane:DQed buffer 0
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Capture Plane:Qed buffer 1
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Capture Plane:Qed buffer 6
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Output Plane:DQed buffer 6
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 6
[DEBUG] (NvV4l2ElementPlane.cpp:151) <enc1> :Capture Plane:DQed buffer 2
[DEBUG] (NvV4l2ElementPlane.cpp:151) <enc1> :Output Plane:DQed buffer 1
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Capture Plane:Qed buffer 7
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Capture Plane:Qed buffer 2
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Output Plane:DQed buffer 7
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 7
[DEBUG] (NvV4l2ElementPlane.cpp:151) <enc1> :Capture Plane:DQed buffer 3
[DEBUG] (NvV4l2ElementPlane.cpp:151) <enc1> :Output Plane:DQed buffer 2
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Capture Plane:Qed buffer 8
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Capture Plane:Qed buffer 3
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Output Plane:DQed buffer 8
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 8
[DEBUG] (NvV4l2ElementPlane.cpp:151) <enc1> :Capture Plane:DQed buffer 4
[DEBUG] (NvV4l2ElementPlane.cpp:151) <enc1> :Output Plane:DQed buffer 3
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Capture Plane:Qed buffer 9
[DEBUG] (NvV4l2ElementPlane.cpp:257) <enc1> :Capture Plane:Qed buffer 4
[DEBUG] (NvV4l2ElementPlane.cpp:151) <enc1> :Capture Plane:DQed buffer 5
[DEBUG] (NvV4l2ElementPlane.cpp:763) <enc1> :Capture Plane:Exiting DQthread
[DEBUG] (NvV4l2ElementPlane.cpp:151) <conv0> :Output Plane:DQed buffer 9
[DEBUG] (NvV4l2ElementPlane.cpp:257) <conv0> :Output Plane:Qed buffer 9

I created a 320x180 UYVY raw/avi files and measured the performance of the same pipeline using the multimedia API and gstreamer C API. The pipeline: read 320x180 frame → scale 1920x1080 → encode. For both projects this pipeline is started 1-4 times using OpenMP. Results:

Encode using 4 pipelines, H265: 111 fps (gstreamer), 72 (MM API)
Encode using 4 pipelines, H264: 128 fps (gstreamer), 47 (62 when output write is on ??) (MM API)
So gstreamer acts as anybody would guess: H264 is slightly faster than H265. However, MM API behavior is strange, as H264 is considerably slower than H265. And unfortunately MM API is much slower than gstreamer.
What am I missing here? Shouldn’t be MM API at least as fast as gstreamer? I would really like to use it as I would need frame by frame synchronization between camera sources, post processing, etc, all of them seem easier to do in MM API.
And one more thing: for approximately 1 in 5-6 runs I get
[ERROR] (NvV4l2Element.cpp:67) Error in VIDIOC_QUERYCAP: Inappropriate ioctl for device
Could not create Video Converter

In case it is required, the project can be downloaded:
http://home.mit.bme.hu/~szanto/tegra/tx2_conv_enc.zip
It is relatively large, because it contains an input UYVY video. Usage:
./video_convert bunny_320_UYVY.yuv 320 180 1920 1080 1 4
(input file, input resolution, output resolution, H265 enable, number of concurrent streams)

Thank you in advance.

DaneLLL · April 25, 2017, 1:30am

Hi Tessier,
1 Please refer to read_video_frame() in tegra_multimedia_api\samples\common\classes\NvUtils.c
Looks like you miss

plane.bytesused = plane.fmt.stride * plane.fmt.height;

2 Please refer to https://devtalk.nvidia.com/default/topic/994281/jetson-tx1/v4l2-video-encoder-performance-/post/5090266/#5090266

Tessier · April 25, 2017, 11:39am

Hi Dane,

Thanks, somehow I completely missed NvUtils.c. Sorry.

I was aware of that topic, but the TX2 L4T release does not allow to set the encoder’s HW Preset and the linked sources/libraries in the above topic are for r24.2.1. I tried to use them on the TX2 anyway according to the readme, but it does not work:

Failed to query video capabilities: Inappropriate ioctl for device
[DEBUG] (NvV4l2Element.cpp:62) <conv0> :Opened, fd = 7
Failed to query video capabilities: Inappropriate ioctl for device
[DEBUG] (NvV4l2Element.cpp:62) <enc1> :Opened, fd = 8
[DEBUG] (NvV4l2Element.cpp:182) <conv0> :Set controls
[DEBUG] (NvVideoConverter.cpp:200) <conv0> :Setting output plane buffer layout to 0: success
[DEBUG] (NvV4l2ElementPlane.cpp:290) <conv0> :Output Plane:VIDIOC_S_FMT at capture plane successful
[DEBUG] (NvVideoConverter.cpp:143) <conv0> :Setting output plane format: success
[DEBUG] (NvV4l2Element.cpp:182) <conv0> :Set controls
[DEBUG] (NvVideoConverter.cpp:220) <conv0> :Setting capture plane buffer layout to 1: success
[DEBUG] (NvV4l2ElementPlane.cpp:290) <conv0> :Capture Plane:VIDIOC_S_FMT at capture plane successful
[DEBUG] (NvVideoConverter.cpp:114) <conv0> :Setting capture plane format: success
320 180 1920 1080
[DEBUG] (NvV4l2ElementPlane.cpp:290) <enc1> :Capture Plane:VIDIOC_S_FMT at capture plane successful
NvMMLiteOpen : Block : BlockType = 8
===== MSENC =====
NvMMLiteBlockCreate : Block : BlockType = 8
[DEBUG] (NvV4l2ElementPlane.cpp:290) <enc1> :Output Plane:VIDIOC_S_FMT at capture plane successful
892744264
842091865
[DEBUG] (NvV4l2Element.cpp:182) <enc1> :Set controls
[DEBUG] (NvVideoEncoder.cpp:187) <enc1> :Setting encoder bitrate to 4000000: success
[DEBUG] (NvV4l2Element.cpp:182) <enc1> :Set controls
[DEBUG] (NvVideoEncoder.cpp:221) <enc1> :Setting encoder profile to 0: success
[DEBUG] (NvV4l2ElementPlane.cpp:509) <enc1> :Output Plane:Successfully set stream parameters
[DEBUG] (NvVideoEncoder.cpp:163) <enc1> :Setting framerate to 60/1: success
libv4l2_nvvidconv (0):(765) (INFO) : Allocating (10) OUTPUT PLANE BUFFERS Layout=0
libv4l2_nvvidconv (0):(867) (ERROR) : NvRmMemHandleAllocAttr function failed
[ERROR] (NvV4l2ElementPlane.cpp:409) <conv0> Output Plane:Error in VIDIOC_REQBUFS at output plane: Invalid argument
[ERROR] (NvV4l2ElementPlane.cpp:635) <conv0> Output Plane:Error during setup
[DEBUG] (NvV4l2ElementPlane.cpp:446) <conv0> :Output Plane:Already in STREAMOFF
[DEBUG] (NvV4l2ElementPlane.cpp:831) <conv0> :Output Plane:Stopped DQ Thread
[DEBUG] (NvV4l2ElementPlane.cpp:433) <conv0> :Output Plane:Reqbuf returned 0 buffers
[DEBUG] (NvV4l2ElementPlane.cpp:372) <conv0> :Output Plane:deinit successful
Error while setting up output plane for conv0
VideoConverter0 is in error
[DEBUG] (NvV4l2ElementPlane.cpp:446) <conv0> :Output Plane:Already in STREAMOFF
[DEBUG] (NvV4l2ElementPlane.cpp:831) <conv0> :Output Plane:Stopped DQ Thread
[DEBUG] (NvV4l2ElementPlane.cpp:433) <conv0> :Output Plane:Reqbuf returned 0 buffers
[DEBUG] (NvV4l2ElementPlane.cpp:372) <conv0> :Output Plane:deinit successful
[DEBUG] (NvV4l2ElementPlane.cpp:446) <conv0> :Capture Plane:Already in STREAMOFF
[DEBUG] (NvV4l2ElementPlane.cpp:831) <conv0> :Capture Plane:Stopped DQ Thread
[DEBUG] (NvV4l2ElementPlane.cpp:433) <conv0> :Capture Plane:Reqbuf returned 0 buffers
[DEBUG] (NvV4l2ElementPlane.cpp:372) <conv0> :Capture Plane:deinit successful
[DEBUG] (NvV4l2Element.cpp:87) <V4l2Element> Device closed, fd = 7
[DEBUG] (NvV4l2ElementPlane.cpp:446) <enc1> :Output Plane:Already in STREAMOFF
[DEBUG] (NvV4l2ElementPlane.cpp:831) <enc1> :Output Plane:Stopped DQ Thread
[DEBUG] (NvV4l2ElementPlane.cpp:433) <enc1> :Output Plane:Reqbuf returned 0 buffers
[DEBUG] (NvV4l2ElementPlane.cpp:372) <enc1> :Output Plane:deinit successful
[DEBUG] (NvV4l2ElementPlane.cpp:446) <enc1> :Capture Plane:Already in STREAMOFF
[DEBUG] (NvV4l2ElementPlane.cpp:831) <enc1> :Capture Plane:Stopped DQ Thread
[DEBUG] (NvV4l2ElementPlane.cpp:433) <enc1> :Capture Plane:Reqbuf returned 0 buffers
[DEBUG] (NvV4l2ElementPlane.cpp:372) <enc1> :Capture Plane:deinit successful
[DEBUG] (NvV4l2Element.cpp:87) <V4l2Element> Device closed, fd = 8

Can you provide MMAPI sources/libraries with encoder preset setting for the TX2?

Thanks.

DaneLLL · April 26, 2017, 2:00am

Hi Tessier,
Please apply the patch attached to MMAPI sources and give it a try.

The prebuilt libs are with the changes already.
0001-multimedia_api-videoenc-Add-extended-control.txt (6.41 KB)

DaneLLL · April 26, 2017, 2:51am

From ff1ff3cda3a712196ffbfb6d644a33c049aec5e0 Mon Sep 17 00:00:00 2001
From: Amit Pandya <apandya@nvidia.com>
Date: Tue, 29 Nov 2016 13:57:26 +0530
Subject: [PATCH] multimedia_api: videoenc: Add extended control

Add function for following encoder feature
- Encoder HW Preset

Bug 200249574

Change-Id: I25d3078e3e016f1030c063010c8978150876b4f7
Reviewed-on: http://git-master/r/1261277
(cherry picked from commit 4667a7a98686520b9197d4846137fd926348518a)
---
 multimedia_api/ll_samples/include/NvVideoEncoder.h | 13 ++++++++++++
 .../samples/01_video_encode/video_encode.h         |  1 +
 .../01_video_encode/video_encode_csvparser.cpp     | 24 ++++++++++++++++++++++
 .../samples/01_video_encode/video_encode_main.cpp  |  6 ++++++
 .../samples/common/classes/NvVideoEncoder.cpp      | 23 +++++++++++++++++++++
 5 files changed, 67 insertions(+)

diff --git a/multimedia_api/ll_samples/include/NvVideoEncoder.h b/multimedia_api/ll_samples/include/NvVideoEncoder.h
index 8e42514..0cd818f 100644
--- a/multimedia_api/ll_samples/include/NvVideoEncoder.h
+++ b/multimedia_api/ll_samples/include/NvVideoEncoder.h
@@ -240,6 +240,19 @@ public:
     int setSliceLength(v4l2_enc_slice_length_type type, uint32_t length);
 
     /**
+     * Set the encoder HW Preset Type.
+     *
+     * Calls the VIDIOC_S_EXT_CTRLS ioctl internally with Control id
+     * #V4L2_CID_MPEG_VIDEOENC_HW_PRESET_TYPE_PARAM. Must be called after setFormat on
+     * both the planes and before #requestBuffers on any of the planes.
+     *
+     * @param[in] type HW Preset Type, one of
+     *                         enum v4l2_enc_hw_preset_type_param
+     * @returns 0 for success, -1 for failure
+     */
+    int setHWPresetType(v4l2_enc_hw_preset_type type);
+
+    /**
      * Sets the Region of Interest (ROI) parameters for the next buffer, which will
      * be queued on the output plane with index \a buffer_index.
      *
diff --git a/multimedia_api/ll_samples/samples/01_video_encode/video_encode.h b/multimedia_api/ll_samples/samples/01_video_encode/video_encode.h
index 3a04589..8c32368 100644
--- a/multimedia_api/ll_samples/samples/01_video_encode/video_encode.h
+++ b/multimedia_api/ll_samples/samples/01_video_encode/video_encode.h
@@ -72,6 +72,7 @@ typedef struct
     uint32_t fps_d;
 
     enum v4l2_enc_temporal_tradeoff_level_type temporal_tradeoff_level;
+    enum v4l2_enc_hw_preset_type hw_preset_type;
     v4l2_enc_slice_length_type slice_length_type;
     uint32_t slice_length;
     uint32_t virtual_buffer_size;
diff --git a/multimedia_api/ll_samples/samples/01_video_encode/video_encode_csvparser.cpp b/multimedia_api/ll_samples/samples/01_video_encode/video_encode_csvparser.cpp
index ef77721..298f46c 100644
--- a/multimedia_api/ll_samples/samples/01_video_encode/video_encode_csvparser.cpp
+++ b/multimedia_api/ll_samples/samples/01_video_encode/video_encode_csvparser.cpp
@@ -67,6 +67,7 @@ print_help(void)
             "\t-vbs <size>           Virtual buffer size [Default = 0]\n"
             "\t-nrf <num>            Number of reference frames [Default = 1]\n\n"
             "\t-slt <type>           Slice length type (1 = Number of MBs, 2 = Bytes) [Default = 1]\n"
+            "\t-hpt <type>           HW preset type (1 = ultrafast, 2 = fast, 3 = medium,  4 = slow)\n"
             "\t-slen <length>        Slice length [Default = 0]\n"
             "\t-sir <interval>       Slice intrarefresh interval [Default = 0]\n\n"
             "\t-nbf <num>            Number of B frames [Default = 0]\n\n"
@@ -460,6 +461,29 @@ parse_csv_args(context_t * ctx, int argc, char *argv[])
                             "Unsupported value for slice length type: " << *argp);
             }
         }
+        else if (!strcmp(arg, "-hpt"))
+        {
+            argp++;
+            CHECK_OPTION_VALUE(argp);
+            switch (atoi(*argp))
+            {
+                case 1:
+                    ctx->hw_preset_type = V4L2_ENC_HW_PRESET_ULTRAFAST;
+                    break;
+                case 2:
+                    ctx->hw_preset_type = V4L2_ENC_HW_PRESET_FAST;
+                    break;
+                case 3:
+                    ctx->hw_preset_type = V4L2_ENC_HW_PRESET_MEDIUM;
+                    break;
+                case 4:
+                    ctx->hw_preset_type = V4L2_ENC_HW_PRESET_SLOW;
+                    break;
+                default:
+                    CSV_PARSE_CHECK_ERROR(true,
+                            "Unsupported value for encoder HW Preset Type: " << *argp);
+            }
+        }
         else if (!strcmp(arg, "-slen"))
         {
             argp++;
diff --git a/multimedia_api/ll_samples/samples/01_video_encode/video_encode_main.cpp b/multimedia_api/ll_samples/samples/01_video_encode/video_encode_main.cpp
index 838dfbe..55b87ca 100644
--- a/multimedia_api/ll_samples/samples/01_video_encode/video_encode_main.cpp
+++ b/multimedia_api/ll_samples/samples/01_video_encode/video_encode_main.cpp
@@ -598,6 +598,12 @@ main(int argc, char *argv[])
         TEST_ERROR(ret < 0, "Could not set slice length params", cleanup);
     }
 
+    if (ctx.hw_preset_type)
+    {
+        ret = ctx.enc->setHWPresetType(ctx.hw_preset_type);
+        TEST_ERROR(ret < 0, "Could not set encoder HW Preset Type", cleanup);
+    }
+
     if (ctx.virtual_buffer_size)
     {
         ret = ctx.enc->setVirtualBufferSize(ctx.virtual_buffer_size);
diff --git a/multimedia_api/ll_samples/samples/common/classes/NvVideoEncoder.cpp b/multimedia_api/ll_samples/samples/common/classes/NvVideoEncoder.cpp
index 2cc1ed0..e6fa3bc 100644
--- a/multimedia_api/ll_samples/samples/common/classes/NvVideoEncoder.cpp
+++ b/multimedia_api/ll_samples/samples/common/classes/NvVideoEncoder.cpp
@@ -389,6 +389,29 @@ NvVideoEncoder::setSliceLength(v4l2_enc_slice_length_type type, uint32_t length)
 }
 
 int
+NvVideoEncoder::setHWPresetType(v4l2_enc_hw_preset_type type)
+{
+    struct v4l2_ext_control control;
+    struct v4l2_ext_controls ctrls;
+
+    RETURN_ERROR_IF_FORMATS_NOT_SET();
+    RETURN_ERROR_IF_BUFFERS_REQUESTED();
+
+    memset(&control, 0, sizeof(control));
+    memset(&ctrls, 0, sizeof(ctrls));
+
+    ctrls.count = 1;
+    ctrls.controls = &control;
+    ctrls.ctrl_class = V4L2_CTRL_CLASS_MPEG;
+
+    control.id = V4L2_CID_MPEG_VIDEOENC_HW_PRESET_TYPE_PARAM;
+    control.value = type;
+
+    CHECK_V4L2_RETURN(setExtControls(ctrls),
+            "Setting encoder HW Preset type to " << type);
+}
+
+int
 NvVideoEncoder::setROIParams(uint32_t buffer_index,
         v4l2_enc_frame_ROI_params &params)
 {
-- 
2.1.4

Tessier · April 29, 2017, 2:35pm

Hi Dane,

Thank you, now performance is on par with gstreamer.

ACervantes · May 8, 2017, 10:06pm

Hi All,

Somebody knows is there an example for capture+video_convert? I was searching in the examples but I only found capturing+encode or videoconvert from file!

Thanks,

snarky · May 8, 2017, 10:30pm

Where are you getting stuck in taking the “from file” part of the second example, and adding to the “encode” part of the first example?