video encode error

708922276 · August 10, 2018, 11:45am

I was trying to encode video stream from onboard camera but failed.

The errors are:
[gstreamer] gstreamer mysource ERROR Internal data flow error.
[gstreamer] gstreamer Debugging info: gstbasesrc.c(2948): gst_base_src_loop (): /GstPipeline:pipeline0/GstAppSrc:mysource:
streaming task paused, reason not-negotiated (-4)

I use the codes from one of the developers working in nvidia:GitHub - dusty-nv/jetson-utils: C++/Python Linux utility wrappers for NVIDIA Jetson - camera, codecs, CUDA, GStreamer, HID, OpenGL/XGL.

Here is my main.cpp:

#include "gstCamera.h"
#include "cudaNormalize.h"
#include "gstUtility.h"
#include "gstEncoder.h"


int main(int argc, char *argv[])
{

	gst_init(&argc, &argv);

	gstEncoder *enc = gstEncoder::Create(GST_CODEC_H265,1280,720,"1.h265");

	gstCamera* camera = gstCamera::Create(-1);


	if( !camera )
	{
		printf("\ngst-camera:  failed to initialize video device\n");
		return 0;
	}

	printf("\ngst-camera:  successfully initialized video device\n");
	printf("    width:  %u\n", camera->GetWidth());
	printf("   height:  %u\n", camera->GetHeight());
	printf("    depth:  %u (bpp)\n", camera->GetPixelDepth());

	if( !camera->Open() )
	{
		printf("\ngst-camera:  failed to open camera for streaming\n");
		return 0;
	}

	printf("\ngst-camera:  camera open for streaming\n");

	int num = 0;
	const size_t texSz = camera->GetWidth() * camera->GetHeight() * 3 / 2;


		while( 1 )
		{
			num++;
			if(num==5)
				break;

			void* imgCPU  = NULL;
			void* imgCUDA = NULL;

			// get the latest frame
			if( !camera->Capture(&imgCPU, &imgCUDA, 1000) )
				printf("\ngst-camera:  failed to capture frame\n");
			else
				printf("gst-camera:  recieved new frame  CPU=0x%p  GPU=0x%p\n", imgCPU, imgCUDA);

			// convert from YUV to RGBA
//			void* imgRGBA = NULL;

//			if( !camera->ConvertRGBA(imgCUDA, &imgRGBA, true) )
//				printf("gst-camera:  failed to convert from NV12 to RGBA\n");


			enc->EncodeFrame(imgCPU,texSz);//This function cannt work.





		}
		printf("\ngst-camera:  un-initializing video device\n");


		camera->Close();
		delete camera;
		delete enc;

	return 0;
}

708922276 · August 10, 2018, 11:47am

[gstreamer] gstEncoder - pipeline launch string:
appsrc name=mysource ! omxh265enc ! video/x-h265 ! filesink location=1.h265
[gstreamer] gstEncoder - buffer caps string:
video/x-raw-yuv,width=1280,height=720,format=(string)I420,framerate=30/1

(gstCamera:19418): GStreamer-WARNING **: 0.10-style raw video caps are being created. Should be video/x-raw,format=(string)… now.
[gstreamer] gstEncoder - transitioning pipeline to GST_STATE_PLAYING
Inside NvxLiteH264DecoderLowLatencyInitNvxLiteH264DecoderLowLatencyInit set DPB and MjstreamingInside NvxLiteH265DecoderLowLatencyInitNvxLiteH265DecoderLowLatencyInit set DPB and Mjstreaming[gstreamer] initialized gstreamer, version 1.8.3.0
[gstreamer] gstreamer decoder pipeline string:
nvcamerasrc fpsRange=“30.0 30.0” ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)I420 ! nvvidconv flip-method=2 ! video/x-raw ! appsink name=mysink
[gstreamer] gstEncoder - AppSrc requesting data (4096 bytes)

gst-camera: successfully initialized video device
width: 1280
height: 720
depth: 12 (bpp)
[gstreamer] gstreamer transitioning pipeline to GST_STATE_PLAYING

Available Sensor modes :
2592 x 1944 FR=30.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
2592 x 1458 FR=30.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
1280 x 720 FR=120.000000 CF=0x1109208a10 SensorModeType=4 CSIPixelBitDepth=10 DynPixelBitDepth=10
[gstreamer] gstreamer changed state from NULL to READY ==> mysink
[gstreamer] gstreamer changed state from NULL to READY ==> capsfilter2
[gstreamer] gstreamer changed state from NULL to READY ==> nvvconv0
[gstreamer] gstreamer changed state from NULL to READY ==> capsfilter1
[gstreamer] gstreamer changed state from NULL to READY ==> nvcamerasrc0
[gstreamer] gstreamer changed state from NULL to READY ==> pipeline1
[gstreamer] gstreamer changed state from READY to PAUSED ==> capsfilter2
[gstreamer] gstreamer changed state from READY to PAUSED ==> nvvconv0
[gstreamer] gstreamer changed state from READY to PAUSED ==> capsfilter1
[gstreamer] gstreamer stream status CREATE ==> src
[gstreamer] gstreamer changed state from READY to PAUSED ==> nvcamerasrc0
[gstreamer] gstreamer changed state from READY to PAUSED ==> pipeline1
[gstreamer] gstreamer stream status ENTER ==> src
[gstreamer] gstreamer msg new-clock ==> pipeline1
[gstreamer] gstreamer msg stream-start ==> pipeline1
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> capsfilter2
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> nvvconv0
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> capsfilter1
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> nvcamerasrc0

NvCameraSrc: Trying To Set Default Camera Resolution. Selected 1280x720 FrameRate = 30.000000 …

gst-camera: camera open for streaming
[gstreamer] gstreamer decoder onPreroll
[cuda] cudaAllocMapped 1382400 bytes, CPU 0x102800000 GPU 0x102800000
[cuda] cudaAllocMapped 1382400 bytes, CPU 0x102a00000 GPU 0x102a00000
[cuda] cudaAllocMapped 1382400 bytes, CPU 0x102c00000 GPU 0x102c00000
[cuda] cudaAllocMapped 1382400 bytes, CPU 0x102e00000 GPU 0x102e00000
[cuda] cudaAllocMapped 1382400 bytes, CPU 0x103000000 GPU 0x103000000
[cuda] cudaAllocMapped 1382400 bytes, CPU 0x103200000 GPU 0x103200000
[cuda] cudaAllocMapped 1382400 bytes, CPU 0x103400000 GPU 0x103400000
[cuda] cudaAllocMapped 1382400 bytes, CPU 0x103600000 GPU 0x103600000
[cuda] cudaAllocMapped 1382400 bytes, CPU 0x103800000 GPU 0x103800000
[cuda] cudaAllocMapped 1382400 bytes, CPU 0x103a00000 GPU 0x103a00000
[cuda] cudaAllocMapped 1382400 bytes, CPU 0x103c00000 GPU 0x103c00000
[cuda] cudaAllocMapped 1382400 bytes, CPU 0x103e00000 GPU 0x103e00000
[cuda] cudaAllocMapped 1382400 bytes, CPU 0x104000000 GPU 0x104000000
[cuda] cudaAllocMapped 1382400 bytes, CPU 0x104200000 GPU 0x104200000
[cuda] cudaAllocMapped 1382400 bytes, CPU 0x104400000 GPU 0x104400000
[cuda] cudaAllocMapped 1382400 bytes, CPU 0x104600000 GPU 0x104600000
[cuda] gstreamer camera – allocated 16 ringbuffers, 1382400 bytes each
[gstreamer] gstreamer changed state from READY to PAUSED ==> mysink
[gstreamer] gstreamer msg async-done ==> pipeline1
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> mysink
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> pipeline1
gst-camera: recieved new frame CPU=0x0x104000000 GPU=0x0x104000000
[gstreamer] gstreamer changed state from NULL to READY ==> filesink0
[gstreamer] gstreamer changed state from NULL to READY ==> capsfilter0
[gstreamer] gstreamer changed state from NULL to READY ==> omxh265enc-omxh265enc0
[gstreamer] gstreamer changed state from NULL to READY ==> mysource
[gstreamer] gstreamer changed state from NULL to READY ==> pipeline0
[gstreamer] gstreamer changed state from READY to PAUSED ==> capsfilter0
[gstreamer] gstreamer changed state from READY to PAUSED ==> omxh265enc-omxh265enc0
[gstreamer] gstreamer stream status CREATE ==> src
[gstreamer] gstreamer changed state from READY to PAUSED ==> mysource
[gstreamer] gstreamer changed state from READY to PAUSED ==> pipeline0
[gstreamer] gstreamer stream status ENTER ==> src
[gstreamer] gstreamer msg new-clock ==> pipeline0
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> capsfilter0
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> omxh265enc-omxh265enc0
[gstreamer] gstreamer changed state from PAUSED to PLAYING ==> mysource
gst-camera: recieved new frame CPU=0x0x103c00000 GPU=0x0x103c00000
[gstreamer] gstreamer mysource ERROR Internal data flow error.
[gstreamer] gstreamer Debugging info: gstbasesrc.c(2948): gst_base_src_loop (): /GstPipeline:pipeline0/GstAppSrc:mysource:
streaming task paused, reason not-negotiated (-4)
[gstreamer] gstreamer msg stream-start ==> pipeline0
[gstreamer] gstreamer changed state from READY to PAUSED ==> filesink0
gst-camera: recieved new frame CPU=0x0x103a00000 GPU=0x0x103a00000
[gstreamer] gstEncoder - AppSrc signalling enough data
gst-camera: recieved new frame CPU=0x0x103600000 GPU=0x0x103600000
[gstreamer] gstEncoder - pipeline full, skipping frame (1382400 bytes)

gst-camera: un-initializing video device
[gstreamer] gstreamer transitioning pipeline to GST_STATE_NULL
[gstreamer] gstEncoder - shutting down pipeline, sending EOS
^C

708922276 · August 10, 2018, 11:49am

This program can not stop running after “[gstreamer] gstEncoder - shutting down pipeline, sending EOS”, so I have to input control+C to stop it.

Thanks for your reading. I’m looking forward to your reply.

DaneLLL · August 13, 2018, 2:21am

Hi,
There are some posts about gstreamer encoding for your reference:
[url]Usage of NvBuffer APIs - Jetson TX1 - NVIDIA Developer Forums
[url]What is maximum video encoding resolution in pixels? - Jetson TX2 - NVIDIA Developer Forums

708922276 · August 14, 2018, 7:44am

I change the caps string from:

[gstreamer] gstEncoder - buffer caps string:
video/x-raw-yuv,width=1280,height=720,format=(string)I420,framerate=30/1

to

[gstreamer] gstEncoder - buffer caps string:
video/x-raw,width=1280,height=720,format=(string)I420,framerate=30/1

Now it can encode video and the output file “1.h265” is no longer 0 bytes.

However, this program still cannot stop running after “[gstreamer] gstEncoder - shutting down pipeline, sending EOS”, and the file “1.h265” can not be played by ffplay or vlc player.

708922276 · August 14, 2018, 7:49am

[hevc @ 0x7f700092e0] Format hevc detected only with low score of 1, misdetection possible!
[hevc @ 0x7f70000c80] missing picture in access unit
[hevc @ 0x7f70000c80] No start code is found. 0KB sq= 0B f=0/0
[hevc @ 0x7f70000c80] Error splitting the input into NAL units.
[hevc @ 0x7f700092e0] decoding for stream 0 failed
[hevc @ 0x7f700092e0] Could not find codec parameters for stream 0 (Video: hevc, none): unspecified size
Consider increasing the value for the ‘analyzeduration’ and ‘probesize’ options
/home/nvidia/Documents/gstCamera/Debug/1.h265: could not find codec parameters

708922276 · August 14, 2018, 11:05am

Now my program can stop normally, but the video file still can not be player. The output from ffplay is the same as my last reply.

DaneLLL · August 14, 2018, 1:49pm

Hi,
You can try to run omxh265enc ! matroskamux ! filesink to have mux’ed mkv.

708922276 · August 20, 2018, 11:46am

Now I find the way to solve it.
Replace “omxh265enc !” to “omxh265enc ! video/x-h265,stream-format=(string)byte-stream !”.
Thanks for all replys!

kcrossman · October 5, 2018, 6:02am

Hi 708922276,

Did you use the gstEncoder found in the dev branch of the dusy-nv Jetson inference repo?

I am struggling to get this encoder to run. I get a memory error when trying to create the gstEncoder.

Could you perhaps share your gstencoder.cpp code?

Thanks