How to close gstreamer pipeline in python

Autonomous Machines Jetson & Embedded Systems Jetson Nano
1

I’m using the gstreamer pipeline as demonstrated in some other posts on this forum but the program won’t completely stop once I’m done taking pictures. After taking the picture, the terminal shows

GST_ARGUS: Cleaning up
GST_ARGUS: 
PowerServiceHwVic::cleanupResources

and won’t relinquish control back to me unless I ctrl-Z out of the program. How can I close the pipeline? Additional question: Is there a way to suppress the terminal output associated with using the gstreamer pipeline?

Here is the python code being tested.

def gstreamer_pipeline (capture_width=3280, capture_height=2464, display_width=3280,
						display_height=2464, framerate=21, flip_method=2):   
	return ('nvarguscamerasrc ! ' 
	'video/x-raw(memory:NVMM), '
	'width=(int)%d, height=(int)%d, exposuretimerange=(string)2000000 2000000, '
	'format=(string)NV12, framerate=(fraction)%d/1 ! '
	'nvvidconv flip-method=%d ! '
	'video/x-raw, width=(int)%d, height=(int)%d, format=(string)BGRx ! '
	'videoconvert ! '
	'video/x-raw, format=(string)BGR ! appsink '
	% (capture_width, capture_height, framerate, flip_method, display_width, display_height))

def testGStreamer():
	camera = cv2.VideoCapture(gstreamer_pipeline(), cv2.CAP_GSTREAMER)

	t1 = time.time()
	ret, image = camera.read()
	t2 = time.time()
	saveImage(image, 'GStreamer_' + time.strftime('%m.%d_%H.%M.%S'))
	print 'Time elapsed = ' + str(t2 - t1) + ' seconds'	
	camera.release()

testGStreamer()

EDIT: Additionally, setting the exposure time like I did in the code above crops the image for some reason. The exposure appears to be unaffected. There seems to be some sort of auto-exposure or auto-whitebalance going on as well, is there a way to disable that? For my purposes, anything auto needs to be turned off. The auto settings are resulting in very poor image quality.

2

For setting exposure to a fixed value, you may have a look to this pipeline. You would adapt for your sensor and scene lightening. Obviously, if you don’t want to manage exposure and white balance yourself, you may rely on auto settings algorithms.

3

Gainrange and ispdigitalgainrange make a difference, but exposuretimerange does not. This results in a grainy image with poor detail. Is this a camera issue? The camera works flawlessly with the picamera module on Raspberry Pi.

Do you know how to properly dispose of the pipeline? I have to close the terminal every time I run the program because of this issue. Ctrl-C will not exit the program, only Ctrl-\ will force it to quit.

4

Hi,
Is the issue observed in running gst-launch-1.0?

$ gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM),format=NV12,width=3280,height=2464,framerate=21/1' ! nvvidconv flip-method=2 ! video/x-raw,format=BGRx ! videoconvert ! video/x-raw,format=BGR ! fakesink

Or it is present in python only?

5

Running gst-launch-1.0 from the terminal with those settings produces this:

GST_ARGUS: PowerService: requested_clock_Hz=37126320
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.

Then nothing happens. Ctrl-C starts the cleanup service just like in python, but won’t kill the program.

^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:01:50.720542956
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
GST_ARGUS: Cleaning up
GST_ARGUS: 
PowerServiceHwVic::cleanupResources

I have to Ctrl-\ to kill.

^\Caught SIGQUIT
#0  0x0000007fb3e77ce4 in __waitpid (pid=<optimized out>, stat_loc=0x7fdd6ebbf4, options=<optimized out>) at ../sysdeps/unix/sysv/linux/waitpid.c:30
#1  0x0000007fb3eb32a0 in g_on_error_stack_trace ()
#2  0x000000557e5f4c3c in  ()
#3  0x0000007fb3e8c000 in  ()
Quit (core dumped)
6

Running this in terminal kinda works:

gst-launch-1.0 nvarguscamerasrc ! 'video/x-raw(memory:NVMM),width=3820, height=2464, framerate=21/1, format=NV12' ! nvvidconv flip-method=2 ! 'video/x-raw,width=1920, height=1080' ! nvvidconv ! nvegltransform ! nveglglessink -e

It pulls up a window preview of the camera. Pressing X on the window does nothing but Ctrl-C properly exits the program.

I got a new camera in today. Specifically this https://leopardimaging.com/product/li-imx219-mipi-ff-nano/. All issues remain the same as with the old camera.

7

Main difference with above seems to be the width changed from 3280 to 3820…How a typo can make your life harder !
The behavior with nveglglessink is usual.

8

The sensor is 3280x2464. Changing the 3820 to 3280 changes the preview to full resolution but otherwise doesn’t affect the issues I’d like to fix.

9

Hi AgallenSMX,

Below command can get video preview and can be closed properly with Ctr + C. The same as the command in your comment#6.

gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! 'video/x-raw(memory:NVMM), width=(int)3280, height=(int)2464'  ! nvvidconv flip-method=0 ! 'video/x-raw, format=(string)I420' ! xvimagesink -e

Can this fix the “pipeline-close” issue?

BTW, we have an ISP file for our IMX219 module in case you need it. You should be able to get better image quality with this file.

https://www.dropbox.com/s/ougz816496m4ilw/camera_overrides.isp?dl=0

Please copy camera_overrides.isp to /var/nvidia/nvcam/settings and do below commands to install the ISP file.

sudo chmod 664 /var/nvidia/nvcam/settings/camera_overrides.isp
sudo chown root:root /var/nvidia/nvcam/settings/camera_overrides.isp
10

Thanks for the response! This command closed properly. I did some testing in python and it seems setting the framerate to anything higher than 8 fps causes the pipeline-close issue, even though gstreamer claims to support 21 fps at 3280x2464. The framerate has no effect on still image picture quality.

The ISP file results in much darker images. This would be OK if I could adjust the exposure settings, but changing the exposuretimerange still appears to do nothing.

EDIT: ISP file definitely reduces grain effects. Using an exposure time of 80000000 (80 million nanoseconds) finally resulted in an image close to what the raspberry pi produces.

11

Sorry if I miss something, but be aware that 80 ms exposure is about double your frame period at 21 fps framerate you mentioned. Does increasing gain results in such bad image you can’t get a shorter exposure ? If yes you may consider lightening more the scene.

12

Based on my limited understanding, gain is a postprocessing effect which has an end result of lowering image quality, even if it might look better to the eye. I’d rather change the exposure to adjust the light level of my pictures.

I’m not too concerned what the actual framerate is because I am not recording video. I’m recording concurrent still captures. If gstreamer functions similarly to the raspberry pi, the ‘acquisition speed’ of the program does not need to be synced to the camera’s internal framerate. If I request two images from the camera too quickly, I’d expect to just get two of the exact same frame because the camera is not ready to send a new frame yet.

Either way, 21 fps at 8MP resolution causes some sort of system instability issue. 8 fps seems to be the highest I can go without the program hanging on cleanup.

13

In my (limited, too) understanding digitalgain would be post acquisition but gain would apply to analog level. Not sure though.
However increasing gain will also increase noise if there is not enough light and would not improve SNR.

For taking snapshots it should be fine if you’re happy with it.

For someone else trying to acheive higher framerate, not sure at all but it may be worth trying to increase socket buffer size for such high resolution.

14

Thank you for that overrides file. The color balance is much better now (I have a 160 degree IMX219 from Waveshare and this fixed any pink tinting towards the edges)!

15

How could I modify this file for other IMX219 cameras? Is that possible?

16

I don’t believe I saw an answer regarding suppressing the terminal output associated with using the gstreamer pipeline?

Anyone have a followup to this?

Thanks in advance.

17

We also were unable to close the gstreamer pipeline started with the following command:

stream = cv2.VideoCapture("nvarguscamerasrc ! video/x-raw(memory:NVMM), width=1280, "
                                       "height=720, framerate=30/1, format=NV12 ! nvvidconv ! "
                                       "video/x-raw, format=BGRx, width=640, height=360 ! "
                                       "videoconvert ! video/x-raw, format=BGR ! appsink")

However, adding the following line worked:

stream = cv2.VideoCapture("nvarguscamerasrc ! video/x-raw(memory:NVMM), width=1280, "
                                       "height=720, framerate=30/1, format=NV12 ! nvvidconv ! "
                                       "video/x-raw, format=BGRx, width=640, height=360 ! "
                                       "videoconvert ! video/x-raw, format=BGR ! appsink "
                                       "wait-on-eos=false drop=true max-buffers=60 -e -vvv")

For more information, see Gstreamer pipeline hangs with appsink element · Issue #5 · InES-HPMM/linux-l4t · GitHub

18

I was having the hang issue on release and the suggestion by pdboef resolved it… thanks!

19

Additionally, I am no longer having hang on release issues with this capture method:

def gstreamer_auto(self):
    return ('nvarguscamerasrc ! '
    'video/x-raw(memory:NVMM), format=NV12, '
    'width=3280, height=2464, '
    'framerate=10/1 ! '
    'nvvidconv flip-method=2 ! '
    'video/x-raw, format=I420 ! '
    'appsink max-buffers=1 drop=True ')
20

I have used the camera_overrides.isp file for “raspberry pi NoIR camera V2” but the image quality is still not the one I expected. Though it has reduced blue distortion from middle and decreased pink distortion a little bit but the quality is still far from normal.

@AastaLLL please help