Is there some unfreed memory in grtreamer or deepstream plugin ?

Hi all.
For some needs,we need to start a pipeline frequently in the program. After running it for a while, we got a segment error always, But Wo can’t find some error in the program. Is there some unfreed memory?

Here is my test program.

#include <gst/gst.h>

int main(int argc, char *argv[])
{
    gchar *str = NULL;
    GstElement *pipeline = NULL;
    GstBus * bus = NULL;
    GstMessage *msg = NULL;
	gst_init(&argc, &argv);

    str = g_strdup_printf(" filesrc location=\"%s\" ! h264parse ! nvv4l2decoder ! nvoverlaysink ", argv[1]);
    pipeline = gst_parse_launch(str, NULL);
    g_free(str);
    str = NULL;
    if(pipeline)
    {
        gst_element_set_state(pipeline, GST_STATE_PLAYING);
        bus = gst_element_get_bus(pipeline);
        msg = gst_bus_timed_pop_filtered(bus, GST_CLOCK_TIME_NONE, GST_MESSAGE_EOS);
        gst_object_unref(bus);
        if(msg != NULL)
            gst_message_unref(msg);
        gst_element_set_state(pipeline, GST_STATE_NULL);
        gst_object_unref(pipeline);
        pipeline = NULL;
    }
    gst_deinit();
    return 0;
}

here is the log got with valgrind.
valgrind --tool=memcheck --leak-check=full --show-leak-kinds=all --log-file=test.txt ./test

==12157== Memcheck, a memory error detector
==12157== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==12157== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==12157== Command: ./video_demo /home/zalend/cache/video-201911261421080.h264
==12157== Parent PID: 12156
==12157== 
==12157== Warning: noted but unhandled ioctl 0x4e04 with no size/direction hints.
==12157==    This could cause spurious value errors to appear.
==12157==    See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper.
==12157== Source and destination overlap in memcpy(0x5e3f180, 0x5e3f180, 208)
==12157==    at 0x484A080: __GI_memcpy (in /usr/lib/valgrind/vgpreload_memcheck-arm64-linux.so)
==12157== 
==12157== Thread 2:
==12157== Syscall param ioctl(generic) points to unaddressable byte(s)
==12157==    at 0x4B8CB1C: ioctl (ioctl.S:26)
==12157==    by 0x66B9EA7: nvdcOpen (in /usr/lib/aarch64-linux-gnu/tegra/libnvdc.so)
==12157==    by 0x7F9265F: ??? (in /usr/lib/aarch64-linux-gnu/tegra/libnvomx.so)
==12157==    by 0x7F6F30F: ??? (in /usr/lib/aarch64-linux-gnu/tegra/libnvomx.so)
==12157==    by 0x7F50B3B: ??? (in /usr/lib/aarch64-linux-gnu/tegra/libnvomx.so)
==12157==    by 0x7F5AD77: ??? (in /usr/lib/aarch64-linux-gnu/tegra/libnvomx.so)
==12157==    by 0x610F627: ??? (in /usr/lib/aarch64-linux-gnu/tegra/libnvos.so)
==12157==    by 0x4D65087: start_thread (pthread_create.c:463)
==12157==    by 0x4B944EB: thread_start (clone.S:78)
==12157==  Address 0x6 is not stack'd, malloc'd or (recently) free'd
==12157== 
==12157== Syscall param ioctl(generic) points to unaddressable byte(s)
==12157==    at 0x4B8CB1C: ioctl (ioctl.S:26)
==12157==    by 0x66B4AB7: nvdcGetWindow (in /usr/lib/aarch64-linux-gnu/tegra/libnvdc.so)
==12157==    by 0x7F926E7: ??? (in /usr/lib/aarch64-linux-gnu/tegra/libnvomx.so)
==12157==    by 0x7F6F30F: ??? (in /usr/lib/aarch64-linux-gnu/tegra/libnvomx.so)
==12157==    by 0x7F50B3B: ??? (in /usr/lib/aarch64-linux-gnu/tegra/libnvomx.so)
==12157==    by 0x7F5AD77: ??? (in /usr/lib/aarch64-linux-gnu/tegra/libnvomx.so)
==12157==    by 0x610F627: ??? (in /usr/lib/aarch64-linux-gnu/tegra/libnvos.so)
==12157==    by 0x4D65087: start_thread (pthread_create.c:463)
==12157==    by 0x4B944EB: thread_start (clone.S:78)
==12157==  Address 0x4 is not stack'd, malloc'd or (recently) free'd
==12157== 
==12157== 
==12157== HEAP SUMMARY:
==12157==     in use at exit: 16,223,934 bytes in 4,416 blocks
==12157==   total heap usage: 57,324 allocs, 52,908 frees, 73,791,075 bytes allocated
==12157== 
==12157== Thread 1:
==12157== 31 bytes in 1 blocks are definitely lost in loss record 1 of 14
==12157==    at 0x4846D10: realloc (in /usr/lib/valgrind/vgpreload_memcheck-arm64-linux.so)
==12157== 
==12157== 55 bytes in 1 blocks are indirectly lost in loss record 2 of 14
==12157==    at 0x4844BFC: malloc (in /usr/lib/valgrind/vgpreload_memcheck-arm64-linux.so)
==12157== 
==12157== 216 bytes in 14 blocks are possibly lost in loss record 3 of 14
==12157==    at 0x4844B3C: malloc (in /usr/lib/valgrind/vgpreload_memcheck-arm64-linux.so)
==12157== 
==12157== 576 bytes in 18 blocks are still reachable in loss record 4 of 14
==12157==    at 0x4C2D50C: g_closure_new_simple (in /usr/lib/aarch64-linux-gnu/libgobject-2.0.so.0.5600.4)
==12157== 
==12157== 720 bytes in 18 blocks are still reachable in loss record 5 of 14
==12157==    at 0x4C2D4C4: g_closure_new_simple (in /usr/lib/aarch64-linux-gnu/libgobject-2.0.so.0.5600.4)
==12157== 
==12157== 904 bytes in 4 blocks are possibly lost in loss record 6 of 14
==12157==    at 0x4846D10: realloc (in /usr/lib/valgrind/vgpreload_memcheck-arm64-linux.so)
==12157== 
==12157== 4,524 bytes in 48 blocks are possibly lost in loss record 7 of 14
==12157==    at 0x4846B0C: calloc (in /usr/lib/valgrind/vgpreload_memcheck-arm64-linux.so)
==12157== 
==12157== 4,944 bytes in 84 blocks are still reachable in loss record 8 of 14
==12157==    at 0x4C54D84: g_type_create_instance (in /usr/lib/aarch64-linux-gnu/libgobject-2.0.so.0.5600.4)
==12157== 
==12157== 13,616 bytes in 301 blocks are still reachable in loss record 9 of 14
==12157==    at 0x4844B3C: malloc (in /usr/lib/valgrind/vgpreload_memcheck-arm64-linux.so)
==12157== 
==12157== 16,592 bytes in 84 blocks are still reachable in loss record 10 of 14
==12157==    at 0x4C54D3C: g_type_create_instance (in /usr/lib/aarch64-linux-gnu/libgobject-2.0.so.0.5600.4)
==12157== 
==12157== 16,791 (16,736 direct, 55 indirect) bytes in 3 blocks are definitely lost in loss record 11 of 14
==12157==    at 0x4844BFC: malloc (in /usr/lib/valgrind/vgpreload_memcheck-arm64-linux.so)
==12157== 
==12157== 21,208 bytes in 144 blocks are still reachable in loss record 12 of 14
==12157==    at 0x4846D10: realloc (in /usr/lib/valgrind/vgpreload_memcheck-arm64-linux.so)
==12157== 
==12157== 460,514 bytes in 1,290 blocks are still reachable in loss record 13 of 14
==12157==    at 0x4846B0C: calloc (in /usr/lib/valgrind/vgpreload_memcheck-arm64-linux.so)
==12157== 
==12157== 15,658,978 bytes in 2,304 blocks are still reachable in loss record 14 of 14
==12157==    at 0x4844BFC: malloc (in /usr/lib/valgrind/vgpreload_memcheck-arm64-linux.so)
==12157== 
==12157== LEAK SUMMARY:
==12157==    definitely lost: 16,767 bytes in 4 blocks
==12157==    indirectly lost: 55 bytes in 1 blocks
==12157==      possibly lost: 5,644 bytes in 66 blocks
==12157==    still reachable: 16,177,148 bytes in 4,243 blocks
==12157==                       of which reachable via heuristic:
==12157==                         length64           : 1,080 bytes in 27 blocks
==12157==                         newarray           : 1,584 bytes in 19 blocks
==12157==         suppressed: 0 bytes in 0 blocks
==12157== 
==12157== For counts of detected and suppressed errors, rerun with: -v
==12157== ERROR SUMMARY: 9 errors from 8 contexts (suppressed: 0 from 0)

Hi,
Please share which Jetson platform you use. Also there is deepstream sdk plugins( nvinfer, nvstreammux ) present in your gstreamer pipeline. Do you run a deepstream usecase or a pure decoding usecase?

Hi. DaneLLL.

I am using JetPack 4.2.1 for TX2, deepstream_sdk_v4.0_x86_64.tbz2 .

some plugins used in the main is showd as bellows.this pipeline would be started just once.

nvstreammux name=streammux -> nvinfer -> nvtracker -> nvstreamdemux name=streamdemux 
v4l2src -> nvvideoconvert -> streammux.sink_0 
v4l2src -> nvvideoconvert -> streammux.sink_1 
streamdemux.src_0 -> nvvideoconvert -> nvv4l2h264enc -> qtmux -> filesink 
streamdemux.src_1 -> tee name=t 
t. -> nvvideoconvert -> nvv4l2h264enc -> qtmux -> filesink 
t. -> nvvideoconvert -> nvdsosd -> nvoverlaysink

and in a thread showed as bellows.this pipeline would be started frequently.

concat name=c -> nvv4l2h264enc -> mp4mux -> filesink 
filesrc -> h264parse -> nvv4l2decoder -> c. 
filesrc -> h264parse -> nvv4l2decoder -> c.

void pipeline_run(gchar *str)
{
    GstBus * bus = NULL;
    GstMessage *msg = NULL;
    GstElement *pipeline = NULL;
    pipeline = gst_parse_launch(str, NULL);
    if(pipeline != NULL)
    {
        gst_element_set_state(pipeline, GST_STATE_PLAYING);
        bus = gst_element_get_bus(pipeline);
        msg = gst_bus_timed_pop_filtered(bus, GST_CLOCK_TIME_NONE, (GstMessageType)(GST_MESSAGE_ERROR | GST_MESSAGE_EOS));
        gst_object_unref(bus);
        if(msg != NULL)
        {
            GError *err;
            gchar *debug_info;
            switch(GST_MESSAGE_TYPE(msg))
            {
                case GST_MESSAGE_ERROR:
                    gst_message_parse_error(msg, &err, &debug_info);
                    NVGSTDS_ERR_MSG_V("Error received from element %s: %s\n", GST_OBJECT_NAME(msg->src), err->message);
                    NVGSTDS_ERR_MSG_V("Debugging information: %s\n", debug_info ? debug_info : "none");
                    g_clear_error(&err);
                    g_free(debug_info);
                    break;
                default:
                    break;
            }
            gst_message_unref(msg);
        }
        gst_element_set_state(pipeline, GST_STATE_NULL);
        gst_object_unref(pipeline);
        pipeline = NULL;
    }
}

In the thread,this function is called to start the pipeline,and after 10 to 20 times a segment error is reported alaways. if not called,the program run stablely.

Hi,
Do you loop the test program and hit the error? The test program only runs one-time video playback and then exit. Seems not same as your usecase. Please share a sample we can simply build and run to reproduce the failure. Thanks.

DaneLLL.

I’m Sorry,I had not hit the error for any test program.

Mybe the code bellows can, every times I tested it, it is killed by the system.
with the cammand ‘top -d 1’,I found the ‘%MEM’ changed from 0.9 to 2.0.

#include <stdio.h>
#include <string.h>
#include <errno.h>
#include <dirent.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <gst/gst.h>
#include <glib.h>
#include <gio/gio.h>

static void pipeline_run(gchar *video_name, gchar *output_name)
{
    gchar *str = NULL;
    GString *string = NULL;
    GstElement *pipeline = NULL;
    GstBus * bus = NULL;
    GstMessage *msg = NULL;
    string = g_string_new("");
    g_string_append_printf(string, " filesrc location=\"%s\" ! ", video_name);
    g_string_append(string, " h264parse ! nvv4l2decoder ! nvvideoconvert ! video/x-raw(memory:NVMM),format=I420 ! ");
    g_string_append(string, " videorate ! video/x-raw(memory:NVMM),format=I420,framerate=15/1 ! ");
    g_string_append_printf(string, " nvv4l2h264enc ! h264parse ! mp4mux ! filesink location=\"%s\"", output_name);
    str = g_string_free(string, FALSE);
    pipeline = gst_parse_launch(str, NULL);
    g_free(str);
    str = NULL;
    if(pipeline)
    {
        gst_element_set_state(pipeline, GST_STATE_PLAYING);
        bus = gst_element_get_bus(pipeline);
		msg = gst_bus_timed_pop_filtered(bus, GST_CLOCK_TIME_NONE, (GstMessageType)(GST_MESSAGE_ERROR | GST_MESSAGE_EOS));
        gst_object_unref(bus);
        if(msg != NULL)
            gst_message_unref(msg);
        gst_element_set_state(pipeline, GST_STATE_NULL);
        gst_object_unref(pipeline);
        pipeline = NULL;
    }
}

static gpointer thread_func(gpointer data)
{
    while(1)
    {
		sleep(1);
        pipeline_run("video-201911261421080.h264", "1.mp4");
    }
    return g_thread_self();
}

int main(int argc, char *argv[])
{
	gchar *str = NULL;
    GString *string = NULL;
    GstElement *pipeline = NULL;
    GstBus * bus = NULL;
    GstMessage *msg = NULL;	
    GThread *thread = NULL;
    gst_init(&argc, &argv);
    thread = g_thread_new("thread_func", thread_func, NULL);

    string = g_string_new("");
    g_string_append(string, " v4l2src device=/dev/video0 ! video/x-raw,format=UYVY,width=1920,height=1080 ! ");
	g_string_append(string, " videoconvert ! video/x-raw,format=NV12,width=1920,height=1080 ! ");
	g_string_append(string, " timeoverlay ! ");
	g_string_append(string, " nvvideoconvert ! video/x-raw(memory:NVMM),format=I420,width=1920,height=1080 ! ");
	g_string_append(string, " nvoverlaysink sync=false ");	
    str = g_string_free(string, FALSE);
    pipeline = gst_parse_launch(str, NULL);
    g_free(str);
    str = NULL;
    if(pipeline)
    {
        gst_element_set_state(pipeline, GST_STATE_PLAYING);
        bus = gst_element_get_bus(pipeline);
        msg = gst_bus_timed_pop_filtered(bus, GST_CLOCK_TIME_NONE, GST_MESSAGE_EOS);
        gst_object_unref(bus);
        if(msg != NULL)
            gst_message_unref(msg);
        gst_element_set_state(pipeline, GST_STATE_NULL);
        gst_object_unref(pipeline);
        pipeline = NULL;
    }
    g_thread_join(thread);
    gst_deinit();
    return 0;
}

the video-201911261421080.h264 I used is just 10s, 15 fps.

Hi,
So the leak is observed while using nvv4l2h264enc? We would like to break down the pipelines to clarify which plygin triggers memleak. In #1, the pipeline has nvv4l2decoder only. In #6, it has nvv4l2decoder and nvv4l2h264enc. It is a bit confusing.

Hi 2251582984,

We haven’t heard back from you in a couple weeks, so marking this topic is not an issue now.
Please open a new forum issue when you are ready and we’ll pick it up there.

Hi kayccc,

I had changed to usd ffmepg to do the same thing, and had never had hit the same error。