The problem of playing coordinate for nvoverlaysink

Hello,

We have one problem about playing coordinate for nvoverlaysink. Please refer to the following sample code and I also highlight the symptom. We set the coordinates before playing and the coordinates are correct. But, the coordinates can not be changed after playing. The symptom happened both on R23.2 and R24.2. Can you kindly help to check this issue ?

static void playback_pad_added(GstElement* src, GstPad* new_pad, gpointer user_data)
{
PlaybackData* s = (PlaybackData*)user_data;
printf(“Received new pad ‘%s’ from ‘%s’:\n”, GST_PAD_NAME(new_pad), GST_ELEMENT_NAME(src));

AutoGstCaps caps = gst_pad_get_current_caps(new_pad);
GstStructure* structure = gst_caps_get_structure(caps, 0);
const gchar* pad_type = gst_structure_get_name(structure);
if(g_str_has_prefix(pad_type, "video/"))
{
	if(!gst_element_link(src, s->overlay))
		printf("overlay could not be linked.\n");
}
else if(g_str_has_prefix(pad_type, "audio/"))
{
}

}

static void playback_no_more_pads(GstElement* object, gpointer user_data)
{
PlaybackData* s = (PlaybackData*)user_data;
s->no_more_pads_event.set_event();
}

    GstElement* pipeline = gst_pipeline_new("pipeline");
    GstElement* source = gst_element_factory_make("uridecodebin", "source");
    g_object_set(source, "uri", "file://test.mp4", NULL);
    g_signal_connect(source, "pad-added", G_CALLBACK(playback_pad_added), s);
    g_signal_connect(source, "no-more-pads", G_CALLBACK(playback_no_more_pads), s);

    GstElement* sink = gst_element_factory_make("nvoverlaysink", NULL);

    
    /* The coordinates are correct here. */


    g_object_set(sink,
             "overlay-x", x,
             "overlay-y", y,
             "overlay-w", cx,
             "overlay-h", cy,
             NULL);

    if(!pipeline || !source || !sink)
    {
             printf("Not all elements could be created.\n");
             return -1;
    }

    gst_bin_add_many(GST_BIN(pipeline), source, sink, NULL);

    if(gst_element_set_state(pipeline, GST_STATE_PLAYING) == GST_STATE_CHANGE_FAILURE)
    {
             printf("Unable to set the pipeline to the playing state.\n");
             return -1;
    }


    /* Waiting for play. */


    usleep(5000 * 1000);


    /* After playing, we changed the new coordinates but the original coordinates was not replaced by new one. */

    g_object_set(sink,
             "overlay-x", x,
             "overlay-y", y,
             "overlay-w", cx,
             "overlay-h", cy,
             NULL);

We confirmed your observation and are checking internally now. Will provide more update when available. Thanks.

Hi James,
We have confirmed changing overlay x y w h dynamically is not supported on r24.2. For setting new overlay x y w h, the whole gst pipeline has to be restarted.

Could you share detail of your usecase? Is it obligatory to have this functionality? Or it is acceptable to restart the pipeline?

Hi James,
On r24.2, please

1. replace
   /usr/lib/aarch64-linux-gnu/tegra/libnvomx.so
   /usr/lib/aarch64-linux-gnu/gstreamer-1.0/libgstomx.so
   2 compile the test app
   gcc overlay_param_test.c -I /lib/aarch64-linux-gnu/libglib-2.0.so.0 /usr/lib/aarch64-linux-gnu/libgobject-2.0.so.0 /usr/lib/aarch64-linux-gnu/libgstreamer-1.0.so.0 -o overlay_param_test
   3 run ./overlay_param_test

You should see overlay changing dynamically on HDMI out
overlay_param_test.c (6.42 KB)
prebuilt.zip (279 KB)

Hi Danel,

Thanks for your great help. Sorry for later reply. Our main use case of playing coordinate of nvoverlaysink is resizing the display window when playback. We also have finished the playback sample code as below list for your reference. And, we have saw that you have provided the solution. We will test your solution asap. Thanks again.

#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <stdio.h>
#include <termios.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <sys/time.h>
#include <math.h>
#include <gst/gst.h>
#include
#include
using namespace std;

struct AppData
{
GstElement* pipeline;
GstElement* source;
GstElement* overlay;

void* operator new(size_t sz)
{
	void* p = malloc(sz);
	memset(p, 0, sz);
	return p;
}
void operator delete(void* p)
{
	free(p);
}

};

static void playback_pad_added(GstElement* src, GstPad* new_pad, gpointer user_data)
{
AppData* s = (AppData*)user_data;
printf(“Received new pad ‘%s’ from ‘%s’:\n”, GST_PAD_NAME(new_pad), GST_ELEMENT_NAME(src));

GstCaps* caps = gst_pad_get_current_caps(new_pad);
GstStructure* structure = gst_caps_get_structure(caps, 0);
const gchar* pad_type = gst_structure_get_name(structure);
if(g_str_has_prefix(pad_type, "video/"))
{
	if(!gst_element_link(src, s->overlay))
		printf("overlay could not be linked.\n");
}
else if(g_str_has_prefix(pad_type, "audio/"))
{
}
gst_caps_unref(caps);

}

static int play_file(const char* filename)
{
AppData* s = new AppData();

GstElement* pipeline = s->pipeline = gst_pipeline_new("pipeline");
GstElement* source = s->source = gst_element_factory_make("uridecodebin", "source");
GstElement* sink = s->overlay = gst_element_factory_make("nvoverlaysink", NULL);
if(!pipeline || !source || !sink)
{
	printf("Not all elements could be created.\n");
	return -1;
}

string uri = "file://";
uri += filename;
g_object_set(source, "uri", uri.c_str(), NULL);
g_signal_connect(source, "pad-added", G_CALLBACK(playback_pad_added), s);

gst_bin_add_many(GST_BIN(pipeline), source, sink, NULL);

if(gst_element_set_state(pipeline, GST_STATE_PLAYING) == GST_STATE_CHANGE_FAILURE)
{
	printf("Unable to set the pipeline to the playing state.\n");
	return -1;
}

//it is OK to set coordinates herei
g_object_set(sink,
	"overlay-x", 0,
	"overlay-y", 0,
	"overlay-w", 320,
	"overlay-h", 240,
	NULL);
usleep(5000000);

//it failed after playing for a while
g_object_set(sink,
	"overlay-x", 0,
	"overlay-y", 0,
	"overlay-w", 640,
	"overlay-h", 480,
	NULL);
usleep(5000000);

gst_element_set_state(s->pipeline, GST_STATE_NULL);
gst_object_unref(s->pipeline);
delete s;

return 0;

}

int main()
{
gst_init(NULL, NULL);
const char* filename = “/root/test.mp4”;
play_file(filename);
}

Hi Daniel,

Thanks for your solution. We have tested it and it was workable and correct.

Hi DaneLLL,

Is this issue fixed in R24.2.1?

Hi SagarGaonkar,
Yes, the fix is in R24.2.1.

Thanks DaneLLL

Hi DaneLLL,

I’m using R24.2.1 and changing overlay x y w h dynamically which works without any issues. But dynamic overlay-depth change is not taking effect. I have two windows and I want to dynamically change the order of these using overlay-depth. The call succeeds, but it has no effect, i.e the depth remains the same. Can you please help?

Hi zeitgeist, dynamic overlay-depth is a feature request. What is your usecase?

I have an application with two nvoverlaysink windows playing back two different files and I want to bring one in front of the other based on the user selection.

We will not have dynamic overlay-depth implemented.