Gstreamer alpha plugin does not work

Hi

I’m trying to use alpha plugin with gst-launch.

following pipeline is from official page and I only changed sink part.

gst-launch-1.0 videotestsrc pattern=snow ! mixer.sink_0 \
  videotestsrc pattern=smpte75 ! alpha method=green ! mixer.sink_1 \
  videomixer name=mixer sink_0::zorder=0 sink_1::zorder=1 ! \
  videoconvert ! video/x-raw,format=I420 ! xvimagesink

This code works fine on my host Linux machine (18.04LTS) with gstreamer 1.14.1.
But does not work on TX2 (1.8.3) and Xavier (1.14.1).

The result image should be smpte75 image which have snow pattern in the middle.
However, Jetson gives me an image with black region instead of snow pattern.

Can anyone run this pipeline successfully?

Thanks.

Hi,
I can see video mixer taking effect in running:

$ gst-launch-1.0 videotestsrc pattern=snow ! mixer.sink_0   videotestsrc pattern=smpte75 ! alpha method=green ! mixer.sink_1   videomixer name=mixer sink_0::alpha=0.7 sink_1::alpha=0.5 ! videoconvert ! video/x-raw,format=I420 ! xvimagesink

Your pipeline has 3rdparty elements only. Suggest you go to gstreamer forum to get further help.
http://gstreamer-devel.966125.n4.nabble.com/

Hi Dane,

Thank you for reply.
I’ll post it to forum later.

Before that, may I confirm that result with your pipeline is the same as below?

Hi,

Yes, it is the result of seetting ‘sink_0::alpha=0.7 sink_1::alpha=0.5’

Hi

Thank you for your reply.

FYI my result on TX2 was like this

I’ll keep digging. thank you.

Hi rary, did you solve the problem? I have the same issue with the jetson nano.
Best regards,
Wilhelm

Hi
I could not solve this.
For my application, I did like this instead.

src->mix.
src->mix.
compositor name=mix (multiple image into one image, just merging without alpha channel)
→ nvivafilter (takes composited image and alpha blending)

nvivafilter only takes 1 input so I use compositor before it.

Hi rary,
thank you for the tip with nvivafilter. I followed your suggestions, but I get an error message with and without alpha channel:
erroneous pipeline: could not link mix to nvivafilter0.
Please can you show me a part of your code?
Thank you in advance,
Wilhelm

Hi,
I cannot share my code, sorry.
But your problem seems not be relevant with a code.

erroneous pipeline: could not link mix to nvivafilter0.

This error usually indicates that caps negotiation is failed.
Please check sink and src capabilities with following command.

gst-inspect-1.0 <element_you_use>

Please share your pipeline if you could not get it work.

Hi rary,
thank you for your quick reply.
I will test it and share my pipeline later if I have no success.
Best regards,
Wilhelm

I found no solution.
With this code the right blue bar of the testpattern is black (should be transparent) and the whole pattern is transparent, see screenshot.

gst-launch-1.0 \
 videotestsrc pattern=snow ! video/x-raw, framerate=10/1, width=1920, height=1080 ! \
 alpha alpha=1 ! nvvidconv ! mix.sink_0 \
 \
 videotestsrc pattern=smpte75 ! video/x-raw, framerate=10/1, width=700, height=700 ! \
 <b>alpha method=blue </b>! nvvidconv ! mix.sink_1 \
 \
 nvcompositor name=mix sink_1::xpos=500 sink_1::ypos=100 \
 sink_1::width=600 sink_1::height=600 sink_1::alpha=0.5 sink_0::alpha=1 ! \
  nv3dsink \

Now I have added nvivafilter, but I get the above mentioned error message.

gst-launch-1.0 \
 videotestsrc pattern=snow ! video/x-raw, framerate=10/1, width=1920, height=1080 ! \
 alpha alpha=1 ! nvvidconv ! mix.sink_0 \
 \
 videotestsrc pattern=smpte75 ! video/x-raw, framerate=10/1, width=700, height=700 ! \
 alpha method=blue ! nvvidconv ! mix.sink_1 \
 \
 nvcompositor name=mix sink_1::xpos=500 sink_1::ypos=100 \
 sink_1::width=600 sink_1::height=600 sink_1::alpha=0.5 sink_0::alpha=1 ! \
  nvivafilter cuda-process=true customer-lib-name="libnvsample_cudaprocess.so" ! nv3dsink \

I do not understand nvivafilter and did also find no documentation.
rary, what is my mistake?
Thank you in advance,
Wilhelm

The same with alpha method=green and sink_1::alpha=1
Instead of the black bar in the middle, the place should be completely transparent

Hi,
Here is a sample of adding alpha blending in nvivafilter.

Or you may configure alpha in nvcompositor as demonstrated in #2.

Hi,

nvcompositor and nvivafilter cannot be connected each other without format conversion.
Please refere Gstreamer official site about src and sink, also caps negotiation.
https://gstreamer.freedesktop.org/documentation/application-development/basics/elements.html

gst-inspect-1.0 nvivafilter
gst-inspect-1.0 nvcompositor

This shows src and sink capabilities.
src caps of nvcompositor must be included in sink caps of nvivafilter to connect.

I’ve tested alpha plugin in Jetson and it seems not working.
You can write your own alpha blending plugin with nvivafilter suggested by DaneLLL.

Hi,
thank you for your suggestions.
The example in #2 does not work.
It would be very nice, if someone would inform the developers that chroma keying (alpha method=green) is not working.
Best regards,
Wilhelm

Hi,

We have verified in on r32.2.1/TX2. Which release version/Jetson platform do yo hit the failure?

Hi DaneLLL,
your code in #2 produces the video/picture shown in #3 and #5.
There is transparency, but that is not chroma keying. Finally, only the black (former green) bar must be fully transparent, see the attached picture.
Because it is working for you, I guess, we are talking about different things. Therefore a screenshot would be very helpfull.
I’ m using jetson nano with jetson-nano-sd-r32.2.1.zip
Nevertheless, thank you for your support.
Best regards,
Wilhelm

Hi DaneLLL,
I think, I misunderstood your post #16. Sorry.
Did you inform the developers?
Chroma keying will me give new features for searches with a thermal camera and a ususal vis-camera. Nvivafilter is too complex form me.
What is the normal procedure? Will they release a patch?
Best regards,
Wilhelm

Hi WiSi-Testpilot,
Please make a new post for your issue and share steps in detail.
This post is about nvcompositor and we have confirmed it working well.

Hi DaneLLL,
thank you.
I made a new thread in Jetson Nano.
[url]https://devtalk.nvidia.com/default/topic/1065511/jetson-nano/no-chroma-keying-with-gstreamer-alpha-plugin/[/url]
Best regards,
Wilhelm