Segmentation as PGIE with Object Detection as SGIE

[dconnolly2]

Several topics discuss using object detection as PGIE and then segmentation as an SGIE. For example:

Can we use both object detection and semantic segmentation simultaneously in deepstream

Please provide complete information as applicable to your setup. • Hardware Platform (Jetson / GPU) • DeepStream Version • JetPack Version (valid for Jetson only) • TensorRT Version • NVIDIA GPU Driver Version (valid for GPU only) • Issue Type( questions, new requirements, bugs) • How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing) • Requirement details( This is for …

How to interpolate object cropping between object detection model and segmentation model？

Hardware Platform : GPU DeepStream Version : 5.1 TensorRT Version : 7.2.1 NVIDIA GPU Driver Version : 460.32.03 Issue Type(questions : new requirements Requirement details : I am using deepstream-test2 to achieve something like this: YOLOv5 model plus secondary model U-Net segmentation model; Add image cropping between pgie and sgie, crop the target of pgie, and then segment the cropped objects separately. Currently, I have connected pgie and sgie. But I don’t know how to do image segme…

We are interested in doing this the other way around: using segmentation as PGIE then doing object detection as SGIE. The goal is to only report objects that are at least partially contained within a given semantic class (e.g. only birds that are in front of the sky, or only dogs that are on the road)?

Is this possible within Deepstream? If so, are there any example available? If not, is there an alternate method to accomplish what I’ve outlined above?

Thanks,
Dan C.

[Fiona.Chen]

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

@dconnolly2

Is this possible within Deepstream?

If your segmentation model can output bboxes, it can be used as PGIE.

@dconnolly2

If so, are there any example available?

No. We don’t have such segmentation model.

@dconnolly2

If not, is there an alternate method to accomplish what I’ve outlined above?

DeepStream is only a SDK. There is no limitation for you to implement it. The only thing you need is a segmentation model which can output bboxes of your interested classes because SGIEs only work with bboxes.