Can I call inference and display module without waiting for all channel to complete decoding

Hi,

I have a problem about Deepstream in my application. Basically, I define 4 channels for my application to decode, inference and display four videos through network streams simultaneously. It works pretty well if four videos have the same frame rate.

However, the problem is both the inference and display module will wait for all video to finish decoding and pass all image to inference (with a batch size of 4). So I’m wondering if the inference task can be called individually after each decoding channel is complete without waiting for all channels to complete?

My use case is that different videos have different frame rates. So I don’t want video with lower frame rate delay the inference of video with higher frame rate.

Thanks

Hi,

Looks like you also post another topic on our TX2 board:
https://devtalk.nvidia.com/default/topic/1032410/

Could you share which platform do you use with us first?
Thanks.

Sorry for the duplication. I use a Tesla P100 GPU to run DeepStream.

To clarify the question, I implement my workflow with DecodeTask, ColorSpaceConvertorTask, and a UserDefineModule. The channel number is set to 4. I found that the input tensor for the UserDefineModule (which is also the output of the ColorSpaceConvertorTask) is usually with the batch size of 4. This means the ColorSpaceConvertorTask and UserDefineModule does not run immediately after a frame from one channel is decoded, but wait until four frames from different channels are ready in the queue, and pass them together to the following module. This operation significantly increases the latency of each channel.

So is there a way that the decoded frame can be immediately passed to ColorSpaceConvertorTask and UserDefineModule with batch size 1?

Thanks.

Can anyone help?

Hi,

If your pipelines are independent, you can try to launch four DeepStream pipelines for each input.

Thanks.

Hi,

4 pipelines are not independent. There will be some cuda memory copies between different tasks. Will it allow between different pipelines?

Hi,

Could you share more information about your use-case?
If there is some dependency among pipelines, it may need to wait for each other before going into TensorRT inference.

Thanks.