How to add color space conversion of different format?

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

I want to convert the nv12 format into bgr24 instead of RGB planar.
But the enum definition in include/module.h is:

/**

  • \enum IMAGE_FORMAT
  • \brief RGB planar or BGR planar

*/
typedef enum {
RGB_PLANAR = 0,
BGR_PLANAR = 1,
GRAY = 2
} IMAGE_FORMAT;

Can I convert the nv12 to bgr24 directly through plugins or I have to convert the RGB planar into BGR24?

2

Hi,
You can implement it via CUDA. Please refer to
https://github.com/dusty-nv/jetson-inference/blob/master/util/cuda/cudaYUV-NV12.cu

3

what about converting a float number to uint8_t number?
CUDA function like __float2uint_rn doesn’t do the work.

4

Hi,
__float2uint_rn converts float to unsigned int.

For unit8_t, you may refer to Performance of conversion byte->float vs float->byte - CUDA Programming and Performance - NVIDIA Developer Forums

5

I have the same problem since our inference task need special data layout. We change the network input data layout and train again, or need a converter to change the decoded NV12 frame to other format. Still, we need to know how does deepstream engine when and how calling each module with its each function and parameter. I’ll issue a title if we can’t predict the IO of IModule::execute function.

6

I think it is impossible or ambiguity to add custom color space converter, because we can’t set this custom module’s input. see this:

IModule *decoder = pDeviceWorker->addDecodeTask(cudaVideoCodec_H264);

the addDecodeTask just return void, then we can’t tell deepstream which module’s output is the custom module’s input, Is it allow return empty pair in IModule::getPreModule function?
I want to ask another question: Is it allowed to add more than one color space converter?

7

I’m using debugger to study when and which and what is the parameter the module’s function will be called. I known the initialize is called when the user code call IDeviceWorker::addCustomTask, and the getNbInputs is called in another thread after user code called IDeviceWorker::start. It is hard to write clean code without known these.

8

I checked the color converter result with IMAGE_FORMAT::GRAY by add a custom task after color converter, found out the output is float32 type, range from 0.0 to 255.0. I just mis-understand the IMAGE_FORMAT enumeration at beginning.

9

I missed something and I write it down here: the color space converter has two output stream tensor, the second one is the NV12 data.