Regarding the mean file used in deepstream?

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

I’m currently trying to input custom pre-trained caffe classifier models inside deepstream, the one thing I came across which confused me initially was the mean file used in deepstream and the mean file generated using caffe,

According to the following link,

https://devtalk.nvidia.com/default/topic/1066579/how-to-generate-the-mean-file-for-classifier-which-is-used-by-deepstream-/

it shows the generation of mean.ppm file which is used inside deepstream.

So I tried loading the caffe mean.binaryproto file and convert it to a mean.ppm file and use the caffe model inside deepstream. The current way how I’m trying to achieve it is as follows,

import os
import numpy as np
import matplotlib.pyplot as plt
import caffe
from PIL import Image

mean_filename='./mean.binaryproto'
proto_data = open(mean_filename, "rb").read()
a = caffe.io.caffe_pb2.BlobProto.FromString(proto_data)
mean  = caffe.io.blobproto_to_array(a)[0]

mean = np.reshape(mean, (256, 256, 3))

Image.fromarray(np.array(mean, dtype=np.uint8)).save('test.ppm')

As far as I know these files are used to substract the mean value from the images, Are they used for any other purpose.

And is my mean file conversion right in this case?

2

Can you refer to the python code:

https://devtalk.nvidia.com/default/topic/1066579/deepstream-sdk/how-to-generate-the-mean-file-for-classifier-which-is-used-by-deepstream-/post/5402720/#5402720

3

@ChrisDing, I did post that question and was following it, I never knew why you chose those values to generate the ppm file, i.e,

z=np.array([124, 117, 104])

And according to my understanding the mean file is used to subtract the mean value from the image before inferencing on that image.

However, there is also another parameter in the config file, “offsets” which does a similar thing.
Now I’m confused as to how to input the mean file which is generated in caffe i.e mean.binaryproto inside deepstream.

So I thought converting them to a ppm file was the right thing to do but I’m unsure of it.

So to summarize the questions,

  1. What is the purpose of the mean.ppm file which is used in the config?
  2. How do we use the mean.binaryproto file which is used in caffe inferencing?
  3. Why were those values chosen to begin with in the code you shared?
4

@ChrisDing anything on this?

5

we dont use mean.binaryproto file in DeepStream
mean file is similar to binaryproto where we subtract mean pixel values from every corresponding frame pixels

6

Hi ChrisDing

The ppm file, I exported through the following code,does not work properly.

import os
import numpy as np
import matplotlib.pyplot as plt
import caffe
from PIL import Image

mean_filename='./mean.binaryproto'
proto_data = open(mean_filename, "rb").read()
a = caffe.io.caffe_pb2.BlobProto.FromString(proto_data)
mean  = caffe.io.blobproto_to_array(a)[0]

mean = np.reshape(mean, (256, 256, 3))

Image.fromarray(np.array(mean, dtype=np.uint8)).save('test.ppm')
7

Can you check if your ppm file is correct?