Mean average_precision is 0% for all classes using Detectnet_V2

Accelerated Computing Intelligent Video Analytics TAO Toolkit
1

Hello,

I am receiving a mean average precision of 0% for all classes when training. I have attached my training file and a sample label file. Converting to tfrecords causes no issues.
000000.txt (251 Bytes)
train.txt (4.83 KB)

2

Hi kwindham,
After running tlt-dataset-convert, each class’ name will be lowercase.

Or you can check its log and find the expected class name.
“For the dataset_config in the experiment_spec, please use labels in the tfrecords file, while writing the classmap.”

So, please modify the class name to lowercase in your training spec.

3

Hi Morganh,

This worked! I converted all label file classes to lower case and the training configuration to lowercase and the mAPs are now no longer 0%. I have a new issue now with inference though, but that will be in another post. Thanks again!

Sincerely,
-kwindham

4

Hi, I’m getting zero AP / precision / recall. I’m not using tf records, just images and labels in separate training and validation folders as in the ipynb notebook in the examples on this container: Transfer Learning Toolkit for Video Streaming Analytics | NVIDIA NGC

The only thing I can see that could be a problem is that many of my images in the validation folder do not have any object of interest, so the corresponding annotation text file is an empty file. Could that be the reason? Thanks

5

Hi chandrachud,
Please move the images which have no labels outside the validation folder , then run tlt-evaluate again.
More, in default training spec of detectnet_v2, tlt-train will select the “fold 0” of tfrecords to do validation.

validation_fold:0

So, please generate tfrecords via tlt-dataset-convert firstly, and run tlt-train again, to see if you can see a non-zero mAP.

6

Hi morgan,
Thanks for the reply. I moved images with no labels outside the folder, still getting 0 everywhere.
Please note that I am not using tfrecords. As per the example jupyter notebook in your latest container v1.0.1_py2 here: Transfer Learning Toolkit for Video Streaming Analytics | NVIDIA NGC we don’t need tfrecords, just images and labels are enough.

I have only one class, and here’s what I get:

2020-01-18 11:35:06,570 [INFO] /usr/local/lib/python2.7/dist-packages/iva/faster_rcnn/scripts/test.pyc: 640/642
2020-01-18 11:35:06,742 [INFO] /usr/local/lib/python2.7/dist-packages/iva/faster_rcnn/scripts/test.pyc: Elapsed time = 0.172304868698
2020-01-18 11:35:06,750 [INFO] /usr/local/lib/python2.7/dist-packages/iva/faster_rcnn/scripts/test.pyc: 641/642
2020-01-18 11:35:06,930 [INFO] /usr/local/lib/python2.7/dist-packages/iva/faster_rcnn/scripts/test.pyc: Elapsed time = 0.179259061813

Class AP precision recall

my_class 0.0000 0.0000 0.0000

mAP = 0.0000

If you have any suggestions, please let me know. Thanks!

7

Hi chandrachud,
From your log, you were training with frcnn.
It is different from this topic.
Please create a new topic and paste your training spec there.