tensorrt int8 calibration failed for a tensor is uniformly zero.

I am converting my caffe model to tensorrt int8 version.
However, I stuck in calibration phase and the program cannot create a calibration table for the error below.

Here is the error msg:
ERROR: Tensor (Unnamed ITensor* 352) is uniformly zero; network calibration failed.

I have several questions:
1.
How could I find out which layer output is the “Unnamed ITensor* 352”? I could not even check if it is really uniformly zero without knowing which layer output that has been referring to.
2.
The calibrator reads images in the same format as the engine input. Is image the only thing needed for calibration? or I need to provide labels to the calibrator? If labels are needed for calibration, how can I do this? is there any code example?
3.
I actually do not believe that there would be a tensor output uniformly zero in the network, for fp32 and fp16 version of my model can perform correctly as I expected and my model is for segmentation. If that after I checked, the tensor specified turned out not outputting uniformly zero, any ideas what could be the cause of this error and how to fix it?
4.
I tried to use 1000 calibration images for that there is an advice in the forum says lack of calibration images could be the problem. But still, increasing the number of calibration data does not solve the error. Any other advice on calibration dataset manipulation?

I encountered this error with the following environment:
GPU : nvidia GeForce RTX 2080 Ti
TRT : version 5.0.2.6
Cuda : version 10
cudnn : version 7.3

Hi,

  1. You should be able to name the layers/blobs of your network in Caffe before building your TensorRT engine - I believe this will give you a named Tensor to help debug.

  2. You only need to provide images that are representative of the data that you’ll be doing inference on. You can read more here: Developer Guide :: NVIDIA Deep Learning TensorRT Documentation

  3. There are ways to get a tensor of all zero’s. One example would be the output of a RELU layer whose inputs were all negative or zero. This can be debugged better if you can see the names of the tensors.

  4. In general, performance increases as you increase your calibration dataset size. However, I don’t think that’s relevant to the issue here.

Let me know if you can further debug after naming your layers. There may be some helpful information for you here as well:

  • https://devtalk.nvidia.com/default/topic/1046066/int8-calibration-problem
  • https://devtalk.nvidia.com/default/topic/1047780/tensorrt/int8-calibration-failing-when-one-layers-output-is-uniformly-zero/

NVIDIA Enterprise Support

Hi NVES_R,

Thanks for all the answers!

I’ve checked and found out that there is indeed a deconvolution layer that outputs uniformly zero, due to my lack of notice to the parameter setting in training phase.

Therefore, the uniformly zero output is now actually a valid output for the model.

Do you have any advice on this situation? or can I only retrain the model with proper deconvolution layer setting to prevent it from outputting all zeroes again?

Thanks again for your help!

Lewin

Hi Lewin,

I don’t believe calibration is supported when there is a uniformly zero output. So I think you’ll have to ensure that you don’t get that output as you mentioned.

I think things have changed in TensorRT 5.1.5.0 and this problematic error
“Tensor activation_83/Relu is uniformly zero; network calibration failed.”
is no more a problem. Previously my program used to crash, but now TensorRT allows you to do
inference with INT8 even if some layers are uniformly zero.

Hi NVES_R, danxpy,

Thanks for all the suggestions.

My model works after I delete the layers that have uniformly zero output.
I will definitely try TensorRT 5.1.5.0 to see if this version is capable of calibrating my original model.

I will update the result after I tried.

Thank you!

Lewin