TensorRT: Int8 calibration with hand-tuned scale factors

Autonomous Machines Jetson & Embedded Systems Jetson TX2
1

I’m looking to do int8 calibration for a particular model. The scales generated by calibration are not performing adequately. Is there a way to hand-tune the scales?

I’ve noticed that the cached calibration file contains a hex value for each layer. Can you explain what these values represent? That way it becomes possible to try to tune these values for improved model accuracy.

Thanks!

2

Hi,

Here is an useful tutorial for INT8 on TensorRT:
http://on-demand.gputechconf.com/gtc/2017/presentation/s7310-8-bit-inference-with-tensorrt.pdf

You can also check this source for calibrator information:
/usr/src/tensorrt/samples/sampleINT8/LegacyCalibrator.h

Thanks.

3

Thanks for the fast response and the additional resources! I have actually already implemented a calibration with TensorRT. I am curious if you can explain what the output values mean?

The cached calibration is a set 64-bit hex values. How are these used as scales?

4

Hmm sorry it’s actually a 32-bit hex value. Looks like it represents a float. Is this is the scale created by calibration?

5

Hi,

Please check this page for information:

Thanks.

6

Same question!