TensorRT inference produces unexpected results

I am adding support for TensorRT models in an open source project called DonkeyCar which is a self-driving RC Car.

Here is the source code that freezes the model.

Here is the code that does the inference
https://github.com/autorope/donkeycar/blob/dev/donkeycar/parts/tensorrt.py

When testing the converted UFF model I see TensorRT perform well with some sets of input data. Here is an example where the model does well:

Pasteboard - Uploaded Image [Good TensorRT results]

However sometimes it produces very unexpected results, while the original Keras based (.h5) that the UFF model is based on performs well.

Pasteboard - Uploaded Image [Good fit based on Keras Model]
Pasteboard - Uploaded Image [Bad TensorRT UFF Model

The second graph shows that inference is quite poor. The converted model is based on the Keras model shown in the above graph.

I am using TF 1.13.1 (with TF-GPU)
CUDA 10.0
TensorRT 5.1 GA


Do you any thoughts on how I can fix this ?

I figured it out.

Hi Rahul,

Sorry for the slow response - I’m glad you figured it out.

Do you mind sharing what your problem was as a future reference for others?

Thanks,
NVIDIA Enterprise Support

The reason why I was running into this problem was the difference in input formats between Keras and TensorRT.

Keras uses WHC tuples (width, height, channels) where as TensorRT used (height, width, and channels).
The reason it worked for some data sets was because those data sets used square images (width = height).

- parser.register_input(name, (self.cfg.TARGET_D, self.cfg.TARGET_W, self.cfg.TARGET_H))
+ parser.register_input(name, (self.cfg.TARGET_D, self.cfg.TARGET_H, self.cfg.TARGET_W))

Great, thanks for sharing.