I am adding support for TensorRT models in an open source project called DonkeyCar which is a self-driving RC Car.
Here is the source code that freezes the model.
'''
Usage:
freeze_model.py --model="mymodel.h5" --output="frozen_model.pb"
Note:
This requires that TensorRT is setup correctly. For more instructions, take a look at
https://docs.nvidia.com/deeplearning/sdk/tensorrt-install-guide/index.html
'''
import os
from docopt import docopt
import json
from pathlib import Path
import tensorflow as tf
args = docopt(__doc__)
in_model = os.path.expanduser(args['--model'])
output = os.path.expanduser(args['--output'])
output_path = Path(output)
output_meta = Path('%s/%s.metadata' % (output_path.parent.as_posix(), output_path.stem))
This file has been truncated. show original
Here is the code that does the inference
https://github.com/autorope/donkeycar/blob/dev/donkeycar/parts/tensorrt.py
When testing the converted UFF model I see TensorRT perform well with some sets of input data. Here is an example where the model does well:
Pasteboard - Uploaded Image [Good TensorRT results]
However sometimes it produces very unexpected results, while the original Keras based (.h5) that the UFF model is based on performs well.
Pasteboard - Uploaded Image [Good fit based on Keras Model]
Pasteboard - Uploaded Image [Bad TensorRT UFF Model
The second graph shows that inference is quite poor. The converted model is based on the Keras model shown in the above graph.
I am using TF 1.13.1 (with TF-GPU)
CUDA 10.0
TensorRT 5.1 GA
Do you any thoughts on how I can fix this ?
NVES_R
July 15, 2019, 8:37pm
3
Hi Rahul,
Sorry for the slow response - I’m glad you figured it out.
Do you mind sharing what your problem was as a future reference for others?
Thanks,
NVIDIA Enterprise Support
The reason why I was running into this problem was the difference in input formats between Keras and TensorRT.
Keras uses WHC tuples (width, height, channels) where as TensorRT used (height, width, and channels).
The reason it worked for some data sets was because those data sets used square images (width = height).
- parser.register_input(name, (self.cfg.TARGET_D, self.cfg.TARGET_W, self.cfg.TARGET_H))
+ parser.register_input(name, (self.cfg.TARGET_D, self.cfg.TARGET_H, self.cfg.TARGET_W))
NVES_R
July 15, 2019, 8:57pm
5
Great, thanks for sharing.