TensorRT 2.1 inference on MNIST 40000 images

Hi

I tested MNIST with TensorRT 2.1.2 and got below results:

INT8 run:400 batches of size 100 starting at 100

Top1: 0.9918, Top5: 1
Processing 40000 images averaged 0.00144474 ms/image and 0.144474 ms/batch.

FP32 run:400 batches of size 100 starting at 100

Top1: 0.9918, Top5: 1
Processing 40000 images averaged 0.00220669 ms/image and 0.220669 ms/batch.

I have some questions:

  1. Why does TensorRT runs inference on 40000 images ? Shouldn’t it run only on test dataset of MNIST i.e 10000 images ?

  2. Why there is not 3x to 4x improvement in time ms/batch ? Is it due to network size is small of MNIST ?

Can anyone please answer this ? Thanks

Sorry, I don’t have an answer for you. But you did post something I have yet to see.

MNIST/Lenet FP32: 453,167 images a second
MNIST/Lenet INT8: 692,166 images a second

What GPU are you using for this?

This is awesome to me, even though you only got a 53% speed increase. It makes sense, given they say you can get 20,000 images a second with GoogLenet.

The best I got out of LeNet was 35,000 images a second with caffe and a GTX 970.

Thanks!

Paul

I was using Nvidia GTX 1080 TI. But I am not sure if that application correctly getting those number of images. It’s probably too high