FP16 support on gtx 1060 and 1080

Accelerated Computing GPU-Accelerated Libraries
1

Hello everyone,

I am a newbee with TensorRT.
I am trying to use TensorRT on my dev computer equipped with a GTX 1060.
When optimizing my caffe net with my c++ program (designed from the samples provided with the library), I get the following message

“Half2 support requested on hardware without native FP16 support, performance will be negatively affected.”

when I try to use FP16. I get the same message with a GTX 1080 on a second computer.
I am using cuda 8.0.0 and cudnn 6.

My question is :

  • is FP16 supported on these gpus ?
  • is there a minimum driver version (mine is 375.26)
  • is there a minimum cuda revision ?
  • where could I find these informations ?

Thank you for your help.

2

FP16 is supported but at a low rate. So performance won’t be interesting. The driver version you have should be fine. I would recommend using CUDA 8.0.61 (CUDA 8 GA2) which is what is currently publicly available.

The only GPUs with full-rate FP16 performance are Tesla P100, Quadro GP100, and Jetson TX1/TX2.

All GPUs with compute capability 6.1 (e.g. GTX 1050, 1060, 1070, 1080, Pascal Titan X, Titan Xp, Tesla P40, etc.) have low-rate FP16 performance. It’s not the fast path on these GPUs. All of these GPUs should support “full rate” INT8 performance, however.

2 Likes
3

Thank you for your promt reply.

I will update my CUDA 8 version, work on INT8 I will try FP16 when I will have the right card.

4

Thanks for the information. I have another question.
Does TensorRT support INT8 inference on 9X0 series?
The information shows TensorRT support GPU capability 6.1, so how do I use TendorRT on GTX980?
Thanks

5

No, INT8 is only supported on devices that support INT8, which is cc6.1 or cc7.0

To use TensorRT on GTX980, be sure to only use 32-bit float types.

6

The RTX 1080 can it be used with TF 1.7? The hardware has a TensorRT pipeline. I’m not sure if it’s useful for training or not though.

7

There is no RTX 1080

All GPUs of compute capability 3.0 and higher can be used with TF. The level of effort may vary depending on how you install or setup TF, and which version.

8

Any benchmarks for gtx 1080 vs rtx 2080? The gamers always say don’t buy the rtx cards.

9

Some relevant benchmark numbers can be found here:

[url]Puget Custom Computers: PC Hardware Articles

With a bit of Google-ing you can probably find more benchmark data. Gaming != AI. Which GPUs will be cost effective will depend on your workloads and usage patterns.

10

Thanks perfect njuffa.

11

Hello,

I have a newbie question:

May I use tensor RT in FP16 mode on new RTX 2060 / 2070 / 2080 / 2080 TI GPU’s ?
How can I deduce this information (for future cards) by myself ?

Thanks for your help :)

12

Hi,

I have a same question here…
I found a link here showing Supported hardware of FP16 and more…

Does this means that if the compute capability fit the number shows in the matrix, whatever the GPU is, it has the correspond supporting data type?

13

Thanks for your replay,

can we quantify “FP16 is supported but at a low rate” with respect to supported device with full rate.

Regards,
Rohit

14

Please see chart on https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#arithmetic-instructions

15

You can see from that table that every GPU compute capability of 5.3 or newer has “full rate” FP16 (i.e. 2x+ FP32 rate) except cc6.1 devices.