serialized engine validity

Hi folks,

Thanks for the great library. I’d like to understand the necessary (and hopefully sufficient) conditions in which an engine serialized to disk may be deserialized. For example, serializing from one video card and deserializing onto another card will not in general work* (we’ve tried and this makes sense). Likewise if we try to exchange between identical devices but different driver versions this will likely crash.

So, is it possible to list the necessary and sufficient conditions in which a model may be exchanged between devices via serialization?

Regards,
Tom

  • I believe we were able to successfully exchange models between a GTX1080 GTX1080-ti, but I may be mistaken.

Hello,

A generated TensorRT PLAN/serialized file is valid for a specific GPU — more precisely, a specific CUDA Compute Capability. For example, if you generate a PLAN for an NVIDIA P4 (compute capability 6.1) you can’t use that PLAN on an NVIDIA Tesla V100 (compute capability 7.0).

In your case, GTX1080 and GTX1080ti have the same CUDA compute capability 6.1

thanks!

How about driver versions on identical hardware? I’m pretty sure we tried sharing a serialized engine between identical hardware with different drivers and it crashed.

I’m not aware of explicit drivers dependency. If you can narrow down on the versions, please share with us.

Thanks for your responses. My colleague informs me that sharing between a GTX1080ti with 410 driver and GTX1080 with 396 driver resulted in a runtime crash.

I’m not sure it’s possible to setup different drivers for different graphics card on the same host.

indeed. they were on different hosts.

I could also imagine this happening after a driver update on the same host.

I have used hpl based on Fermi framework such as hpl-2.0_FERMI_v15.But my GPUs are GeForce GTX TITAN X,I think they are based on Maxwell’s architecture.So I want to know if NVIDIA has a hpl test program based on Maxwell architecture like hpl-2.MAXWELL ? If so, please give me a download link for it. If not, please help me find the link to hpl-2.0_FERMI_v15
In addition, I would like to ask if NVIDIA has a similar GPU performance collector similar to Intel Vtune Amplifier.Can you help me?

In addition to the Compute Capability, the TensorRT version apparently also matters (including the minor version, possibly even the build and patch number). I upgraded from TensorRT 5.0 to 5.1 and the new version could not load a PLAN file from the old version.

It would be helpful to know a complete set of invariants which allow one to say for sure whether a PLAN file is valid for a given environment.